Bug? UDP packet transmitted via sendto() intermitently not being transmitted to network

Estabrook, Edward Edward.Estabrook@Polycom.com
Fri Dec 10 22:37:00 GMT 2010


Hello,
I have a fairy complicated SIPp scenario running in cygwin-1.7.7(0.230/5/3) on Windows XP which is losing a single UDP packet intermittently.  At this point I am fairly confident SIPp is not the culprit and that the packet is being dropped 'somewhere' between the sendto() call and wireshark / being transmitted to the interface. The same test case running on Linux (Fedora 9) works consistently.  

I have shown (via debugger and printf statements) that the missing packet is being transmitted via the sendto() system call with the appropriate message buffer, size and socket information and yet is not being placed on the wire.  Indeed detailed analysis of SIPp execution shows that the sequence of steps within SIPp is identical in the working and failing case.  As a result I am now concerned that there may be something amiss with the UDP packet processing layer in cygwin.  Within the same SIPp scenario, it is always the same packet that is lost if one is lost, but the loss occurs only about 50% of the time it is executed.  The packet is a SIP 200 OK response which is supposed to be transmitted after the script has received (and ignored) ~45 UDP REGISTER messages spread out over ~3 minutes.  

I will explain the symptom in more detail a little further on in this e-mail.

Generally I am seeking information regarding:
1) Where can I download and install an older version of Cygwin (say 1.5 era) to test if the problem is newly introduced?
2) What tools are available to measure potential packet loss within cygwin itself? (the Windows stack is not reporting dropped UDP packets, and the packets are not even making it into a locally running instance of wireshark).  Is there a way to dump / access / sniff the inputs and outputs to the cygwin IP stack?  Traffic volume is low so verbosity would not pose a problem.
3) Is there additional documentation regarding the cygwin IP translation layer?  I see in the release notes that various enhancements have been made recently, and I cannot help but worry that perhaps an obscure issue in this aggregation of packets for delivery to Windows is introducing a problem.  Is it possible to shut this off and simply force each packet to be delivered without additional buffering?

Unfortunately the only steps to reproduce (that I am yet aware of) is a 50-step sequence of SIP signaling which would be difficult to detail sufficiently here.  I will cover some pertinent details below.

The Windows XP system is configured with several IPs associated with an interface.  Two parallel instances of SIPp are started, each associated with a different IP (but using the same MAC address).  SIPp is simulating two SIP proxies while a SIP device sends and receives traffic.  The traffic represented by the scenario is ~50 packets over 3 minutes (though the interface is in promiscuous mode).  I have included the scenario details below.  It is always the packet marked  '48  <---------- 200(11)' which is dropped, but dropping only happens about ½ the time.  At this point I am fairly confident SIPp is not the culprit and that the packet is being dropped 'somewhere' between sendto() and wireshark / the interface.

The code:
DEBUG("sendto(%d, buffer, %d, 0, %s:%hu [&=%p], %d)", 
socket->ss_fd, len, inet_ntoa( ((struct sockaddr_in*)dest)->sin_addr ), 
ntohs(((struct sockaddr_in*)dest)->sin_port), 
dest, 
SOCK_ADDR_SIZE(dest));
rc = sendto(socket->ss_fd, buffer, len, 0, (struct sockaddr *)dest, SOCK_ADDR_SIZE(dest));
DEBUG_OUT("return %d", rc);

Produces this debug output:
socket_write_primitive() in sipp.cpp:2484 - sendto(7, buffer, 491, 0, 172.23.79.237:5060 [&=0x6fc8b8], 16)
socket_write_primitive() in sipp.cpp:2492 - (Leaving) - return 491
and I have verified that the buffer contents are correct.  The packet, however, does not appear in wireshark or the network.

Milliseconds later SIPp executes the same code path to generate a SUBSCRIBE message (step 49 below) and this packet is transmitted.  

All suggestions will be much appreciated!
                                Messages  Retrans   Timeout   Unexpected-Msg
1  ----------> REGISTER(1 )           1         0         0         0        
  <---------- 200(1 )                1         0                            
  <---------- SUBSCRIBE(2 )          1         0                            
  ----------> 200(2 )                1         0         0         0        
  ----------> NOTIFY(2 )             1         0         0         0        
  <---------- 200(2 )                1         0                            
  ----------> SUBSCRIBE(3 )          1         0         0         0        
  <---------- 202(3 )                1         0                            
  <---------- NOTIFY(3 )             1         0                            
10  ----------> 200(3 )                1         0         0         0        

  ----------> REGISTER(1 )           1         0         0         0        
  <---------- 200(1 )                1         0                            
              [ NOP ]              
  ----------> NOTIFY(2 )             1         0         0         0        
  <---------- 200(2 )                1         0                            
              [ NOP ]              
  ----------> INVITE(4 )             1         0         0         0        

  <---------- 183(4 )                1         0                            
  ----------> NOTIFY(2 )             1         0         0         0        
20  <---------- 200(2 )                1         0                            
  <---------- 200(4 )                1         0                            
  ----------> ACK(4 )                1         0         0         0        
  ----------> NOTIFY(2 )             1         0         0         0        
  <---------- 200(2 )                1         0                            
  [   2000ms] Pause              1                             0        
              [ NOP ]              
  ----------> INVITE(4 )             1         0         0         0        
  ----------> INVITE(4 )             1         0         0         0        
  ----------> INVITE(4 )             1         0         0         0        
30  ----------> INVITE(4 )             1         0         0         0        
  ----------> INVITE(4 )             1         0         0         0        
  ----------> INVITE(4 )             1         0         0         0        
  ----------> INVITE(4 )             1         0         0         0        

  ----------> REGISTER(1 )           1         0         0         0        
  ----------> REGISTER(1 )           1         0         0         0        
  ----------> REGISTER(1 )           1         0         0         0        
  ----------> REGISTER(1 )           1         0         0         0        
  ----------> REGISTER(1 )           1         0         0         0        
  ----------> REGISTER(1 )           1         0         0         0        
40  ----------> REGISTER(1 )           1         0         0         0        
41  ----------> REGISTER(1 )           1         0         0         0        
42  ----------> REGISTER(1 )           1         0         0         0        
43  ----------> REGISTER(1 )           1         0         0         0        
44  ----------> REGISTER(7 )           1         0         0         0        29961878-68089d97-40475bea@172.23.79.237
45  ----------> REGISTER(7 )           1         0         0         0        
46  ----------> REGISTER(7 )           1         0         0         0        29961878-68089d97-40475bea@172.23.79.237
47  ----------> REGISTER(11)           1         0         0         0        f63768a9-a13cc854-432ed2c3@172.23.79.237
48  <---------- 200(11)                1         0                            
49  <---------- SUBSCRIBE(12)          1         0                            
50  ----------> 200(12)                0         0         0         1        
  ----------> NOTIFY(12)             0         0         0         0        
  <---------- 200(12)                0         0                            
  ----------> SUBSCRIBE(13)          0         0         0         0        
  <---------- 202(13)                0         0                            
  <---------- NOTIFY(13)             0         0                            
  ----------> 200(13)                0         0         0         0        

  [   1000ms] Pause              0                             0        

              [ NOP ]              
  ----------> NOTIFY(12)             0         0         0         0        
  <---------- 200(12)                0         0                            
              [ NOP ]              
  ----------> INVITE(14)             0         0         0         0        

  <---------- 183(14)                0         0                            
  ----------> NOTIFY(12)             0         0         0         0        
  <---------- 200(12)                0         0                            
  <---------- 200(14)                0         0                            
  ----------> ACK(14)                0         0         0         0        
  ----------> NOTIFY(12)             0         0         0         0        
  <---------- 200(12)                0         0                            
  ----------> REGISTER(11)           0         0         0         0        
  <---------- 200(11)                0         0                            
  ----------> REGISTER(11)           0         0         0         0        
  <---------- 200(11)                0         0                            
------- Waiting for active calls to end. Press [q] again to force exit. -------

Best Regards,
Ed

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple



More information about the Cygwin mailing list