[newlib-cygwin] Fix limited Internet speeds caused by inappropriate socket buffering

Corinna Vinschen corinna@sourceware.org
Fri Feb 3 20:55:00 GMT 2017


https://sourceware.org/git/gitweb.cgi?p=newlib-cygwin.git;h=609d2b22afc63d80029a4fe85e64b51c84ccae08

commit 609d2b22afc63d80029a4fe85e64b51c84ccae08
Author: Corinna Vinschen <corinna@vinschen.de>
Date:   Fri Feb 3 21:46:01 2017 +0100

    Fix limited Internet speeds caused by inappropriate socket buffering
    
    Don't set SO_RCVBUF/SO_SNDBUF to fixed values, thus disabling autotuning.
    
    Patch modeled after a patch suggestion from Daniel Havey <dhavey@gmail.com>
    in https://cygwin.com/ml/cygwin-patches/2017-q1/msg00010.html:
    
    At Windows we love what you are doing with Cygwin.  However, we have
    been getting reports from our hardware vendors that iperf is slow on
    Windows.  Iperf is of course compiled against the cygwin1.dll and we
    believe we have traced the problem down to the function fdsock in
    net.cc.  SO_RCVBUF and SO_SNDBUF are being manually set.  The comments
    indicate that the idea was to increase the buffer size, but, this code
    must have been written long ago because Windows has used autotuning
    for a very long time now.  Please do not manually set SO_RCVBUF or
    SO_SNDBUF as this will limit your internet speed.
    
    I am providing a patch, an STC and my cygcheck -svr output.  Hope we
    can fix this.  Please let me know if I can help further.
    
    Simple Test Case:
    I have a script that pings 4 times and then iperfs for 10 seconds to
    debit.k-net.fr
    
    With patch
    $ bash buffer_test.sh 178.250.209.22
    usage: bash buffer_test.sh <iperf server name>
    
    Pinging 178.250.209.22 with 32 bytes of data:
    Reply from 178.250.209.22: bytes=32 time=167ms TTL=34
    Reply from 178.250.209.22: bytes=32 time=173ms TTL=34
    Reply from 178.250.209.22: bytes=32 time=173ms TTL=34
    Reply from 178.250.209.22: bytes=32 time=169ms TTL=34
    
    Ping statistics for 178.250.209.22:
        Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
    Approximate round trip times in milli-seconds:
        Minimum = 167ms, Maximum = 173ms, Average = 170ms
    ------------------------------------------------------------
    Client connecting to 178.250.209.22, TCP port 5001
    TCP window size: 64.0 KByte (default)
    ------------------------------------------------------------
    [  3] local 10.137.196.108 port 58512 connected with 178.250.209.22 port 5001
    [ ID] Interval       Transfer     Bandwidth
    [  3]  0.0- 1.0 sec   768 KBytes  6.29 Mbits/sec
    [  3]  1.0- 2.0 sec  9.25 MBytes  77.6 Mbits/sec
    [  3]  2.0- 3.0 sec  18.0 MBytes   151 Mbits/sec
    [  3]  3.0- 4.0 sec  18.0 MBytes   151 Mbits/sec
    [  3]  4.0- 5.0 sec  18.0 MBytes   151 Mbits/sec
    [  3]  5.0- 6.0 sec  18.0 MBytes   151 Mbits/sec
    [  3]  6.0- 7.0 sec  18.0 MBytes   151 Mbits/sec
    [  3]  7.0- 8.0 sec  18.0 MBytes   151 Mbits/sec
    [  3]  8.0- 9.0 sec  18.0 MBytes   151 Mbits/sec
    [  3]  9.0-10.0 sec  18.0 MBytes   151 Mbits/sec
    [  3]  0.0-10.0 sec   154 MBytes   129 Mbits/sec
    
    Without patch:
    dahavey@DMH-DESKTOP ~
    $ bash buffer_test.sh 178.250.209.22
    
    Pinging 178.250.209.22 with 32 bytes of data:
    Reply from 178.250.209.22: bytes=32 time=168ms TTL=34
    Reply from 178.250.209.22: bytes=32 time=167ms TTL=34
    Reply from 178.250.209.22: bytes=32 time=170ms TTL=34
    Reply from 178.250.209.22: bytes=32 time=169ms TTL=34
    
    Ping statistics for 178.250.209.22:
        Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
    Approximate round trip times in milli-seconds:
        Minimum = 167ms, Maximum = 170ms, Average = 168ms
    ------------------------------------------------------------
    Client connecting to 178.250.209.22, TCP port 5001
    TCP window size:  208 KByte (default)
    ------------------------------------------------------------
    [  3] local 10.137.196.108 port 58443 connected with 178.250.209.22 port 5001
    [ ID] Interval       Transfer     Bandwidth
    [  3]  0.0- 1.0 sec   512 KBytes  4.19 Mbits/sec
    [  3]  1.0- 2.0 sec  1.50 MBytes  12.6 Mbits/sec
    [  3]  2.0- 3.0 sec  1.50 MBytes  12.6 Mbits/sec
    [  3]  3.0- 4.0 sec  1.50 MBytes  12.6 Mbits/sec
    [  3]  4.0- 5.0 sec  1.50 MBytes  12.6 Mbits/sec
    [  3]  5.0- 6.0 sec  1.50 MBytes  12.6 Mbits/sec
    [  3]  6.0- 7.0 sec  1.50 MBytes  12.6 Mbits/sec
    [  3]  7.0- 8.0 sec  1.50 MBytes  12.6 Mbits/sec
    [  3]  8.0- 9.0 sec  1.50 MBytes  12.6 Mbits/sec
    [  3]  9.0-10.0 sec  1.50 MBytes  12.6 Mbits/sec
    [  3]  0.0-10.1 sec  14.1 MBytes  11.7 Mbits/sec
    
    The output shows that the RTT from my machine to the iperf server is
    similar in both cases (about 170ms) however with the patch the
    throughput averages 129 Mbps while without the patch the throughput
    only averages 11.7 Mbps.  If we calculate the maximum throughput using
    Bandwidth = Queue/RTT we get (212992 * 8)/0.170 = 10.0231 Mbps.  This
    is just about what iperf is showing us without the patch since the
    buffer size is set to 212992 I believe that the buffer size is
    limiting the throughput.  With the patch we have no buffer limitation
    (autotuning) and can develop the full potential bandwidth on the link.
    
    If you want to duplicate the STC you will have to find an iperf server
    (I found an extreme case) that has a large enough RTT distance from
    you and try a few times.  I get varying results depending on Internet
    traffic but without the patch never exceed the limit caused by the
    buffering.
    
    Signed-off-by: Corinna Vinschen <corinna@vinschen.de>

Diff:
---
 winsup/cygwin/net.cc | 14 +++++++++++---
 1 file changed, 11 insertions(+), 3 deletions(-)

diff --git a/winsup/cygwin/net.cc b/winsup/cygwin/net.cc
index 2ff2148..fd903b1 100644
--- a/winsup/cygwin/net.cc
+++ b/winsup/cygwin/net.cc
@@ -517,8 +517,6 @@ cygwin_getprotobynumber (int number)
 bool
 fdsock (cygheap_fdmanip& fd, const device *dev, SOCKET soc)
 {
-  int size;
-
   fd = build_fh_dev (*dev);
   if (!fd.isopen ())
     return false;
@@ -607,6 +605,13 @@ fdsock (cygheap_fdmanip& fd, const device *dev, SOCKET soc)
      Mbits/sec with a 65535 send buffer. We want this to be a multiple
      of 1k, but since 64k breaks WSADuplicateSocket we use 63Kb.
 
+     NOTE 4.  Tests with iperf uncover a problem in setting the SO_RCVBUF
+     and SO_SNDBUF sizes.  Windows is using autotuning since Windows Vista.
+     Manually setting SO_RCVBUF/SO_SNDBUF disables autotuning and leads to
+     inferior send/recv performance in scenarios with larger RTTs, as is
+     basically standard when accessing the internet.  For a discussion,
+     see https://cygwin.com/ml/cygwin-patches/2017-q1/msg00010.html.
+
      (*) Maximum normal TCP window size.  Coincidence?  */
 #ifdef __x86_64__
   ((fhandler_socket *) fd)->rmem () = 212992;
@@ -615,6 +620,9 @@ fdsock (cygheap_fdmanip& fd, const device *dev, SOCKET soc)
   ((fhandler_socket *) fd)->rmem () = 64512;
   ((fhandler_socket *) fd)->wmem () = 64512;
 #endif
+#if 0 /* See NOTE 4 above. */
+  int size;
+
   if (::setsockopt (soc, SOL_SOCKET, SO_RCVBUF,
 		    (char *) &((fhandler_socket *) fd)->rmem (), sizeof (int)))
     {
@@ -633,7 +641,7 @@ fdsock (cygheap_fdmanip& fd, const device *dev, SOCKET soc)
 			(size = sizeof (int), &size)))
 	system_printf ("getsockopt(SO_SNDBUF) failed, %u", WSAGetLastError ());
     }
-
+#endif
   /* A unique ID is necessary to recognize fhandler entries which are
      duplicated by dup(2) or fork(2).  This is used in BSD flock calls
      to identify the descriptor. */



More information about the Cygwin-cvs mailing list