This is the mail archive of the cygwin-patches mailing list for the Cygwin project.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Re: Fix performance on 10Gb networks


Hi Iuliu,

On Nov 18 19:30, Iuliu Rus wrote:
> Hello,
> Google is running Cygwin apps on its 10Gb networks and we are seeing
> extremely bad performance in a couple of cases. For example, iperf
> with the defaults results in only 10Mbits/sec.
> We tracked this down to a combination of non-blocking sockets with
> Nagle+delayed ack kicking in, since the apps eventually end up sending
> a very small packets (2 bytes).
> We have a case open against Microsoft but since everything is moving
> very slow we would like to work around by picking socket buffers that
> are multiple of 4k.

Thanks for the patch.  One question:

> Change log:
> 2014-11-18 Iuliu Rus <rus.iuliu@gmail.com>
> 
> * net.cc Change default values for socket buffers to fix performance
> on 10Gb networks.
> 
> Index: winsup/cygwin/net.cc
> ===================================================================
> RCS file: /cvs/src/src/winsup/cygwin/net.cc,v
> retrieving revision 1.320
> diff -u -p -r1.320 net.cc
> --- winsup/cygwin/net.cc	13 Oct 2014 08:18:18 -0000	1.320
> +++ winsup/cygwin/net.cc	18 Nov 2014 19:12:00 -0000
> @@ -621,13 +621,16 @@ fdsock (cygheap_fdmanip& fd, const devic
>       this is no problem on 64 bit.  So we set the default buffer size to
>       the default values in current 3.x Linux versions.
>  
> -     (*) Maximum normal TCP window size.  Coincidence?  */
> +     (*) Maximum normal TCP window size.  Coincidence?  
> +
> +     NOTE 3. Setting the window size to 65535 results in extremely
> bad performance for apps that send data in multiples of Kb, as they
> eventually end up sending 1 byte on the network and naggle + delay ack
> kicks in. For example, iperf on a 10Gb network gives only 10 Mbits/sec
> with a 65535 send buffer. We want this to be a multiple of PAGE_SIZE,
> but since 64k breaks WSADuplicateSocket we use 60Kb.

We do?  See below.

> +*/
>  #ifdef __x86_64__
>    ((fhandler_socket *) fd)->rmem () = 212992;
>    ((fhandler_socket *) fd)->wmem () = 212992;
>  #else
> -  ((fhandler_socket *) fd)->rmem () = 65535;
> -  ((fhandler_socket *) fd)->wmem () = 65535;
> +  ((fhandler_socket *) fd)->rmem () = 63488;
> +  ((fhandler_socket *) fd)->wmem () = 63488;

This is 62K, certainly not a multiple of the native PAGE_SIZE of 4K.
And this makes me wonder.  Did you intend to use 60K and ended up with
62K for a reason?  And then, why not 63K as a multiple of 1K?


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

Attachment: pgpxBPcZH5CHs.pgp
Description: PGP signature


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]