Rsync performance increase through buffering (cygwin/ssh solution?)

[Jim Kleckner]

Below is a description of a buffering speed improvement.
The original message with complete patch for performance can be found here:
 http://groups.google.com/groups?hl=en&lr=&ie=UTF-8&oe=UTF-8&newwindow=1&selm=at1i0e%241tj3%241%40FreeBSD.csie.NCTU.edu.tw

This may explain the horrendous performance of rsync
with ssh under cygwin.  Perhaps the way that cygwin
emulates these calls.  It is especially bad under the
circumstances mentioned here - with a large sync of data
where the files are identical (-I). 

The hang fix has been identified elsewhere in cygwin:
Check out these links:
  http://www.cygwin.com/ml/cygwin/2002-10/msg00308.html
  http://sources.redhat.com/ml/cygwin/2002-09/msg01155.html
patched rsync for hang under cygwin:
  http:// www.addr.com/~dmitry/rsync.exe

Jim

craig@atheros.com (Craig Barratt) wrote in message 
news:<at1i0e$1tj3$1@FreeBSD.csie.NCTU.edu.tw>...
 > I've been studying the read and write buffering in rsync and it turns
 > out most I/O is done just a couple of bytes at a time.  This means there
 > are lots of system calls, and also most network traffic comprises lots
 > of small packets.  The behavior is most extreme when sending/receiving
 > file deltas of identical files.
 >
 > The main case where I/O is buffered is writes from the server (when
 > io multiplexing is on). These are usually buffered in 4092 byte
 > chunks with a 4 byte header. However, reading of these packets is
 > usually unbuffered, and writes from the client are generally not
 > buffered.  For example: when receiving 1st phase checksums (6 bytes
 > per block), 2 reads are done: one of 4 bytes and one of 2 bytes,
 > meaning there are 4 system calls (select/read/select/read) per 6
 > bytes of checksum data).
 >
 > One cost of this is some performance, but a significant issue is that
 > unbuffered writes generate very short (and very many) ethernet packets,
 > which means the overhead is quite large on slow network connections.

... see this link for the rest:
http://groups.google.com/groups?hl=en&lr=&ie=UTF-8&oe=UTF-8&newwindow=1&selm=at1i0e%241tj3%241%40FreeBSD.csie.NCTU.edu.tw




--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/