rsync performance degradation? Could be a windows issue?
Thu Mar 31 20:49:00 GMT 2016
I'm not sure this is on-topic, but at least I'm in cygwin when I
notice the below:
I do a lot of large data transfers between USB drives. Often I get
great speeds (70MB/sec or more).
Sometimes it falls way off to closer to 20MB/sec with the same class
I experienced the "slow" transfer speed today, so I thought I'd ask if
anyone knew a way to resolve it?
The PC I'm running on is spec'ed for data processing so a 6 core (12
thread) setup with 128GB or RAM.
No significant data processing is going on. Basically the machine is
currently dedicated to running rsync (as below).
I have 2 USB drives hooked up. Both are USB-3 and should be able to
do 70MB/sec with no issue.
Neither drive has any unrelated I/O going on that I know of. (I don't
see any in the Windows Performance Monitor.)
I have a folder with almost 200 4GB files ( 750GB total and all the
files are large).
I'm using "rsync -av" to copy files from the source to the destination
and the folder on the destination started out empty
Using Windows "Resource Monitor" I normally would see both drives
hitting over 70 MB/sec. Today I'm seeing the source drive jump all
the way up to 125 MB/sec or so, but the destination is down around 20
When the 4GB source file completes reading, the source drive basically
goes idle for a while until the file is written out.
Then the source drive jumps back to 125 MB/sec until it finishes
reading the next file. The destination drive on the other hand is
stuck down around 20 MB/sec.
I really don't think this is caused by slow hardware. Could the
destination drive be dropping down to USB-2 mode?
And now that I've written up the above, the system has returned to a
more normal 75 MB/sec consistently on the output drive.
I'm still curious what might be going on. As I say I move a lot of
data around as part of my work, and much of the time it is big groups
of either 1.5GB or 4GB files.
Problem reports: http://cygwin.com/problems.html
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
More information about the Cygwin