Cygwin Filesystem Performance degradation 1.7.5 vs 1.7.7, and methods for improving performance

Yoni Londner yonihola2@gmail.com
Wed Sep 22 05:50:00 GMT 2010


Hi,

 > I'm not exactly concerned about Linux being way faster accessing an NTFS
 > drive.  After all it's the OS itself and comes with it's own NTFS driver
 > which obviously is streamlined for typical POSIX operations.

I did not test & compare to using the Linux NTFS, rather I compared with 
Linux on VMWARE using the same Windows NTFS.SYS (via the same 
kernel32.dll APIs):

Cygwin: "C:/cygwin/bin/ls.exe /bin" -> cygwin1.dll -> kernel32.dll -> 
NTOS kernel -> NTFS.SYS driver -> HD

linux: "/bin/ls /mnt/hgfs/C/cygwin/bin" -> glibc -> linux kernel -> 
VMWARE hgfs driver -> vmware_player.exe (on Win32) ->  kernel32.dll -> 
NTOS kernel -> NTFS.SYS driver -> HD

As you can see the VMWARE path is much longer than Cygwin, and it passes 
the same APIs and NTFS.SYS driver, and yet it executes much faster.

This helps us understand that there is a lot that still can be done in 
Cygwin's filesystem performance.

Yoni


On 14/9/2010 11:38 AM, Corinna Vinschen wrote:
> On Sep 13 13:45, Yoni Londner wrote:
>> Hi,
>>
>>>> Abstract: I prepared a patch that improves Cygwin Filesystem
>>>> performance by x4 on Cygwin 1.7.5 (1.7.5 vanilla 530ms -->   1.7.5
>>>> patched 120ms). I ported the patch to 1.7.7, did tests, and found
>>>> out that 1.7.7 had a very serious 9x (!) performance degradation
>>>> from 1.7.5 (1.7.5 vanilla 530ms -->   1.7.7 vanilla 3900ms -->   1.7.7
>>>> patched 3500ms), which does makes this patch useless until the
>>>> performance degradation is fixed.
>>>
>>> The problem is, I can't reproduce such a degradation.  If I run sometimg
>>> like `time ls -l /bin>   /dev/null', the times are very slightly better
>>> with 1.7.7 than with 1.7.5 (without caching effect 1200ms vs. 1500ms,
>>> with caching effect 500ms vs. 620ms on average).  Starting with 1.7.6,
>>> Cygwin reused handles from symlink_info::check in stat() and other
>>> syscalls.  If there is such degradation under some circumstances, I need
>>> a reproducible scenario, or at least some strace output which shows at
>>> which point this happens.  Apart from actual patches this should be
>>> discussed on the cygwin-developer list.
>>>
>>
>> First of all, even your results of 1200-1500ms (1st time) and
>> 500-600ms (2nd time) is still way way way too long. On linux with an
>> NTFS mount of C:/cygwin, this took<2ms!
>>
>> And even on Win32 CMD.EXE this same operation will take you less
>> than 100ms. which is 5x to 10x faster.
>>
>> The main reason for the difference: the Windows CMD.EXE does not
>> open file handles, which make the NTFS file system to actually go
>> and read each file's first 16KB of contents (even though you did not
>> ask for it!).
>
> I'm not exactly concerned about Linux being way faster accessing an NTFS
> drive.  After all it's the OS itself and comes with it's own NTFS driver
> which obviously is streamlined for typical POSIX operations.
>
> And then there's Win32 which can go through a dir much faster as well,
> since it doesn't have to care for POSIX compatibility of the result, and
> the OS function calls coincidentally match what a cmd "dir" call needs.
>
> If you're looking for a fair comparision, why don't you look for
> Interix?  I did, and what I see is pretty much the same thing we do in
> Cygwin.  Actually, with the last Cygwin from CVS an ls -lR on a
> non-marginal directory tree is already faster than the same operation
> under Interix.
>
> That doesn't mean I won't look for more ways to enhance Cygwin's
> performance, but it won't be by adding CYGWIN environment switches
> or by neglecting correct information in stat.
>
>
> Corinna
>



More information about the Cygwin-patches mailing list