Cygwin Filesystem Performance degradation 1.7.5 vs 1.7.7, and methods for improving performance

Derry Shribman
Tue Oct 5 14:29:00 GMT 2010


 > I saw the numbers for 1.7.7 but I was talking about current CVS.

Yoni is currently busy working on a "cygwin kernel-mode accelerator" prototype. 
Hopefully by tomorrow he will be able to post it, and also test the performance 
numbers on the current CVS cygwin1.dll.


On 10/4/2010 12:23 PM, Corinna Vinschen wrote:
> On Oct  3 23:33, Derry Shribman wrote:
>>> I'm missing a comparison using the latest Cygwin from CVS.  That's
>>> much more interesting than 1.7.5.
>> All the comparisons where done BOTH on 1.7.5 AND on 1.7.7!
>> Here are the 1.7.7 results again:
> I saw the numbers for 1.7.7 but I was talking about current CVS.  Due to
> Yoni's mail to cygwin-patches I already changed quite a few things.  A
> comparison should take the latest incarnation of the code into account
> since, obviously, development is going into this direction.
>> Could you compile and run it and report the results on your PCs?
> Compile the latest Cygwin from CVS and do the comparison on the machines
> you already used.  What sense does it make to add YA machine with
> different hardware?  Especially since all my Windows machines except for
> a single netbook are running in VMs.
>>> You also never replied to my mail describing the suffix problem when
>>> using the NtQueryDirectoryFile function:
>> Yoni was busy with preparing the stat performance test program, so I
>> will answer this one:
>> Calling NtQueryDirectoryFile() is implemented in the kernel by
>> IRP_MJ_DIRECTORY_CONTROL, and RestartScan is flags |=
>> SL_RESTART_SCAN, ReturnSingleEntry is flags |=
>> What is missing is being able to set flags |= SL_INDEX_SPECIFIED
>> when calling from user-mode.
> The WDK description does not say that SL_INDEX_SPECIFIED provides
> this functionality and it also specifies that using the FileIndex
> does not make any sense on NTFS.  Which in turn makes sense, given
> that NTFS keeps directories always in name sorted order.
>> So - as you wrote: This means that NtQueryDirectoryFile() should be
>> called with a 'fresh' handle every time.
>> Thats why in the performance test application Yoni wrote, a new
>> handle was called for every NtQueryDirectoryFile() - so that this
>> was taken into account.
> That's not really the same.  The test application does not take .lnk and
> .exe suffixes into account.  It just stats what it already got from a
> former readdir.  My private test within the Cygwin code itself was not
> as positive and in fact slowed down `ls -l' since now there was YA
> handle to open.  But anyway.  Maybe the method using a filter with
> wildcards is a winner.
>> We have NOD32 anti-virus installed on all XP PCs at our work place.
>> So that may probably be one of the reasons for the XP slowdown. Can
>> you run the test program on an XP at your place and see how results
>> differ?
> Well, I can try, but it's a VM, too.
>>> However, it's easy to make a speed comparsion which ignores the bunch of
>>> problems Cygwin is fighting against.  What about the problems of various
>>> filesystems, for instance?
>> Cygwin anyway today caches the volume type. So it can know whether
>> it is NTFS/FAT without a system call.
> No, not really.  You can make this point for the drive which hosts the
> systemroot, but every other drive (or junction) can change at any given
> time.
>>> This requires at least one open handle to the volume to be sure to have
>>> the right filesystem type.  So, whatever you do, you need to open a
>>> handle first.  Either to the parent dir or to the file.  This is kind of
>>> a chicken-egg problem.
>> This is exactly what Yoni cached in the patch he submitted to cygwin-patches!
> I didn't look into the patch for the well-known reason.  The description
> in Yoni's mail implies that the caching is based on the path, just like
> in 1.5.x.  That's correct only in an environment which you control
> yourself, but it's not necessarily correct for any other environment.
> That's why I added a new caching algorithm to current CVS which is based
> on the results of a single FileFsVolumeInformation result.  This still
> requires an open handle, either of the file itself, or of it's parent
> dir.
> Corinna

More information about the Cygwin-developers mailing list