This is the mail archive of the cygwin-developers mailing list for the Cygwin project.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Re: Cygwin Filesystem Performance degradation 1.7.5 vs 1.7.7, and methods for improving performance


Hi,
I did a performnace comparison with latest version from CVS, on WinXP.

lstat(1.7.5 unpatched) 3587 files stat() 490.2ms, per file: 0.1367ms
lstat(1.7.5 patched) 3585 files stat() 78.12ms, per file: 0.02179ms
lstat(1.7.7 unpatched) 3588 files stat() 3570ms, per file: 0.9951ms
lstat(1.7.7 patched) 3588 files stat() 3374ms, per file: 0.9404ms
lstat(1.7.8 unpatched)3584 files stat() 3360ms, per file: 0.9376ms

As you can see, there is no performance improvement.

Yoni.


On 4/10/2010 12:23 PM, Corinna Vinschen wrote:
On Oct 3 23:33, Derry Shribman wrote:
I'm missing a comparison using the latest Cygwin from CVS.  That's
much more interesting than 1.7.5.

All the comparisons where done BOTH on 1.7.5 AND on 1.7.7! Here are the 1.7.7 results again:

I saw the numbers for 1.7.7 but I was talking about current CVS. Due to Yoni's mail to cygwin-patches I already changed quite a few things. A comparison should take the latest incarnation of the code into account since, obviously, development is going into this direction.

Could you compile and run it and report the results on your PCs?

Compile the latest Cygwin from CVS and do the comparison on the machines you already used. What sense does it make to add YA machine with different hardware? Especially since all my Windows machines except for a single netbook are running in VMs.

You also never replied to my mail describing the suffix problem when
using the NtQueryDirectoryFile function:
http://cygwin.com/ml/cygwin-patches/2010-q3/msg00073.html

Yoni was busy with preparing the stat performance test program, so I will answer this one:

Calling NtQueryDirectoryFile() is implemented in the kernel by
IRP_MJ_DIRECTORY_CONTROL, and RestartScan is flags |=
SL_RESTART_SCAN, ReturnSingleEntry is flags |=
SL_RETURN_SINGLE_ENTRY.

What is missing is being able to set flags |= SL_INDEX_SPECIFIED
when calling from user-mode.

The WDK description does not say that SL_INDEX_SPECIFIED provides this functionality and it also specifies that using the FileIndex does not make any sense on NTFS. Which in turn makes sense, given that NTFS keeps directories always in name sorted order.

So - as you wrote: This means that NtQueryDirectoryFile() should be
called with a 'fresh' handle every time.

Thats why in the performance test application Yoni wrote, a new
handle was called for every NtQueryDirectoryFile() - so that this
was taken into account.

That's not really the same. The test application does not take .lnk and .exe suffixes into account. It just stats what it already got from a former readdir. My private test within the Cygwin code itself was not as positive and in fact slowed down `ls -l' since now there was YA handle to open. But anyway. Maybe the method using a filter with wildcards is a winner.

We have NOD32 anti-virus installed on all XP PCs at our work place.
So that may probably be one of the reasons for the XP slowdown. Can
you run the test program on an XP at your place and see how results
differ?

Well, I can try, but it's a VM, too.


However, it's easy to make a speed comparsion which ignores the bunch of
problems Cygwin is fighting against.  What about the problems of various
filesystems, for instance?

Cygwin anyway today caches the volume type. So it can know whether it is NTFS/FAT without a system call.

No, not really. You can make this point for the drive which hosts the systemroot, but every other drive (or junction) can change at any given time.

This requires at least one open handle to the volume to be sure to have
the right filesystem type.  So, whatever you do, you need to open a
handle first.  Either to the parent dir or to the file.  This is kind of
a chicken-egg problem.

This is exactly what Yoni cached in the patch he submitted to cygwin-patches!

I didn't look into the patch for the well-known reason. The description in Yoni's mail implies that the caching is based on the path, just like in 1.5.x. That's correct only in an environment which you control yourself, but it's not necessarily correct for any other environment. That's why I added a new caching algorithm to current CVS which is based on the results of a single FileFsVolumeInformation result. This still requires an open handle, either of the file itself, or of it's parent dir.


Corinna




Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]