Cygwin Filesystem Performance degradation 1.7.5 vs 1.7.7, and methods for improving performance

[Yoni Londner]

Hi,

The caching-speed up is trivial:
We store the the FileFullDirectoryInformation fields, and if any of them 
change - we re-read the file.

Its not (in practical life) possible to change a file without causing a 
modification on 
FileIndex/CreationTime/LastWriteTime/ChangeTime/EndOfFile/AllocationSize/FileAttributes/FileName/EaSize!

 From the MSDN we see the info we can get on a 
FileFullDirectoryInformation request:

http://msdn.microsoft.com/en-us/library/cc232068(PROT.10).aspx

NextEntryOffset (4 bytes): A 32-bit unsigned integer that contains the 
byte offset from the beginning of this entry, at which the next 
FILE_FULL_DIR_INFORMATION entry is located, if multiple entries are 
present in a buffer. This member is zero if no other entries follow this 
one. An implementation MUST use this value to determine the location of 
the next entry (if multiple entries are present in a buffer).

FileIndex (4 bytes): A 32-bit unsigned integer that contains the byte 
offset of the file within the parent directory. For file systems such as 
NTFS, in which the position of a file within the parent directory is not 
fixed and can be changed at any time to maintain sort order, this field 
SHOULD be set to 0, and MUST be ignored.<79>

CreationTime (8 bytes): The time when the file was created; see section 
2.1.1. This value MUST be greater than or equal to 0.

LastAccessTime (8 bytes): The last time the file was accessed; see 
section 2.1.1. This value MUST be greater than or equal to 0.

LastWriteTime (8 bytes): The last time information was written to the 
file; see section 2.1.1. This value MUST be greater than or equal to 0.

ChangeTime (8 bytes): The last time the file was changed; see section 
2.1.1. This value MUST be greater than or equal to 0.

EndOfFile (8 bytes): A 64-bit signed integer that contains the absolute 
new end-of-file position as a byte offset from the start of the file. 
EndOfFile specifies the offset to the byte immediately following the 
last valid byte in the file. Because this value is zero-based, it 
actually refers to the first free byte in the file. That is, it is the 
offset from the beginning of the file at which new bytes appended to the 
file will be written. The value of this field MUST be greater than or 
equal to 0.

AllocationSize (8 bytes): A 64-bit signed integer that contains the file 
allocation size, in bytes. The value of this field MUST be an integer 
multiple of the cluster size.

FileAttributes (4 bytes): A 32-bit unsigned integer that contains the 
file attributes. For a list of valid file attributes, see section 2.6.

FileNameLength (4 bytes): A 32-bit unsigned integer that specifies the 
length, in bytes, of the file name contained within the FileName member.

EaSize (4 bytes): A 32-bit unsigned integer that contains the combined 
length, in bytes, of the extended attributes (EA) for the file.

FileName (variable): A sequence of Unicode characters containing the 
file name. When working with this field, use FileNameLength to determine 
the length of the file name rather than assuming the presence of a 
trailing null delimiter. Dot directory names are valid for this field. 
For more details, see section 2.1.5.1.

Yoni

On 10/9/2010 9:39 PM, Corinna Vinschen wrote:
> On Sep 10 13:23, Christopher Faylor wrote:
>> On Fri, Sep 10, 2010 at 05:08:40PM +0200, Corinna Vinschen wrote:
>>> What I'm still mulling over is a good idea to re-use the results of a
>>> former call to readdir in a stat call.  One problem here is to make sure
>>> that a subsequent stat call is *really* accessing the same file as the
>>> former readdir returned.
>>
>> I've considered that before but you really can't cheaply determine that
>> the file hasn't changed without going to the OS.  And, then it isn't cheap.
>
> Yes, that's what it always comes down to.  That's why I'm considering to
> restrict this speedup to fstatat.  I wrote more about this on the
> cygwin-developers list.
>
>
> Corinna
>