Re: Why does (stat() ?) open files ?

[Corinna Vinschen <corinna-cygwin at cygwin dot com>]

On Apr  9 14:12, Ben RUBSON wrote:
> On 09 Apr 2018 12:52, Corinna Vinschen wrote:
> 
> > On Apr  9 12:28, Ben RUBSON wrote:
> > > Hi,
> > > 
> > > This follows the "Why does readdir() open files ?" discussion we had a few
> > > days ago.
> > > Thank you Corinna for your answers and suport there !
> > > 
> > > So, context is Cygwin, especially rsync, working over a Fuse FS.
> > > This Fuse FS is assumed to be mounted on `/cygdrive/x` below.
> > > 
> > > I finally found that readdir() does not open every file.
> > > `ls /cygdrive/x` does not fire any open() call. Perfect.
> > > 
> > > However, `ls -l /cygdrive/x` does, every file is opened, with read access.
> > > As `rsync -an /cygdrive/x /tmp/`, which is a dry-run just grabbing files'
> > > attributes.
> > > 
> > > I then went through Cygwin code and found that
> > > NtCreateFile/NtOpenFile calls
> > > from symlink_info::check() in path.cc may be the culprits.
> > > To demonstrate this I added write access to these calls, and found that
> > > every file was then opened with write access.
> > > 
> > > Later in this function we have a failback to NtQueryDirectoryFile call.
> > > I assume (assume only, I may be wrong) this one does not open the
> > > requested
> > > file.
> > 
> > It's nice that you're testing all this, but you should ask *why* Cygwin
> > does it in the first place.  The reason is that the information one can
> > gather without opening the file on Windows is insufficient to fill in
> > all of the stat struct.  The directory info returned by
> > NtQueryDirectoryFile just isn't, thus it's only a fallback.
> 
> Corinna, thank you very much for your answer.
> What info would be missing without opening the file ?

uid, gid, number of links.

> Do you know where the open call could come from, when only using
> NtQueryDirectoryFile in symlink_info::check() ?
> (certainly related to the previous question)

The answer here is that your FS should handle the open call differently
depending on the access mask.  The NtCreateFile call in symlink_info::check
opens the file with READ_CONTROL | FILE_READ_ATTRIBUTES | FILE_READ_EA
only.  All these access flags only request *meta* info, not actual data
from the data stream of the file.  In other words, a Windows open call
without FILE_READ_DATA/FILE_WRITE_DATA and related flags does not
actually have to open the file at all in the FS driver.   It only has to
provide metadata subsequently, an operation which you usally can have at
much lower cost if the remote FS is running on a POSIX OS:

- NtCreateFile called with only metadata access flags does not have to
  open the file.

- Make sure to ignore the EaBuffer and EaLength parameter, rather than
  to return a failure (this avoids YA NtOpenFile call).

- Just call stat/statvfs on the remote file as required to fulfill
  subsequent NtQueryVolumeInformationFile and NtQueryInformationFile
  calls.

Does that make sense?


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

Attachment: signature.asc
Description: PGP signature