This is the mail archive of the
cygwin-developers
mailing list for the Cygwin project.
Re: Why does (stat() ?) open files ?
On 09 Apr 2018 15:07, Corinna Vinschen wrote:
On Apr 9 14:12, Ben RUBSON wrote:
On 09 Apr 2018 12:52, Corinna Vinschen wrote:
On Apr 9 12:28, Ben RUBSON wrote:
Hi,
This follows the "Why does readdir() open files ?" discussion we had a
few
days ago.
Thank you Corinna for your answers and suport there !
So, context is Cygwin, especially rsync, working over a Fuse FS.
This Fuse FS is assumed to be mounted on `/cygdrive/x` below.
I finally found that readdir() does not open every file.
`ls /cygdrive/x` does not fire any open() call. Perfect.
However, `ls -l /cygdrive/x` does, every file is opened, with read
access.
As `rsync -an /cygdrive/x /tmp/`, which is a dry-run just grabbing
files'
attributes.
I then went through Cygwin code and found that
NtCreateFile/NtOpenFile calls
from symlink_info::check() in path.cc may be the culprits.
To demonstrate this I added write access to these calls, and found that
every file was then opened with write access.
Later in this function we have a failback to NtQueryDirectoryFile call.
I assume (assume only, I may be wrong) this one does not open the
requested
file.
It's nice that you're testing all this, but you should ask *why* Cygwin
does it in the first place. The reason is that the information one can
gather without opening the file on Windows is insufficient to fill in
all of the stat struct. The directory info returned by
NtQueryDirectoryFile just isn't, thus it's only a fallback.
Corinna, thank you very much for your answer.
What info would be missing without opening the file ?
uid, gid, number of links.
If we use "noacl" mount option, I think uid and gid do not really make
sense and could be forced to some default value ?
Number of links should not be critical too.
Thus gathering files' information without opening them could be possible.
However sounds like a more correct way to handle this is in your answer
below, though I'm not sure it will be possible.
Do you know where the open call could come from, when only using
NtQueryDirectoryFile in symlink_info::check() ?
(certainly related to the previous question)
The answer here is that your FS should handle the open call differently
depending on the access mask. The NtCreateFile call in symlink_info::check
opens the file with READ_CONTROL | FILE_READ_ATTRIBUTES | FILE_READ_EA
only. All these access flags only request *meta* info, not actual data
from the data stream of the file. In other words, a Windows open call
without FILE_READ_DATA/FILE_WRITE_DATA and related flags does not
actually have to open the file at all in the FS driver. It only has to
provide metadata subsequently, an operation which you usally can have at
much lower cost if the remote FS is running on a POSIX OS:
- NtCreateFile called with only metadata access flags does not have to
open the file.
- Make sure to ignore the EaBuffer and EaLength parameter, rather than
to return a failure (this avoids YA NtOpenFile call).
- Just call stat/statvfs on the remote file as required to fulfill
subsequent NtQueryVolumeInformationFile and NtQueryInformationFile
calls.
Does that make sense?
Thank you very much for your detailed answer Corinna !
Yes it does make sense, at least I understand how it should work.
The FS is not really running on a POSIX OS, as it stands on the Windows
machine using WinFsp FUSE API.
I'll see with WinFsp team if something can be done in this way.
Thank you !
Ben