stat() and tilde prefix (was bad bash tab completion)

[Corinna Vinschen]

On Jan 14 01:17, Christopher Faylor wrote:
> On Mon, Jan 14, 2013 at 04:21:25PM +1100, Shaddy Baddah wrote:
> >In investigating this, I believe the issue I am having is due to how
> >stat() handles tilde prefixed paths. On linux we see:
> >
> >linux$ $ python -c 'import os; print os.stat("~/..")'
> >Traceback (most recent call last):
> >   File "<string>", line 1, in <module>
> >OSError: [Errno 2] No such file or directory: '~/..'
> >
> >and on cygwin we see:
> >
> >cygwin$ python -c 'import os; print os.stat("~/..")'
> >posix.stat_result(st_mode=16832, st_ino=562949953496729L, 
> >st_dev=4174909669L, st_nlink=1, st_uid=42037, st_gid=10513, st_size=0L, 
> >st_atime=1357616166, st_mtime=1357616166, st_ctime=1357616166)
> 
> It is a bug.  It's not just "~".  Any nonexistent directory will
> work, like "foo/..".

And it's a bug which isn't easily fixed.  Since about the dawn of time,
Cygwin's core path handling evaluates the path in a non-POSIX manner,
mainly for performance reasons.

POSIX demands to evaluate a path from left to right (thus tripping over
the non-existant "~" or "foo" directory).  Windows OTOH skips testing
all parent directories of a path, and while this can be changed(*), it's
the default setting since the earliest Windows NT versions.

So, since Cygwin can't rely on the OS to this job when it has to convert
a POSIX path to a Windows path internally, Cygwin would have to check
the existence of every single path component from left to right to
emulate the POSIX requirements.  But that would be a big performance
hit, so Cygwin's path handling code tries to be clever to avoid having
to call too many OS functions:

The first step of converting a POSIX path to a Windows path is to
normalize the path.  "." and ".." components are simply dropped:

  "a/b/./c"  -> "a\b\c"
  "a/b/../c" -> "a\c"

Then the path prefix is replaced by the matching mount point.

Eventually it evaluates the path from right to left.  Consider a valid,
normalized path with 10 components.  Under POSIX rules this requires 10
checks for existence.  No problem for the Linux kernel since it has
everything under control anyway and the test is blazingly fast.

But Cygwin is not the OS so it has to call the necessary OS functions 10
times.  By checking from right to left, Cygwin has to call the OS
functions only once, if the file exists, two times if the file does not
exist, but its parent dir exists, and so on.  On top of that, the entire
chore has to restart when tripping over a symbolic link.

Since the predominant number of file operations are performed on
existing paths, or at least paths for which the parent dir exists,
Cygwin reduces the number of OS operations to convert a POSIX to a
Windows path.  The price we're paying is this very deviation from the
POSIX standard.


Corinna

(*) User right "Bypass Traverse Checking", by default enabled for
    everyone.

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Project Co-Leader          cygwin AT cygwin DOT com
Red Hat

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple