Performance problems

[Linda W]

Christopher Faylor wrote:

>On Thu, 2 Jun 2005, Linda W wrote:
>
>>In tracing the Win32 file operations, find seems to perform multiple
>>file open operations for each file processed.  One way to speed up
>>operations in this area might be to keep a "cache" of the last "N" file
>>handles.  I suspect it's just the Windows path lookup mechanism being
>>slow to reopen things.  But if the cygwin.dll could cache even the past
>>5 entries, it might speed things up significantly.  If it is opened
>>each time to read different information, it might be much cheaper to
>>collect all the information at one time and cache it in an internal
>>"inode cache" that could expire in a second or so.  If it would "slow"
>>down other programs, it could have some smarts in the system calls to
>>look for calling patterns from programs like find that need a couple or
>>more openings to fully "process a file", that all happen within a few
>>milliseconds of each other.
>>
>Oddly enough, Corinna and I have been discussing the possibility of
>caching opendir/readdir data for subsequent use in stat().  She's for it
>and I'm mildly agin' it.
>  
>
>I think that introducing caching opens the door to all sorts of subtle
>race conditions since only the OS can maintain cache coherency.
>  
>
---
    You are technically accurate, but the cygwin layer is a POSIX
complient-OS emulation layer by some definition, no?

I wouldn't cache data without keeping the associated handles to the
corresponding file objects open.  As long as they are kept open,
Windows would disallow things like deleting the file and replacing
it with a directory.  That should control most race conditions
with some degree of relative safety.

>She thinks that the benefits would outweigh the tiny possibility of bad
>cache data resulting from something like performing an "ls" on a file
>and having, e.g., some other process sneak in, remove the file and
>introduce a directory, but still having "ls" report file data.
>  
>
---
    Isn't this already a problem on networked shares?  I.e.
doesn't Windows cache file info from network shares for a few
seconds (maybe more if one has local-file caching turned on).

>I thought I should mention this in the off chance that Corinna actually
>does implement something just so that history records that this is
>something that Corinna has been considering for a while.
>  
>
---
Ditto.  It's a logical, technical, solution that shouldn't (theory)
have too many race type problems on NT. 

As far as how history records it, um, why do you feel a need
to make such statements for "historical record?  It's not,
exactly, rocket science nor is it a "new" suggestion
(http://sourceware.org/ml/cygwin/2003-11/msg00007.html).

Is it that you just want to prove your point that [tense
modified]:

 > people comment on slowness and speculate on the cause without
 > spending any time to actually figure out what the cause might be.

and that no mere "user" would be of value
("be a hero"[sic]) and "figure out a way to improve  cygwin's 'slowness' "?

Sorry if I misinterpret the meaning of your words, but
if I understand this, you are attempting to emphasize, that, if
this idea makes it into code, it would NOT have been due to
any user having "spent time to actually figure out what the cause
might be" and contributed any useful idea to "improve cygwin's
'slowness'". 

It may be the case that Corinna thought of this separately and/or
in parallel.  If so, all the better -- it indicates some form
of "synchronicity" (in the Jungian sense) -- an idea whose time
has come.

However, you spend time writing how no one _ever_
investigates performance problems or suggests solutions.
That appears to be a cynical view.   Then, when offered
a clear example to the contrary, you discard the effort
as being "unoriginal" and already something that has been
(and is being) considered independantly of their suggestion.

That \could\ be perceived, by some, as "mean-spirited" or
"spiteful".  I don't feel that this _encourages_ people to take
the time to actually "figure out" problems nor "figure out"
improvements.  If they don't know you, some people might
take it personally. :-)  (Not that you would be expected to
care, publically :-) ).

Linda

p.s. -- pay attention to replies; technical to cygwin; cfg's-infamous
personality discussions: have set default Reply-To: cygwin-talk.  :-)




--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/