slow handling of large sets of files?

[Lester Ingber]

The Thinkpad (A31p) is 1.8 GHz, not 1.8 MHz.
If there are no suggestions to try, I may install Microsoft's Services
for Unix (SFU) just to see if there any differences on the same machine.

      +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=
I don't understand why handling large data sets is so slow on my Cygwin?
I have had this problem as far back as I can remember with Cygwin.
Running executables prepared under gcc (without large file I/O) seem fine.

Below I include just the top part of `cygcheck -s`.

To be specific, I have tarred-gzipped directory of about 5300 files,
% ls -l DATA.tar.gz 
[...] 92303 Jan 10 05:27 DATA.tar.gz
which can be retrieved using a browser or just `wget` from
http://www.alumni.caltech.edu/~ingber/DATA.tar.gz

I tested the following commands
(1) gzcat DATA.tar.gz | tar xBpf -
(2) rm -rf DATA
on 4 machines/OS's:
(a) ThinkPad (1.8 GHz)/Cygwin (see below) on mount c:/cygwin//home/ingber
(b) Pentium 3 (700 MHz)/FreeBSD 4.10 (lots of users)
(c) SPARC (300 MHz)/Solaris 8 (lots of users)
(d) SPARC (500 MHz)/Solaris 9

The differences between (a) and (b)-(d) are so large that I think my
estimates get the point across:
Under Cygwin, it takes about 380 secs to perform (1), and about 480 secs
to perform (2).
On the other machines, it takes about 20 secs to perform (1).
To perform (2), it takes about 3 secs on (b) and (d), and about 10 secs on (d). 

Thanks.

Lester

    ------------8<------------ top cut -> bottom ------------->8------------
Cygwin Configuration Diagnostics
Current System Time: Mon Jan 10 05:44:49 2005

Windows XP Professional Ver 5.1 Build 2600 Service Pack 2

Path:	C:\cygwin\usr\local\bin
	C:\cygwin\bin
	C:\cygwin\bin
	C:\cygwin\lib\subversion\bin
	C:\cygwin\usr\X11R6\bin
	C:\cygwin\usr\local\bin
	C:\cygwin\bin
	C:\cygwin\bin
	.\
	c:\WINDOWS\system32
	c:\WINDOWS
	.

Output from C:\cygwin\bin\id.exe (nontsec)
UID: 1005(ingber) GID: 513(None)
513(None)

Output from C:\cygwin\bin\id.exe (ntsec)
UID: 1005(ingber)   GID: 513(None)
0(root)             513(None)           544(Administrators) 545(Users)

SysDir: C:\WINDOWS\system32
WinDir: C:\WINDOWS

CYGWIN = `check_case:strict'
HOME = `C:\cygwin\home\ingber'
MAKE_MODE = `unix'
PWD = `/home/ingber'
USER = `ingber'

Use `-r' to scan registry

a:  fd             N/A    N/A                    
c:  hd  NTFS     55902Mb  51% CP CS UN PA FC     LESTER_C
d:  cd             N/A    N/A                    
i:  net NTFS     55902Mb  51% CP CS UN PA FC     LESTER_C

C:\cygwin                          /                         system  binmode
c:\cygwin\managed                  /managed                  system  binmode
C:\cygwin/bin                      /usr/bin                  system  binmode
C:\cygwin/lib                      /usr/lib                  system  binmode
C:\cygwin\usr\X11R6\lib\X11\fonts  /usr/X11R6/lib/X11/fonts  system  binmode
.                                  /cygdrive                 system  binmode,cygdrive

Found: C:\cygwin\bin\awk.exe
Found: C:\cygwin\bin\bash.exe
Found: C:\cygwin\bin\cat.exe
Found: C:\cygwin\bin\cp.exe
Found: C:\cygwin\bin\cpp.exe
Found: C:\cygwin\bin\find.exe
Found: C:\cygwin\bin\gcc.exe
Found: C:\cygwin\bin\gdb.exe
Found: C:\cygwin\bin\grep.exe
Found: C:\cygwin\bin\ld.exe
Found: C:\cygwin\bin\ls.exe
Found: C:\cygwin\bin\make.exe
Found: C:\cygwin\bin\mv.exe
Found: C:\cygwin\bin\rm.exe
Found: C:\cygwin\bin\sed.exe
Found: C:\cygwin\bin\sh.exe
Found: C:\cygwin\bin\tar.exe
    ------------8<------------ bottom cut <- top ------------->8------------


--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/