1.4.5 gnu sort 2.0.16 extremely slow under Win98 (tiny default buffer)

Patrick Tufts zippy@cs.brandeis.edu
Tue Feb 19 11:03:00 GMT 2002


I looked at the temp files sort creates and noticed that they were around
700 bytes (bytes, not kilobytes) in size. The normal behavior of sort is to
create temp files of around 16kB in size.

When I use the --buffer-size option to force a large buffer,
like --buffer-size=200000 (200MB) sort completes in seconds.

I'm running sort like this:

cat <47k line file> | perl -ne "<script that returns 1000 lines from the
source file>" | sort -k 1nr,1 -k 2,2 -o outputfile

The perl script on its own takes a second or less to run - it's a simple
filter.

--Pat


Cygnus Win95/NT Configuration Diagnostics
Current System Time: Tue Nov 20 17:19:00 2001

Windows 98 SE Ver 4.10 build 67766446  A=20

Path:	C:\bin\lang\cygwin\usr\local\bin
	C:\bin\lang\cygwin\bin
	C:\bin\lang\cygwin\bin
	c:\WINDOWS
	c:\WINDOWS\COMMAND
	c:\BIN\APPS\PGP

SysDir: C:\WINDOWS\SYSTEM
WinDir: C:\WINDOWS

HOME =3D `C:\bin\lang\cygwin\home\ptufts'
MAKE_MODE =3D `unix'
PWD =3D `/cygdrive/c/Windows/'
USER =3D `ptufts'

BLASTER =3D `A220 I5 D1'
CMDLINE =3D `bash --login -i'
COMSPEC =3D `C:\WINDOWS\COMMAND.COM'
OLDPWD =3D `/cygdrive/c'
PROMPT =3D `$p$g'
PS1 =3D `\[\033]0;\w\007
\033[32m\]\u@\h \[\033[33m\w\033[0m\]
$ '
SHLVL =3D `1'
TEMP =3D `c:\WINDOWS\TEMP'
TERM =3D `cygwin'
TMP =3D `c:\WINDOWS\TEMP'
WINBOOTDIR =3D `C:\WINDOWS'
WINDIR =3D `C:\WINDOWS'
_ =3D `/usr/bin/cygcheck'

HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Explorer\Menu=
Order\Start Menu\&Programs\Cygnus Solutions
  (default) =3D (unsupported type)
HKEY_CURRENT_USER\Software\Cygnus Solutions
HKEY_CURRENT_USER\Software\Cygnus Solutions\Cygwin
HKEY_CURRENT_USER\Software\Cygnus Solutions\Cygwin\mounts v2
  (default) =3D `/cygdrive'
  cygdrive flags =3D 0x00000020
HKEY_CURRENT_USER\Software\Cygnus Solutions\Cygwin\Program Options
HKEY_LOCAL_MACHINE\SOFTWARE\Cygnus Solutions
HKEY_LOCAL_MACHINE\SOFTWARE\Cygnus Solutions\Cygwin
HKEY_LOCAL_MACHINE\SOFTWARE\Cygnus Solutions\Cygwin\mounts v2
HKEY_LOCAL_MACHINE\SOFTWARE\Cygnus Solutions\Cygwin\mounts v2\/
  (default) =3D `C:/bin/lang/cygwin/'
  flags =3D 0x00000008
HKEY_LOCAL_MACHINE\SOFTWARE\Cygnus Solutions\Cygwin\mounts v2\/usr/bin
  (default) =3D `C:/bin/lang/cygwin/bin'
  flags =3D 0x00000008
HKEY_LOCAL_MACHINE\SOFTWARE\Cygnus Solutions\Cygwin\mounts v2\/usr/lib
  (default) =3D `C:/bin/lang/cygwin/lib'
  flags =3D 0x00000008
HKEY_LOCAL_MACHINE\SOFTWARE\Cygnus Solutions\Cygwin\Program Options

a:  fd           N/A    N/A                   =20
c:  hd  FAT32   4848Mb  98% CP    UN           HARDDRIVE

.                       /cygdrive  user    textmode,noumount
C:/bin/lang/cygwin/     /          system  textmode
C:/bin/lang/cygwin/bin  /usr/bin   system  textmode
C:/bin/lang/cygwin/lib  /usr/lib   system  textmode

Found: C:\bin\lang\cygwin\bin\bash.exe
Found: C:\bin\lang\cygwin\bin\cat.exe
Not Found: cpp (good!)
Found: C:\bin\lang\cygwin\bin\find.exe
Found: c:\WINDOWS\COMMAND\find.exe
Warning: C:\bin\lang\cygwin\bin\find.exe hides =
c:\WINDOWS\COMMAND\find.exe
Not Found: gcc
Not Found: gdb
Not Found: ld
Found: C:\bin\lang\cygwin\bin\ls.exe
Not Found: make
Found: C:\bin\lang\cygwin\bin\sh.exe

   18k 2000/10/23 C:\bin\lang\cygwin\bin\cyggdbm.dll - os=3D4.0 =
img=3D1.0 sys=3D4.0
                  "cyggdbm.dll" v0.0 ts=3D2000/10/22 19:26
   45k 2001/04/25 C:\bin\lang\cygwin\bin\cygform5.dll - os=3D4.0 =
img=3D1.0 sys=3D4.0
                  "cygform5.dll" v0.0 ts=3D2001/4/24 22:28
   26k 2001/04/25 C:\bin\lang\cygwin\bin\cygmenu5.dll - os=3D4.0 =
img=3D1.0 sys=3D4.0
                  "cygmenu5.dll" v0.0 ts=3D2001/4/24 22:27
  156k 2001/04/25 C:\bin\lang\cygwin\bin\cygncurses++5.dll - os=3D4.0 =
img=3D1.0 sys=3D4.0
                  "cygncurses++5.dll" v0.0 ts=3D2001/4/24 22:29
  226k 2001/04/25 C:\bin\lang\cygwin\bin\cygncurses5.dll - os=3D4.0 =
img=3D1.0 sys=3D4.0
                  "cygncurses5.dll" v0.0 ts=3D2001/4/24 22:17
   15k 2001/04/25 C:\bin\lang\cygwin\bin\cygpanel5.dll - os=3D4.0 =
img=3D1.0 sys=3D4.0
                  "cygpanel5.dll" v0.0 ts=3D2001/4/24 22:27
   34k 2001/09/30 C:\bin\lang\cygwin\bin\cygform6.dll - os=3D4.0 =
img=3D1.0 sys=3D4.0
                  "cygform6.dll" v0.0 ts=3D2001/9/29 19:43
   19k 2001/09/30 C:\bin\lang\cygwin\bin\cygmenu6.dll - os=3D4.0 =
img=3D1.0 sys=3D4.0
                  "cygmenu6.dll" v0.0 ts=3D2001/9/29 19:43
  175k 2001/09/30 C:\bin\lang\cygwin\bin\cygncurses++6.dll - os=3D4.0 =
img=3D1.0 sys=3D4.0
                  "cygncurses++6.dll" v0.0 ts=3D2001/9/29 19:45
  201k 2001/09/30 C:\bin\lang\cygwin\bin\cygncurses6.dll - os=3D4.0 =
img=3D1.0 sys=3D4.0
                  "cygncurses6.dll" v0.0 ts=3D2001/9/29 19:42
   12k 2001/09/30 C:\bin\lang\cygwin\bin\cygpanel6.dll - os=3D4.0 =
img=3D1.0 sys=3D4.0
                  "cygpanel6.dll" v0.0 ts=3D2001/9/29 19:43
   17k 2001/06/28 C:\bin\lang\cygwin\bin\cyghistory4.dll - os=3D4.0 =
img=3D1.0 sys=3D4.0
                  "cyghistory4.dll" v0.0 ts=3D2001/1/6 20:34
   21k 2001/06/28 C:\bin\lang\cygwin\bin\cyghistory5.dll - os=3D4.0 =
img=3D1.0 sys=3D4.0
                  "cyghistory5.dll" v0.0 ts=3D2001/6/27 19:27
  108k 2001/06/28 C:\bin\lang\cygwin\bin\cygreadline4.dll - os=3D4.0 =
img=3D1.0 sys=3D4.0
                  "cygreadline4.dll" v0.0 ts=3D2001/1/6 20:34
  135k 2001/06/28 C:\bin\lang\cygwin\bin\cygreadline5.dll - os=3D4.0 =
img=3D1.0 sys=3D4.0
                  "cygreadline5.dll" v0.0 ts=3D2001/6/27 19:27
   49k 2001/02/03 C:\bin\lang\cygwin\bin\cygz.dll - os=3D4.0 img=3D1.0 =
sys=3D4.0
                  "cygz.dll" v0.0 ts=3D2001/2/3 12:35
   21k 2001/06/20 C:\bin\lang\cygwin\bin\cygintl.dll - os=3D4.0 =
img=3D1.0 sys=3D4.0
                  "cygintl.dll" v0.0 ts=3D2001/6/20 10:09
  119k 2001/06/06 C:\bin\lang\cygwin\bin\cygjpeg6b.dll - os=3D4.0 =
img=3D1.0 sys=3D4.0
                  "cygjpeg6b.dll" v0.0 ts=3D2001/6/5 21:27
  714k 2001/11/14 C:\bin\lang\cygwin\bin\cygwin1.dll - os=3D4.0 =
img=3D1.0 sys=3D4.0
                  "cygwin1.dll" v0.0 ts=3D2001/11/13 20:16
    Cygwin DLL version info:
        DLL version: 1.3.5
        DLL epoch: 19
        DLL bad signal mask: 19005
        DLL old termios: 5
        DLL malloc env: 28
        API major: 0
        API minor: 47
        Shared data: 3
        DLL identifier: cygwin1
        Mount registry: 2
        Cygnus registry name: Cygnus Solutions
        Cygwin registry name: Cygwin
        Program options name: Program Options
        Cygwin mount registry name: mounts v2
        Cygdrive flags: cygdrive flags
        Cygdrive prefix: cygdrive prefix
        Cygdrive default prefix:=20
        Build date: Tue Nov 13 23:16:25 EST 2001
        CVS tag: cygwin-1-3-5-2
        Shared id: cygwin1S3


Cygwin Package Information
Last downloaded files to: C:\My Documents
Last downloaded files from: =
ftp://ftp.nas.nasa.gov/mirrors/cygwin.com/pub/cygwin

Package             Version            =20
ash                 20011018-1         =20
bash                2.05a-1            =20
clear               1.0                =20
crypt               1.0-1              =20
cygrunsrv           0.94-2             =20
cygwin              1.3.5-3            =20
diff                0.0                =20
fileutils           4.1-1              =20
findutils           0.0                =20
gdbm                1.8.0-3            =20
gettext             0.10.38-2          =20
grep                2.4.2-1            =20
gzip                1.3-1              =20
inetutils           1.3.2-15           =20
jpeg                6b-4               =20
less                358-3              =20
libncurses5         5.2-1              =20
libncurses6         5.2-2              =20
login               1.4-2              =20
lynx                2.8.4-1            =20
ncftp               3.0.2-2            =20
ncurses             5.2-7              =20
openssh             3.0p1-1            =20
openssl             0.9.6b-1           =20
perl                5.6.1-2            =20
readline            4.2-3              =20
regex               4.4                =20
sed                 3.02-1             =20
sh-utils            2.0-2              =20
squid               2.4-STABLE20010508 =20
tar                 1.13.19-1          =20
termcap             20010825-1         =20
terminfo            5.2-1              =20
textutils           2.0.16-1           =20
vim                 6.0.93-1           =20
w32api              1.1-1              =20
wget                1.7-1              =20
which               1.5-1              =20
zlib                1.1.3-6            =20

Use -h to see help about each section



--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/



More information about the Cygwin mailing list