PING: fix ARG_MAX

Eric Blake ebb9@byu.net
Tue Sep 20 12:45:00 GMT 2005


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

According to Christopher Faylor on 9/19/2005 8:31 AM:
> If this is really true, then the findutils configury should be
> attempting some kind of timing which finds that magic point where it
> should be ignoring _SC_ARG_MAX.  It shouldn't be vaguely assuming that
> it is in its best interests to ignore it because someone thinks that the
> cost of processing each argument outweighs the benefits of forking fewer
> tests.

POSIX allows xargs to have a default size (currently, xargs defaults to
128k unless otherwise constrained by _SC_ARG_MAX), and that -s can change
that size to anything within the range permitted by _SC_ARG_MAX.

> 
> Given that cost of forking is much more expensive on cygwin than on
> other systems I really don't see how you can use this argument anyway
> and, IMO, it doesn't make much sense on standard UNIX either.  If you
> create more processes via fork you are invoking the OS and incurring
> context switches.  You're still processing the same number of arguments
> but you're just going to the OS to handle them more often.  I don't see
> how that's ever a win.

In isolation, no.  But it is what else you are doing with the arguments -
the text processing of xargs to parse it into chunks, and the invoked
utility's processing of its argv, that also consumes time.  Also, lots of
data tends to imply more page faults, which can be as expensive as context
switches anyways.

> 
> I'm willing to be proven wrong by hard data but I think that you and the
> findutils mailing list shouldn't be making assumptions without data to
> back them up.

Did you not read the thread on bug-findutils?  Bob Proulx proposed a test
that shows that there is NO MEASURABLE DIFFERENCE between a simple xargs
beyond a certain -s:
http://lists.gnu.org/archive/html/bug-findutils/2005-09/msg00038.html

Then I repeated the test on cygwin, and found similar results:
http://lists.gnu.org/archive/html/bug-findutils/2005-09/msg00039.html

There comes a point, where even when all xargs is doing is invoking echo,
that the cost of passing that much information through pipes does overtake
the cost of forks.

However, I am also keen on providing a more reasonable -s behavior in
xargs.  If cygwin were to have pathconf(filename, _PC_ARG_MAX), where a
PATH search were done when filename does not contain '/', then pathconf
could return 32k on Windows processes, and unlimited (or an actual known
limit) for cygwin processes, so that xargs can then allow unlimited -s
sizes for cygwin processes but cap windows processes at 32k and never
encounter the E2BIG.

- --
Life is short - so eat dessert first!

Eric Blake             ebb9@byu.net
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (Cygwin)
Comment: Public key at home.comcast.net/~ericblake/eblake.gpg
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFDMAPo84KuGfSFAYARAry+AKCrEPEhqsTIQwWKrLpNA2M1qC/dFACeLz9k
aPTSZXTkUZCHUkoDNIiPdxA=
=zS83
-----END PGP SIGNATURE-----



More information about the Cygwin-patches mailing list