fix off-by-one in dup2

Eric Blake
Wed Sep 25 23:26:00 GMT 2013

Solves the segfault here:
but does not address the fact that we are still screwy with regards to

Ultimately, based on my understanding of POSIX and glibc, my goal is to
have a number of changes (this patch only scratches the surface; there's
more to go):

dtable.h tracks soft and hard limits, inherited over fork and preserved
across exec

hard limit starts at OPEN_MAX_MAX and can only be reduced
soft limit starts at hard limit, and can be reduced to _POSIX_OPEN_MAX (8)
dtable.size starts at MAX(32, fork/exec size)

getdtablesize() and sysconf(_SC_OPEN_MAX) always returns the soft limit,
as in glibc and permitted by POSIX (_SC_OPEN_MAX is the only sysconf
variable that can be runtime dynamic)

dtable.size is decoupled from soft limit, and is guaranteed to be <=
hard limit.  It can grow up to current soft limit; but soft limit can
later be reduced lower than dtable.size (glibc does this); on fork and
exec, we are careful to still allow fds beyond the current soft limit.

getrlimit(RLIMIT_NOFILE, &r) => returns soft and hard limits from dtable
rather than hard limit as a constant and soft limit as current dtable.size

setrlimit(RLIMIT_NOFILE, &r) => cannot set hard limit to unlimited; soft
limit of unlimited is translated to current hard limit; hard limit
cannot be increased (EPERM) or reduced below dtable.size (EINVAL); soft
limit can be reduced arbitrarily (including below OPEN_MAX of 256)

setdtablesize() => guarantees that dtable.size is at least that large
(must be <= soft limit), but does not lower dtable.size or change limits

2013-09-25  Eric Blake  <>

	dup2: fix off-by-one crash
	* (dup3): Fix off-by-one.
	(find_unused_handle): Reduce time spent expanding during dup.
	* (setdtablesize): Report error on invalid value.

diff --git i/winsup/cygwin/ w/winsup/cygwin/
index 2501a26..c2982a8 100644
--- i/winsup/cygwin/
+++ w/winsup/cygwin/
@@ -233,7 +233,7 @@ dtable::find_unused_handle (int start)
 	if (fds[i] == NULL)
 	  return i;
-  while (extend (NOFILE_INCR));
+  while (extend (MAX (NOFILE_INCR, start - size)));
   return -1;

@@ -754,7 +754,7 @@ dtable::dup3 (int oldfd, int newfd, int flags)

   if (!not_open (newfd))
     close (newfd);
-  else if ((size_t) newfd > size
+  else if ((size_t) newfd >= size
 	   && find_unused_handle (newfd) < 0)
     /* couldn't extend fdtab */
diff --git i/winsup/cygwin/ w/winsup/cygwin/
index e1886e6..8c1c70a 100644
--- i/winsup/cygwin/
+++ w/winsup/cygwin/
@@ -2578,6 +2578,9 @@ system (const char *cmdstring)
 extern "C" int
 setdtablesize (int size)
+  if (size < 0)
+    return -1;
   if (size <= (int)cygheap->fdtab.size || cygheap->fdtab.extend (size -
     return 0;

Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 621 bytes
Desc: OpenPGP digital signature
URL: <>

More information about the Cygwin-patches mailing list