1.1.4 select and poll false positive past 32 descriptors

Ted Soo-Hoo Ted.Soo-Hoo@go.ecitele.com
Tue Aug 8 16:54:00 GMT 2000


Chris Faylor wrote:

> From the linux man page on select:
>        n is the highest-numbered descriptor in any of  the  three
>        sets, plus 1.

I understand everything you're telling me. Please let me explain
some more (citing source) and perhaps this will clarify my point.
It's possible I'm way off, but we're miscommunicating right now.

> What is allocated in select.cc reflects the maxiumum number of fds
> that can be operated on.  It's essentially a bit mask.  I don't know
> why you are mentioning this.  If your maximum fd is something like
> 64, then the fd_set's allocated should be at least that large.

I think that was its intent. I suspect that is not what it's doing.
The strace output for recv_fd_max = 42 (so select was given 43) is:

18931 1299283 [main] selectbug 1327 select_stuff::wait: n 2, ms 4294967295

I agree that the select.cc allocation should hold the specified fds,
but I think it's allocating based on that "n 2". I'll explain later.

I'm not setting FD_SETSIZE because the default of 64 is enough here.
You can see from the printing based on sizeof that it's eight bytes.

> I don't know what "the expected value to copy out" might be or what
> you mean by "not work too hard beyond that".

Copy out refers to the copy_fdset at "out:" below to return the fd_set.
The "expected value to copy out" is the number of bits, 43 in the test.

Working too hard refers to the complex count loop in the sources below.
Just looking superficially, it looks like it's counting something with
wait objects and "active fds" in mind. It then allocates based on that.
My guess is that in terms of filling out the w4 array, it's doing fine,
but it then uses the same value later to allocate and return an fd_set.

> I think what you are reporting is that the winsock version of select
> only handles 32 elements.  That's an unfortunate limitation.  It would
> probably be possible to work around this by forking multiple threads to
> handle multiple winsock selects but this would be a fairly major change
> and I am not aware of anyone contemplating this.  Sorry.

I'm reporting that if I use the workaround described, it handles 200.
I think there may be a problem in the allocation code that counts too
few bits and keeps it from reporting its returned bitmasks correctly.

Let me give some snippets from the source code, along with commentary:

These macros I think are to allocate an fd_set based on needed bits:

#define unix_howmany(x,y) (((x)+((y)-1))/(y))

#define sizeof_fd_set(n) \
  ((unsigned) (NULL_fd_set->fds_bits + unix_howmany((n), UNIX_NFDBITS)))

#define allocfd_set(n) ((fd_set *) alloca (sizeof_fd_set (n)))

I'm not sure what w4 is for, but it seems to have to do with waiting:

  HANDLE w4[MAXIMUM_WAIT_OBJECTS];

Here it starts referring to "active fds" which gets me a bit worried:

  /* Loop through the select chain, starting up anything appropriate and
     counting the number of active fds. */
  while ((s = s->next))
    {
      if (!s->startup (s, this))
        {
          __seterrno ();
          return -1;
        }
      if (s->h == NULL)
        continue;
      for (int i = 1; i < m; i++)
        if (w4[i] == s->h)
          goto next_while;
      w4[m++] = s->h;
  next_while:
      continue;
    }

Here it connects n with m whereas maybe it should have counted an n:

  int n = m - 1;

There are a number of uses of m and of n in the code. Let me show n.
Also, I think this is the strace debug_printf that said "n 2" above:

  /* Allocate some fd_set structures using the number of fds as a guide. */
  fd_set *r = allocfd_set (n);
  fd_set *w = allocfd_set (n);
  fd_set *e = allocfd_set (n);
  UNIX_FD_ZERO (r, n);
  UNIX_FD_ZERO (w, n);
  UNIX_FD_ZERO (e, n);
  debug_printf ("n %d, ms %u", n, ms);

out:
  copyfd_set (readfds, r, n);

I think what's happening is it's coming up with a very low n, and the
macros above are having it allocate the minimum 32-bit fds internally
and have the copyfd_set write that much back into the user's bitmasks
thus leaving the additional bitmasks for more than 32 bits untouched.

Since those bits were one-initialized to be tested, they come back as
false-positive results unless they are cleared. Thus I'm getting back

01 00 00 00 ff 07 00 00

and not

01 00 00 00 00 00 00 00

when I simply have stdin be ready.

I think the successful second result is because it runs sel.poll not
sel.wait (which is where the code is that I'm thinking has a bad n).

I hope this clarifies things. I might still be off but I can get 200
select bits back on my workaround and I'm wondering if it would work
without my workaround if the bit-counting was changed. I don't know.

It would be great if select would get the rest of the way and not be
stuck at 32. Even before it was dynamic, wasn't the limit set at 64?

Regards,
Ted

--
Want to unsubscribe from this list?
Send a message to cygwin-unsubscribe@sourceware.cygnus.com



More information about the Cygwin mailing list