Re: New pthreads library

[Ulrich Drepper <drepper@cygnus.com>]

Mark Kettenis <kettenis@wins.uva.nl> writes:

> Thanks, for looking at the code.  When you say that `The model would
> currently not be usable for Linux', do you mean that it would involve
> a lot of work to merge the manager thread into my current approach,
> which would be a lot of wasted effort since the missing functionality
> is going to be added to the Linux kernel anyway?

Yes, and hopefully yes.  There are some patches for the kernel
floating around which will make the manager unnecessary.  But I have
not yet heard a definitive word from Linus.

> I agree that such optimizations should be made when they're possible.
> However, I'm a little puzzled over the current implementation.  If
> we're compiling the library for a processor that does not support all
> atomic operations (e.g. the i386) the file
> `sysdeps/generic/atomicity.h' is used.  The operations in this file
> aren't really atomic.

Right.  I don't say that the current atomicity.h files are all correct
or that all needed ones and all functionality are available.

E.g., we should have macros like

  #define ATOMIC_VARIABLE(class, type, name)

which could expand for machines without appropriate instructions to

  #define ATOMIC_VARIABLE(class, type, name) \
    class __spin_lock_t name##_lock; \
    class type name

and then ATOMIC_INCREMENT could be defined like

  #define ATOMIC_INCREMENT(name) \
    __spin_lock (name##_lock); \
    ++name; \
    __spin_unlock (name##_lock)


But I really don't care about i386 anymore (and not for old SPARCs
etc) which is why generic/atomicity.h is currently used.  Beside,
there rarely were SMP systems using these processors so the atomicity
primitives need not be SMP safe.

> [Roland and Thomas] said that making the pointer to the thread data
> structure the thread ID was defenitely the right thing (and I
> agree), and said that this might very well be that the intention of
> the standard was to make ESRCH a ``may fail'' condition in all
> cases.  I decided to keep things as they are now until someone
> complains about it.

Using the pointer is only possible if there is a fixed number of
descriptors available and they are all in a preallocated array.  Just
like the old implementations of stdio.  Since I assume that for the
Hurd there will be no limits you cannot use pointers for the reasons
you cited: some function must be able to recognize invalid thread
handles.

I really would suggest to go with an index based thread handle and use
the indirection.  Maybe even with some kind of generation number.
I.e., you a 32/64 bit value where the lower bits are an index and some
of the upper bits are a generation number.  Then a debugging version
could recognize thread handle where the index is again valid, but it
was left over from a different thread.  But this is something for a
debugging version.

> Thanks for explaining what THREAD_GETMEM is for.  I didn't quite get
> why this `ugflication' of the code was necessary.  Of course I never
> looked at the SPARC code.  I will certainly convert my code to do
> this, but I may postpone this until the code has stabilized a bit.

Well, by waiting you only make the number of needed changes bigger.

> By `using this trick for the x86' you probably mean the approach laid
> out in `linuxthreads/sysdeps/i386/useldt.h'.

Yes.

> If you want to support arbitrary stacks (_PTHREAD_STACKADDR and
> _PTHREAD_STACKSIZE support) in an efficient way, such a mechanism is
> almost essential.

It's also essential if you have programs which set up there own
stacks.  E.g., one of the Smalltalk implementations does this.

> A while back, when you were hinting at a rewrite of the threads
> library, you talked about a paper describing the new Mach 3.0 cthreads
> library.  I belive the paper you were talking about is `Randall
> W. Dean, Using Continuations to Build a User-Level Threads Library'.
> I've read the paper and it seems like a good approach to me.

This is one approach but it's not the only way to do this.

> Anyway, I'll design the public interfaces in such a way that moving
> `one-on-one' model to the `many-on-many' model is possible.  I
> believe that this only means that I must avoid exposing details
> about the underlying kernel threads, which is a good goal anyway,
> and is not hard to achieve.

What this does mean is that there must be two kinds of thread
descriptors: kernel and user.  The API only exposes the user threads
but the library must know about both.  And ideally the only machine
dependent thing is the kernel thread part.

Therefore I'd prefer to see this model being taken into account
immediately.  The whole user-thread creation would be generic code.
The pt-start.c files you currently use would not be in sysdeps.  And
it should be possible to implement the normal 1:1 model even with the
structure being the fully development n:m model without performance
decrease.

> If you have any other suggestions for things that I have to keep in
> mind, please don't hesitate to mention them to me.

I'd like to see already at this point a kernel_thread_t separate from
the user pthread_t.  The pthread_t are directly created by the
pthread_create calls, the kernel_thread_t objects are created as an
reaction to a request to run a pthread_t.  I.e., for now always a
kernel thread is created.  Then this can be changed later very easy.

Therefore I'd like to see a user-level scheduler instance (a library
subfunction for now) which gets an pthread_t object reference and then
makes it run.  Either by calling clone() (or whatever Mach uses) or
later by folding the new thread in the set of running user threads.


I know this is all a bit vague and I appreciate that you have done
that much work already.  But I'd really like to see some more planning
and discussion before too much code is written.

-- 
---------------.      drepper at gnu.org  ,-.   1325 Chesapeake Terrace
Ulrich Drepper  \    ,-------------------'   \  Sunnyvale, CA 94089 USA
Cygnus Solutions `--' drepper at cygnus.com   `------------------------