state of the project

[Ulrich Drepper <drepper at redhat dot com>]

With the beginning of the new calendar year I thought it appropriate to
describe my impression of the state of the project.  A lot happened in
the last time and probably not everybody had the chance to follow all
the development closely enough.


2002 saw the release of the 2.3 version of glibc which was another big
step.  2.2 was the release where we achieve feature-completeness, more
or less.  What followed in the 2.3 development was mostly optimization.
 And this we did, a lot, and with good success.  Application startup
time and runtime costs associated with glibc were reduced, often
significantly.  The release was of high quality, we only had do fix a
few minor problems in a 2.3.1 follow-up release.

Since the 2.3.1 release mostly dealt with threads and related issues.
The implementation of thread local storage (TLS) required significant
changes, also to the infrastructure.  The problems are mostly caused by
required to support old kernel versions which cannot provide TLS
functionality.

In addition to TLS the work on a new thread library proceeded.  The
goals for the new implementation were high.  Many of the mistakes made
in the LinuxThreads implementation should be avoided but still we want
to maintain binary compatibility with LinuxThreads.  The complete set of
POSIX interfaces should be implemented, and this time with the correct
semantics.  The fact that LinuxThreads failed to be POSIX-compliant in
some things naturally limits the compatibility of the two
implementations.  But a program does not use functionality outside the
POSIX requirement it should be fine.  Even when compiling in an
compilation environment with the new implementation's headers a program
should work with LinuxThreads.

The result of the work on NPTL, the new thread implementation, can
checked out.  Sources are available:

  http://people.redhat.com/drepper/nptl/nptl-0.14.tar.bz2

(this is the latest version as of this writing, check whether there is a
more recent version before downloading).  This implementation currently
only supports x86 but it is believed to be fully functional and
complete.  In the process of integrating the new thread code in a
systems which also have to be able to run LinuxThreads many changes were
made to make LinuxThreads look more like NPTL.  The resulting changes
improved LinuxThreads significantly so there is no harm done for
architectures != x86.


Now that the positive aspects of the development are covered, on to some
not so good news.

Most of the development is done firstly for x86.  This is only natural;
it's the architecture I and most of the other glibc maintainers use by
default.  The development also happens almost exclusively for Linux.
The development for Hurd is mostly reactionary.

But this is a problem: other architectures and platforms fall behind.
The problem is not that bad for Hammer (x86-64) and IA-64.  Many of the
developers, including myself, have direct access to appropriate machines
which are fast enough to support the development (glibc is a rather
large project).  This is also due to support from Intel and AMD who,
this should be mentioned, provide hardware.

Another noteworthy exception is SH.  Kaz Kajima follows admirably close
the development even though native compilation takes ages and
cross-compiling is never as well supported as native compilation.

SPARC and Alpha are kept somewhat usable mainly due to the efforts of
individuals (Jakub and Richard in these cases).  But that's mostly it.

PowerPC32 suffered in the last year by not having somebody available who
can dedicate a lot of time.  And unfortunately a lot is needed.  Many of
the new glibc features require support in the tools, binutils and gcc,
which has to be done.  In the case of PPC32 it's even worse: due to a
misdesign of the ABI fundamental changes are necessary.  Somebody has to
take the initiative.  Franz Sirl agreed to take over PPC ownership for
glibc but he made clear that he has no time to resolve the outstanding
problems related to ABI and tools.

In a similar situation, i.e., suffering due to lack of time, seems to be
Arm.  Philip unfortunately has to point out the lack of time.  Again,
changes to the ABI and tools are needed.

PPC64, S390, S390x are in a slightly different boat.  Here the
development stopped after the port was finished and glibc worked, at
some time, out of the box.  Continued maintenance is missing and it
shows.  I think glibc cannot even be compiled for any of these platforms
today.  The owner of the port (IBM in all three cases) has to realize
that the port ownership is not a one time thing.  It means continued
efforts.

Some efforts could be observed lately at the HPPA front.  But I think
they are still shy of the goal.  Plus HPPA has one huge problem going
forward: the ill-designed instruction set of the processor will make
adding HPPA support to NPTL very interesting.  Have fun guys.

Andreas Schwab recently seems to have found some time to work on m68k.
I don't know the status and whether this means the architecture is
maintained again.

This brings us finally to the "embedded junk": MIPS, Cris.  The Cris
port was probably dead on arrival.  Maybe it worked at some point in
time for a very short period but it long has been neglected.  Just as it
is done in the gcc project I should consider removing dead code.  It'll
be announced early enough to give interested parties a chance to react.

As for MIPS, if one architecture has more ABIs then all the other
architectures combined you know something is wrong.  The MIPS port
cannot even be compiled correctly for the old ABIs for quite some time
now.  Occasional patches by Andreas Jaeger and HJ couldn't help fighting
the deterioration and they certainly cannot help going forward.  And
about the new ABIs I can only say one thing: I will not allow glibc
development to be hindered by the many, many design failures made in the
MIPS ABIs.  If the support needed to support the ABIs makes the general
sources less readable or maintainable the changes won't go in.  Period.
 I don't allow the project to be hurt by these design mistakes.


I don't know whether the news contained here was good for you or not.
For those interested in non-mainstream architectures this should be a
wakeup call.  Get your butts up and do soemthing.  It is not the
responsibility of the other port maintainers and not my responsibility
to maintain the ports.  Except for occasional global edits no work is
done on the ports the individual maintainer isn't interested in.  This
definitely is bad news for some people.  I know from all the complaints
I get about non-compiling CVS sources that there are lots of people
interested is _using_ glibc for PPC, MIPS, Arm.  The key word here is
"using": these people have no interest in doing anything for it.  Very
mildly said: I find this kind of behavior disgusting, it's pure
exploitation of free software.  You don't get anything for free what
nobody else is interested to do or gets paid for.  Nobody has the right
to expect that every once supported platform remains usable.


Having all this said here's my appeal: if you find yourself in a
situation where you need glibc, do something for it.  Either do the work
yourself or motivate (by whatever means) somebody else to do it.  We
will definitely make a lot progress on improving glibc on x86 and Hammer
this year.  If the current situation wrt the port maintenance does not
change the quality gap between the architectures will widen even more.
We are already in a world where we have at least two tiers of platforms.
 There are now programs which can only be compiled for x86.  This is
contrary to glibc's goals of providing a uniform programming environment
across all support platforms but I won't force anybody to care for all
the different ports.  It's clearly up to the interested parties to react.

-- 
--------------.                        ,-.            444 Castro Street
Ulrich Drepper \    ,-----------------'   \ Mountain View, CA 94041 USA
Red Hat         `--' drepper at redhat.com `---------------------------