Re: Implementing C++1x and C1x atomics

[Lawrence Crowl <crowl at google dot com>]

On 8/13/09, Joseph S. Myers <joseph@codesourcery.com> wrote:
> On Thu, 13 Aug 2009, Lawrence Crowl wrote:
> > > Now a processor D for this architecture comes out.  All code
> > > for A, B and C will work on D, but D also has 8-byte atomic
> > > operations.  GCC 4.7, with -march=D, generates code that
> > > uses these operations inline.  If code built with GCC 4.7
> > > -march=D, and code built with GCC 4.6 or without -march=D,
> > > are used together with the glibc 2.12 shared library, both
> > > implementations of the atomic operations are now used and
> > > things don't work.
> >
> > Here is where the heavy platform nature of the atomics comes in.
> > The installed shared glibc 2.12 on the system with a D processor
> > must have been built with -march=D.  If so, then all operations
> > share the same implementation, and everything works.
>
> The practicalities of GNU/Linux distribution deployment mean that
> you don't want to have many different versions of libc built for
> different processors and that you don't want to require users
> of code optimized for a new processor to have a correspondingly
> new or newly built libc.  This is why STT_GNU_IFUNC is useful;
> in theory you can build libc for every processor variant, in
> practice it's better to have one or very few binaries of it and
> have those dynamically use optimized versions of the few functions
> for which CPU optimization make a major difference.

I was thinking of STT_GNU_IFUNC as an optimization on producing
multiple binaries.

> > > glibc 2.13 changes the out-of-line implementation to test
> > > at runtime whether it is running on D, and use the new
> > > instruction instead of the lock-based implementation if so
> > > (probably using STT_GNU_IFUNC so this test is only run the
> > > first time the symbol is resolved).  That new glibc will now
> > > work with objects built with either 4.6 or 4.7.
> >
> > Well, okay, but there would be less exposure to problems if
> > -march=D implied that the library was compiled with -march=D
> > (or better).
>
> It doesn't.  It doesn't imply then processor D had even been
> invented when the C library was built.  You can reasonably build
> a program with a new compiler, against an old libc, that uses its
> own dynamic choice of code for different CPUs, will run on a range
> of distributions and will use code good for the particular CPU
> it is run on; it simply needs to avoid inlining these particular
> operations, while exploiting all other new features of D.

The atomics need a consistency of implementation, which makes them
particularly imporant to have in the system.

> > > But on GNU/Linux - unlike BSDs, say - it is expected that the
> > > compiler, libc and kernel versions can be updated more or less
> > > independently, and that it should be possible to use a newer
> > > compiler to build code that will run with an older C library.
> > > So the case of GCC 4.7 with glibc 2.12 needs to work.  This
> > > means that code built with GCC 4.7 against the <stdatomic.h>
> > > header provided with glibc 2.12 must not use the 8-byte atomic
> > > instruction that GCC 4.7 knows how to use, because glibc 2.12
> > > will not use it in the out-of-line implementation at runtime.
> >
> > I am suggesting that certain highly processor-dependent routines
> > should be updated with the processor.  That is, I don't think
> > the taxonomy is quite right.
>
> I am saying it is desirable not to have to update those with
> the processor; at most, to have to update the kernel for
> a new processor and keep the same userspace distribution.
> Users routinely run distributions predating their hardware
> (copying an installing from one system to another, or keeping the
> same distribution deployed across multiple systems); they should
> be able to exploit the features of the new hardware in their own
> binaries as far as possible (meaning -march=D should work fine,
> just not inline these operations) while doing so.

So, if -march=D should not imply inlining of the atomic operations,
we need another option that does.  That other option in turn
must require the dynamic library use compatible implementations.
(I'd really like to see errors caught by the loader.)

> > > Are you proposing to avoid this issue by saying that the
> > > platform ABI for GNU/Linux on an X processor is that the
> > > 8-byte operations must never be inlined, and so making GCC
> > > not use the inline operations with -march=D (for GNU/Linux -
> > > it might be different for another OS)?
> >
> > No, I am proposing that an object compiled with -march=D should
> > fail to load on a system that doesn't have both a >=D processor
> > and a >=D library.
>
> Whereas I say -march=D should not cause the operations to be
> inlined unless the library (whose headers are used at compile time)
> is D-aware.  Which means the library communicates to the compiler
> whether it is D-aware.  The simplest way of doing this is through
> macro definitions such as those I suggested, but it could also
> use pragmas to declare properties of the out-of-line functions it
> provides.  Those pragmas could even go in the stdc-predef.h header,
> implicitly preincluded, that I have proposed for other reasons,
> and thereby affect all uses of the relevant compiler intrinsics,
> if you wish to ensure that users can always use the intrinsics
> directly if they wish.  (Note that the stdc-predef.h header has
> not so far been accepted by glibc.)
>
> /* in stdc-predef.h (or stdatomic.h if you don't like stdc-predef.h) */
> #ifdef __arch_D__
> #pragma GCC atomic_8byte_ok_to_inline
> #endif

I must say the header approach really worries me.  The problem is
that the library that you compile under may not be the library that
you execute under.  I shudder to think what shipping preprocessor
output from one system to another will do.

I have no objection to "don't worry about coordinating" -march=D,
but as you say, the atomics cannot use that option.  My conclusion
is that we need another option.

-- 
Lawrence Crowl