Re: Thread-, Signal- and Cancellation-safety documentation

[Rich Felker <dalias at aerifal dot cx>]

On Sun, Jun 02, 2013 at 02:06:36AM -0300, Alexandre Oliva wrote:
> On Jun  1, 2013, Alexandre Oliva <aoliva@redhat.com> wrote:
> 
> > What must not happen is for mem synch primitives, called by users, to
> > fail to propagate effects of posix API calls executed locally to be
> > published to global memory, and make effects of API calls executed in
> > other threads before their latest mem synch primitives visible to the
> > local thread.  Other than that, all bets are off.
> 
> Now, bringing that back to the topic that matters to the project I'm on,
> and trying to show that the guarantees you're looking for are not
> given by posix, consider this:
> 
> char a, b; // globals
> int pipefd[2]; // the two ends of a pipe internal to this process
> 
> thread1 {
>   a = 42;
>   b = 6 * 9;
>   write (pipefd[1], &b, 1);
> }
> 
> thread2 {
>   read (pipefd[0], &a, 1);
>   printf ("%i\n", a);
> }
> 
> 
> Now, obviously there's a happens-before relationship between the write
> and the read, right?  However, since neither read nor write are memory
> synchronization operations, nothing in posix guarantees that the write
> to a in thread1 won't prevail over the write to a implied by the read
> call in thread2: in the absence of a memory sync operation, it is a data
> race, that invokes undefined behavior, even when there's an obvious
> happens-before.

There is no obvious happens-before in your code. The _compiler_ is
free to reorder the store to a with respect to the write call assuming
it's aware that write is a standard POSIX function that could not
possibly examine a. You'd have to work harder to make a true
happens-before, but I'm satisfied that it's possible to do so even
without a good example.

> Now, both read and write are thread safe, so it follows that being
> thread safe doesn't offer ordering guarantees, not even when a
> happens-before is present!
> 
> How could this be possible?  Well, precisely because the internal
> implementation of the read and write calls could use internal mechanisms
> that, in spite of flushing the data in the write buffer all the way to
> the read buffer in another thread, does not guarantee that any other
> data is flushed.
> 
> Now, using streams rather than file descriptors would not make any
> difference as far as ordering guarantees are concerned: in spite of the
> obvious happens-before, no memory synchronization is guaranteed.
> Indeed, nothing in posix precludes the FILE* to be fake pointers used
> just as identifiers, or pointers to memory accessible only to the
> kernel, as long as the stream manipulating interfaces behave as
> specified; they could all be atomic system calls, or they could use any
> form of magic (or sufficiently advanced technology ;-) to implement
> behavior that meets the specification while ensuring consistency of the
> internal data structures even in the presence of concurrent calls.
> 
> Even the availability of explicit stream locking (flockfile) does not
> bring memory synchronization with it, so in spite of mutual exclusion,
> ordering is not guaranteed.  I don't see any requirement that would
> render non-compliant an implementation of stream write operations that,
> given multiple writes within a flockfile/funlockfile pair, queued them
> up in memory local to the thread, getting them all ordered and written
> out at subsequent explicit mem sync calls, or implicitly.
> 
> Any evidence that this is not so?

I agree with your analysis. I'm not sure whether this is intentional
or an oversight. I suspect many historical applications perform
synchronization of shared memory maps with pipes or other
filesystem-based methods, and I also suspect that all historical
implementations have full memory barriers, not by choice but because
they're inevitable, in the kernelspace part of any function capable of
performing inter-process communication. So it may be preferable to
amend the standard to make such applications conforming, though I'm
not even sure _how_ that could be done without imposing overly-strict
synchronization requirements or very complex language...

Rich