Re: [MTASCsft PATCH WIP5 01/33] Multi Thread, Async Signal and Async Cancel safety documentation: intro

[Alexandre Oliva <aoliva at redhat dot com>]

On Nov 28, 2013, Torvald Riegel <triegel@redhat.com> wrote:

> Our functions have sequential specifications, that is, there are
> preconditions and postconditions involving program state, arguments,
> return value.

There's more than that.  There are such effects as calling back, in
unspecified order (not even guaranteed sequential), user code that is
expected to behave in certain ways but that may perform arbitrary
additional operations.  There are such effects as interrupting a
function part-way and performing additional operations there.  There are
such effects as observable interleaving of concurrent executions.  These
are built right into the interface specifications.  I resist referring
to these as sequential specifications in order to retain whatever is
left of my sanity ;-)

Given that the model explicitly exposes unspecified internal states, I
don't see how your attempt to model it as if this wasn't the case is
conducive of intelligent conclusions.  IOW, these square objects won't
fit in these round holes.


> Given that your patch adds MT-Safe docs in the first place, I doubt
> there is a significant number of functions that have a concurrent
> specification.

I don't see how the purported conclusion follows from the premise.
It seems to imply there's some fundamental incompatibility between
thread-safety and concurrency!


> Obviously, this is somewhat simplified, and still incomplete.

It's too simplified and incomplete, and AFAICT built upon a premise that
equates safe with equivalent-to-sequential, which makes it unsuitable to
reason about POSIX.

>> as long as they all refrain from invoking undefined
>> behavior.

> Invoking undefined behavior is anything MT-Unsafe, and anything
> violating the preconditions of the sequential specifications?

If you change s/sequential//, and include the A[SC]-Unsafe cases within
the corresponding A[SC] contexts, and include cases specified as
undefined in the relevant language and other standards, yes.

>> This is perfectly sufficient as a user-visible definition.

> No, see above.  There are choices we have to make and reflect in the
> definition.

The choices you mentioned seemed to me to be artifacts of the unsuitable
model you tried to fit things in.

>> those arise from
>> the user-visible synchronization primitives only, which no functions are
>> required to use internally, for the simple fact that no function is
>> required to perform synchronization.

> If they don't access shared state, they are obviously not required to do
> that.  If they do, they will do in practice because then several threads
> access the same state.  If you don't want to read or produce garbage (in
> the sense of states beyond what's specified in the sequential spec)
> while doing that, you need to synchronize in some way; even if it's
> single-word state, you need to synchronize to prevent the compiler from
> making the assumption that this is sequential code (and thus reload, or
> do any other optimization only safe in sequential code).

> If you mean something else by synchronization, then please define it.  I
> already asked for a proper definition in a prior email.

By synchronization, I meant the synchronization primitives defined by
POSIX, that synchronize the complete memory state (per the definition).
No function is required to do that.  Even if it accesses shared state,
it could refrain from using the global-sync primitives exposed to users,
and rely on stricter ones.  Or C*11 atomics.  Or other features of the
hardware memory model.

Do you agree that only functions that POSIX explicitly specifies as
synchronization functions are required to perform that sort of
synchronization, and that all other implementations can resort to more
efficient primitives that are not exposed by POSIX interfaces, even if
they are available to users through the implementation language?

Do you agree that it follows from this that users of the POSIX APIs must
not rely on any functions' performing global synchronization, and must
perform it on their own if that's what they (the users) require?

>> The internal library implementation may need to be aware of the
>> underlying hardware and language memory models, and the application may
>> want to take advantage of additional flexibility provided by them, but
>> those are outside the scope of POSIX, and of the definition of whether
>> POSIX-defined interfaces are MT-Safe.

> If POSIX doesn't want to rely on a language's memory model (which would
> be unwise in the long term IMHO), fine.

There appears to be some miscommunication here.  Not exposing all of a
model to users doesn't amount to not relying on it internally, nor to
having incompatibilities with it.  Which of these, if any, would be
unwise?

> But then it needs to define it's own memory model precisely.

That's already done.  And, surprise, it's fully compatible with C*11's
memory models!  Thanks for the flexibility afforded by POSIX, C*11 could
evolve and introduce new features and guarantees without conflicting
with it!

> Do you think that's just bloat?

Not really.  It matters when it matters, e.g., to assess properties of
an internal implementation.  For higher abstraction layers, it may very
well not matter at all, as operation most often relies on looser
guarantees typical of higher layers on top of lower ones that take care
of abstract away much of the complexity.

-- 
Alexandre Oliva, freedom fighter    http://FSFLA.org/~lxoliva/
You must be the change you wish to see in the world. -- Gandhi
Be Free! -- http://FSFLA.org/   FSF Latin America board member
Free Software Evangelist      Red Hat Brazil Compiler Engineer