Re: [PATCH 09/14] Add a new pthread_mutexattr_setelision_np interface.

[Torvald Riegel <triegel at redhat dot com>]

On Wed, 2013-07-03 at 09:10 +0200, Dominik Vogt wrote:
> On Tue, Jul 02, 2013 at 12:41:45PM -0400, Carlos O'Donell wrote:
> > On 07/01/2013 08:54 PM, Andi Kleen wrote:
> > > - One scenario that may well happen is that some distributions don't
> > > want to force everyone to use elision (and don't trust the adaptive
> > > algorithms to be good enough). In this case they likely will not 
> > > set the configure flag.  But individual applications may want 
> > > to enable elision, because they have lock scaling problem.
> > > Now one way would be to use the environment variables (which
> > > you also chose not to merge), but then the applications would need to
> > > trust that elision works for every lock.
> > > With the per lock tuning interface they can instead opt-in
> > > only for specific locks, they know have scaling problems and they
> > > explicitely tested.
> > > I think supporting this model is important (even though I personally
> > > prefer elision to be enabled by default).
> > 
> > I agree, but I need to talk this out with the community.
> > 
> > Having a top-down approach to controlling elision per lock
> > is important. I have already stated that I think a perfect API and
> > implementation that infers elision correctly 100% of the time 
> > is an unreasonable goal.
> 
> > Somewhere, sometime, downstream
> > is going to need to disable elision for just one lock, or enable elision
> > for just one lock.
> 
> I doubt that this is realistic.  If an application is harmed by
> elision in general or for specific locks, one of three things is
> probably going to happen: Either the problem is ignored, or
> elision is disabled completely for that application.  Or, if the
> developers think it would be helpful to use transactions in the
> application, it is rewritten so that the work it does benefits
> from transactions without relying on elision.

Maybe.  But we can only find out whether this is true or the other
assumptions about what will happen are true if we actually expose it to
real users.

> The lock elision
> concept is too far away from the hardware instructions, and too
> elusive as a concept from the point of view of the application
> programmer to be a good aread for optimization.  The programmer
> will probably want to use a transaction interface in the language
> rather than relying on library code.

I don't agree.  All the HTMs I'm aware of do not give any
forward-progress guarantee (ie, that transactions without conflicts will
eventually commit and not abort due to any obscure reason).  AMD's ASF
experimented with it (ie, tried to guarantee that small transactions
will finish commit eventually if there are no conflicts), but it seemed
to be complex to implement.  Thus, if a programmer is using
transactions, he/she needs a fallback execution path that does not need
to rely on the HTM.
Having a lock-based implementation as fallback is often a pretty
practical idea: people likely have some experience with locks, they
aren't a new concept, and their behavior in terms of performance is not
too hard to understand.  So, if the programmer decides to use lock-based
code as fallback, he/she will have to build something that's similar to
lock elision.
Other options are (1) custom concurrent code, which is often too
difficult to implement, especially for non-experts, or (2) transactions
implemented in software without use of the HTM (ie, STM), or (3) perhaps
other abstractions such as combinations of atomic snapshots and updates,
RCU, or whatever.
If you implement programming-language-level transactions (e.g.,
transaction statements in C/C++ as supported by GCC), you need a
fallback too.  You can use STMs, but they come with some overhead and if
you want to run STM transactions concurrently with HTM transactions,
you're often slowing down the HTM code paths too because you need to do
additional stuff to synchronize with the STM.  The other option is to
use a single global lock as fallback, but that can obviously limit
scalability quickly.  If you want more fine-grained locking, you could
either ask the programmer to supply a locking scheme (but this kills
lots of the ease-of-use of transactions), or try to let the compiler
infer a fine-grained locking scheme (there is research on this, but it's
hard because it needs a good points-to analysis, and works best with
whole-program analysis).

I agree that language-level transactions have a lot of benefits (and
I've been working on this topic for years), but there are also a lot of
obstacles in the implementation and elsewhere.