This is the mail archive of the libc-alpha@sourceware.org mailing list for the glibc project.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Re: [PATCH 09/14] Add a new pthread_mutexattr_setelision_np interface.


I first should clarify the use models I see for the per lock elision tuning
interfaces.

- Experimentation is one of them
- One scenario that may well happen is that some distributions don't
want to force everyone to use elision (and don't trust the adaptive
algorithms to be good enough). In this case they likely will not 
set the configure flag.  But individual applications may want 
to enable elision, because they have lock scaling problem.
Now one way would be to use the environment variables (which
you also chose not to merge), but then the applications would need to
trust that elision works for every lock.
With the per lock tuning interface they can instead opt-in
only for specific locks, they know have scaling problems and they
explicitely tested.
I think supporting this model is important (even though I personally
prefer elision to be enabled by default).


> Existing glibc functions being poorly designed does not excuse
> us from trying to do better.

Ok so you're completely disregarding existing practice.

I don't think mallopt() is poorly designed: it does exactly
what it needs to do.

For example tuning the mmap threshold (that is controlling when
to return memory back to the OS) makes a lot of sense.

> > Also for experimenting with the algorithms there needs to be some
> > way to turn it on/off per lock. That is what the interface does.
> 
> In which case what you're asking for is an API to be added forever
> to glibc for a potentially temporary problem.

It may be temporary if distributions chose to enable 
elision by default (but I suspect even then there will be always some who
feel the need for per lock tuning)

However if distributions chose to disable it by default,
it's not temporary at all. It would be the only way to use it
(see above)

> It's probably easier to use the generic tunnables inteface I'm
> trying to add to do the kind of experimenting your interested in, 
> but that's another interface to design.

Maybe I'm missing something, but how would a generic tunable 
interface be able to tune something per lock? 

Or are you talking about a generic rwlock_t/mutex_t tuning interface?

> > Could be done, but would be likely awkward.
> Could you describe exactly what you mean by awkward?

You could potentially end up with a lot of flags/parameters.
It would be a very complex interface.

> 
> I'm thinking of:
> 
> * Describe the average critical section for the lock:
> - Does I/O? (yes/no)
> - Size of critical section? (insn count)
> - Syscalls (yes/no)
> - How much data is accessed (bytes)

IO and syscall could be done (although in most cases they
are the same). The others are not well defined in TSX.

> - ...
> * Lock hints recorded in a table.

In what table? A global table? How do per object tunables
fit into your model?

> The mallopt interface is a direct interface into the malloc
> implementation. It shouldn't exist. Instead we should have
> a very easy to use pluggable mechanism to add alternate malloc
> implementations.
> While you can currently plugin an alternate
> malloc, it's incredibly hard to do so correctly,

For me it would seem overkill to link in another malloc, just
because I want to change the mmap threshold.

Also this model doesn't work for tuning things per object.

>  and we don't
> help. Thus users wanted knobs and over the years they got them.
> The fixed knobs make malloc very difficult maintain.

You can always ignore them if you don't like them anymore, 
right?

> I would expect mallopt to be subsumed into the generic tunables
> interface where we explicitly exclude the flags from the ABI
> and make little to no guarantees.

Not sure how that is different from existing mallopt(). You can 
just ignore settings you don't like anymore right?

Of course the user's program may run poorly, but hopefully
you only do that if you really have a much better algorithm.

In fact I was doing some code searches for mallopt() users earlier,
and I found several alternative malloc libraries that just
ignore most settings (but implement others, it seems to be even
portable to other OS)

> 
> >> I will review this patch anyway because I feel that the review
> >> of the patch is instructive and because this patch could get
> >> picked up by a downstream that desires to add these interfaces
> >> and maintain the as an experimental interface.
> > 
> > That would fragment the glibc ABI. Is that what you're suggesting?
> 
> Downstream can do whatever it wants and make whatever guarantees
> it wants. I don't want to judge them.
> 
> A fragmented glibc ABI is no different from running a kernel with
> or without support for certain features.

FWIW the kernel tries very hard to stay binary compatible.

And if I'm allowed to judge I think any ABI fragmentation is a
incredible bad thing.


-Andi
-- 
ak@linux.intel.com -- Speaking for myself only.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]