This is the mail archive of the
libc-help@sourceware.org
mailing list for the glibc project.
Re: Dynamic Linker feature set query
Some background reading of the OpenSSL discussions on openssl-devel.
"libcrypto safe for library use?" -
http://groups.google.com/group/mailing.openssl.dev/browse_thread/thread/b230d5ad9f7589bd/006523c3308ca735?lnk=gst&q=libcrypto+safe+lib
"OpenSSL dependence on external threading functions is a critical design
flaw" -
http://groups.google.com/group/mailing.openssl.dev/browse_thread/thread/c71f94d121d8911a/0cd7e76a9f793185?lnk=gst&q=OpenSSL+dependence
Petr Baudis wrote:
We managed to clarify this after a long IRC discussion.
To be clear here.
Sure we clarified "the problem" since it wasn't so obvious to you from
my previous post.
We did not however clarify "the solution" no change of state occurred
for me on that aspect. Since I am already aware of the points you
raised before you raised them (see my previous posts, before the IRC
discussion in the openssl-devel list). I still maintain an arbitration
mechanism is much needed in the dynamic-linker to address these kinds of
concerns.
Thanks for clarifying the problem in your own words to the libc-help
list and sorry if the matter wasn't described clearly enough by me before.
(i) The OpenSSL pushes the genericity of their threading model to run
time and would like the dynamic linker to do the work. But this makes no
sense on GNU/Linux where the only concievable threading model is
pthreads. The genericity should be at the compile time here.
It doesn't necessary push it to the dynamic linker per say, but to the
consumer of OpenSSL. It would be true to say that OpenSSL is agnostic
of the threading model in use on any platform and that it has never
automatically configured itself for multi-threaded use in the maybe 15+
years history of the project.
To enable multi-thread support in OpenSSL the consumer must follow the
documentation and configure up their choice of threading model (sure
this is an obvious choice for many platforms, but the developer choice
remains there to make). This threading model setup mechanism is not
re-entrant and has more to do with the runtime environment of OpenSSL
than of OpenSSL's build flavor.
OpenSSL's threading primitive requirements largely reduce to provision
and use of a handful of mutexes.
The problem is created by the implicit loading of DSOs due to them being
dependent objects of other DSOs. There appears to be no API for
anything to catch and control that situation so that DSO initialization
is not serialized before first use.
Another way to look at the issue from dlopen() perspective is that the
dlopen() should be a 2 stage operation (so maybe an extra flag can
achieve this, RTLD_TWO_STAGE_OPEN). The first stage does everything it
does now but when dlopen() returns to the caller it continues to keep
the particular DSO and all its dependents in a locked state.
The locked state blocks any other thread trying to call dlopen() or
dlclose() on any of those DSO handles.
This would return control back to the caller of dlopen() to run
initialization of the newly loaded DSO knowing that this initialization
was adequately serialized.
Then you make a new dlopencomplete() API call which would unblock the
loading process. Leaving the state of the DSO like it would be if the
RTLD_TWO_STAGE_OPEN flag was not set for dlopen().
Now you can not do this in application code, since you can never control
_ALL_ the code using dlopen() because some of that code maybe in
dependent libraries those implementation details you could not care less
about.
The basic principal is the same, the dynamic-linker is the arbitration
master. The exact mechanism of arbitration is less important than the
fact arbitration can be achieved in some way.
So my previous post with bullet points at the bottom represents a list
of building blocks via which arbitration could be achieved.
(ii) No cooperation from the dynamic linker is neccessary, you can do
all serialization from within the library easily.
Yes I have put up to the openssl-devel list that it is possible to make
use of CPU-atomic instructions to build a spinlock to protect the
threading model setup.
Doing this doesn't require any additional knowledge of the threading
model in use on the platform and should work on most systems OpenSSL
works on, but it makes a presumption that the memory model in use on the
platform allows it. Which I believe to be true of all the popular
platforms (although there is talk on the openssl-devel list that this
isn't always the case).
Also, there is plenty of available workarounds if the OpenSSL guys do
not want to clean up their model - e.g. you can trivially use atomic
instructions to ensure only one of the threads will do the setup.
It is somewhat debatable that the current OpenSSL model is unclean.
That is certainly not my view. So implying that OpenSSL needs to "clean
up their model" I find laughable maybe due to a hint of ignorance on
your side.
This issue only affects OpenSSL used in conjunction with dynamic
linking, static linking is not affected by this. Other platforms
understand this issue and have mechanisms to address it, in the case of
Windows there is _DllMain callbacks and the guaranteed platform API
symbols InterlockedIncrement of InitializeCriticalSection which has been
provided by the platform for a long time.
Also note that use of atomic instructions is not a trivial matter across
the wider variety of platforms OpenSSL supports, compared to that of
glibc. OpenSSL for example is not tied to the GCC compiler system.
Another view is that this problem can be pushed to distribution
maintainers. i.e. OpenSSL takes up a position this is a runtime
packaging problem, since OpenSSL provide the necessary building blocks
to achieve their goals and address this issue cleanly. If the runtime
chooses not to use it, then do be it. The requirements are clearly
documented and provided.
Darryl