About the dll search algorithm of dlopen

Michael Haubenwallner michael.haubenwallner@ssi-schaefer.com
Wed Jun 1 14:25:00 GMT 2016

On 06/01/2016 01:09 PM, Corinna Vinschen wrote:
> On Jun  1 08:40, Michael Haubenwallner wrote:
>> Hi,
>> two issues with dlopen here (I'm about to prepare patches):
>> *) The algorithm to combine dll file name variants with the search path
>>    entries needs to be reordered, as in:
>>    - for each dll file name variant:
>>    -   for each search path:
>>    + for each search path entry:
>>    +   for each dll file name variant:
>>          check if useable
> Rationale?  We only need to find one version of the file and there
> usually only is one.  This is mainly for moduled systems like perl,
> python, etc.

While I indeed didn't face a problem here yet, the rationale behind is
that I need to install a large application that provides its own (portable)
package build & management system, including lots of packages probably installed
in the host (Cygwin) system as well, most likely in (slightly) different versions.

When these package management system now does provide a different dll naming
scheme than the host system, like stick to "liblib.dll" rather than "cyglib.dll",
and the application wants to dlopen its own "liblib.dll", currently the host's
"cyglib.dll" is loaded.

> However, if you can speed up the search process ignore the
> question...

This also depends on whether find_exec really is necessary here.

>> *) The directory of the current main executable should be searched
>>    after LD_LIBRARY_PATH and before /usr/bin:/usr/lib.
>>    And PATH should be searched before /usr/bin:/usr/lib as well.
> Checking the executable path and $PATH are Windows concepts.  dlopen
> doesn't do that on POSIX systems and we're not doing that either.

Agreed, but POSIX also does have the concept of embedded RUNPATH,
which is completely missing in Cygwin as far as I can see.
However, there is one path name that can easily serve as minimal
"embedded RUNPATH" - the executable's directory.

This is where I do have a problem right now:

My own /application/bin/python2.7.exe is linked to libpython2.7.dll,
located in /application/bin. Now there is some python script that does
have some - strange enough - cygwin-conditional code that reads:

  import _ctypes
  _ctypes.dlopen("libpython%d.%d.dll" % sys.version_info[:2])

While this is questionable by itself, it really shouldn't load another
libpython2.7.dll than /application/bin/python2.7.exe has already loaded
just because dlopen using a different search algorithm than CreateProcess().

However, when dlopen is about to search some path list, I can imagine to
search the list of already loaded dlls first as well, but I'd prefer to
leave this up to LoadLibrary...

> Having said that, LoadLibrary will search the usual paths.  After 2.5.2,
> we're leaving XP/2003 behind, and then we probably should tighten the
> search algorithm along the lines of
>   AddDllDirectory ("/usr/bin");
>   AddDllDirectory ("/usr/lib");
>   [...]

/me fails to see how this does help with the missing embedded RUNPATH.

>>    For consistency, IMO, when any searched path ends in either
>>    x/bin or x/lib, we should search x/bin:x/lib.
> This might make sense, at least in the direction lib->bin.

Fine with me too.

Side note:
We also use some cl.exe/link.exe wrapper that supports LD_PRELOAD,
LD_LIBRARY_PATH, embedded RUNPATH, as well as lazy loading for both
LoadLibrary and CreateProcess: https://github.com/haubi/parity
Basically I'm wondering why Cygwin doesn't provide that (yet?)...


More information about the Cygwin-developers mailing list