[app porting to cygwin] Invoking DLL exported routine problem

[Peter Novak <peter dot novak at tu-clausthal dot de>]

Hello,
I am struggling with porting my application from Linux to Cygwin and
as I am not 100% at home in Cygwin regarding DLLs and friends, I hope to
get some advice, or hints here. I apologize in advance for a question
looking like a newbie cry for help with "everything", but I tried my
best to study the scarce resources on magic of building DLLs and loading
them in Cygwin without a result.

In short, I have a program using plug-ins implemented as DLLs providing
a fixed interface in a form of a C++ class and C exported routines. The
problem is that loading the DLL and invoking the exported C functions
works perfectly well when loaded from a simple testing code, however
fails when the same code is embedded within a more complex application
setting. I suspect a corrupted stack, or some improper pointer
conversion which I do not understand. Obviously a return type of the
routine seems to be important somehow...

As I said, my application loads DLL plug-ins. However, as I need several
independent instances of the same DLL to be loaded at the same time
(plug-ins can be loaded several times and serve completely independently
with different data fed to them), for each I fork a separate process
which loads a single plug-in DLL and communicates with the main process
via shared memory and mutex signaling. This all works well on Linux and
I managed to compile my application successfully under cygwin as well
(everything including the shared memory communication works and is
ported correctly). What does not work is invocation of certain functions
exported from the DLL plug-ins. That means that dl_openext and dl_sym
work properly, however invocation of routing inside the DLL makes the
subprocess crash. The same code, when put to a standalone testing
program without subprocesses works however correctly.

The interface of the DLL includes code similar to the following:

extern "C" 
{
	unsigned int get_api_version()
	{
		return 123;
	}

	CInterface* register_api()
	{
		CMyInterface pModule = new CMyInterface;
		return pModule;
	}
}

where CMyInterface is a subclass of a purely abstract (all methods are
virtual) root class CMyInterface. All the following operation the
plug-in is to perform are invoked as method calls on the instance of
CInterface. The pattern is modeled according to DLL C++ API from
http://aegisknight.org/cppinterface.html.

The code which loads the DLL uses libtool's ltdl library and invokes the
two functions looks similar to the following (error checking omitted):

	// initialize ltdl
        lt_dlinit();

        // open the library (jztemplate is the name of my library above)
        lt_dlhandle m_hLibrary = lt_dlopenext("cygjztemplate-0");

        // find the KR module API version routine <-- WORKS WELL
        lt_ptr hVersionFunc = lt_dlsym (m_hLibrary, "get_api_version");

	// Invocation of the 
        std::cout << ((TPtrVerFunc)hVersionFunc)() << std::endl;

	// find the interface loading routine <-- WORKS WELL
        lt_ptr hLoadFunc = lt_dlsym (m_hLibrary, "regsiter_api");

	// get the instance of the interface <-- PROBLEM HERE
	CInterface* pModule = ((TPtrLoadFunc) hLoadFunc)();
	
	//...

Signatures of the DLL exported functions are defined as:

	typedef unsigned int (*TPtrVerFunc)();
	typedef CInterface* (*TPtrLoadFunc)();


PROBLEM:
When the code above is invoked from a simple testing program (that code
is wrapped into a main() function) it works without a problem. If
however, it is invoked from a subprocess of my complex application, it
crashes on invocation of the hLoadFunc. However hVersionFunc is invoked
correctly as many times as I call it.

Finally, let me also list the linking options for the main program,
testing program and the library itself.

*** Main program
g++ -DDEBUG -DLOGGING_ON -Wall -DPACKAGELIBDIR=\"/usr/local/lib/jazzyk\"
-g -O2 -L/usr/lib -o app.exe app.o (other *.o files omitted)
-lboost_program_options-gcc-mt -lltdl


*** Testing program:
g++ Test.cpp -o test.exe -I/usr/include/boost-1_33_1/ -lltdl


*** DLL library (as produced by automake/libtool):
/bin/sh ./libtool --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I.  -g
-DJZMODULEDLL   -g -O2 -MT libjztemplate_la-module.lo -MD -MP -MF
.deps/libjztemplate_la-module.Tpo -c -o libjztemplate_la-module.lo `test
-f 'module.cpp' || echo './'`module.cpp

mkdir .libs

g++ -DHAVE_CONFIG_H -I. -g -DJZMODULEDLL -g -O2 -MT
libjztemplate_la-module.lo -MD -MP -MF .deps/libjztemplate_la-module.Tpo
-c module.cpp  -DDLL_EXPORT -DPIC -o .libs/libjztemplate_la-module.o

g++ -DHAVE_CONFIG_H -I. -g -DJZMODULEDLL -g -O2 -MT
libjztemplate_la-module.lo -MD -MP -MF .deps/libjztemplate_la-module.Tpo
-c module.cpp -o libjztemplate_la-module.o >/dev/null 2>&1

mv -f .deps/libjztemplate_la-module.Tpo
.deps/libjztemplate_la-module.Plo

/bin/sh ./libtool --tag=CXX   --mode=link g++  -g -O2 -no-undefined
-version-info 0:0:0  -o libjztemplate.la -rpath /usr/local/lib
libjztemplate_la-module.lo  

g++ -shared -nostdlib   .libs/libjztemplate_la-module.o
-L/usr/lib/gcc/i686-pc-cygwin/3.4.4
-L/usr/lib/gcc/i686-pc-cygwin/3.4.4/../../.. -lstdc++ -lgcc -lcygwin
-luser32 -lkernel32 -ladvapi32 -lshell32 -lgcc   -o
.libs/cygjztemplate-0.dll -Wl,--enable-auto-image-base -Xlinker
--out-implib -Xlinker .libs/libjztemplate.dll.a


For completeness here is the output of nm on the DLL:
65881000 T _get_api_version
65881030 T _register_api

65881380 t __GLOBAL__D_get_api_version
65881370 t __GLOBAL__I_get_api_version

Notice, that for get_api_version, there are also two entries with prefix
__GLOBAL, while for the register_api there's nothing like that. Does
that mean anything? I tried to add other functions to the interface and
invoking such which had only a one entry was also OK. It seems to be
rather the return type which causes problems.

Also note that pointers returned by dl_sym correspond exactly to those
in the first two entries of type "T".

Is there perhaps some issue hidden in the interplay of forked
subprocesses which load DLLs I should know about? Obvioulsy fork on
cygwin is a non-trivial issue itself. And why does libtool generate DLL
which contains for some routines a single entry, but for some several?
Is it significant?

I would be extremely thankful if somebody could direct me to something
suspicious in what I described above, or possibly point out some facts
regarding DLLs in Cygwin which might help me resolve the problem.

Thanks in advance,

Peter Novak.

Attachment: pgp00000.pgp
Description: PGP signature