This is the mail archive of the
cygwin-developers
mailing list for the Cygwin project.
[RFC] Cygwin libstdc++ plan (operator new/delete replacement)
- From: Dave Korn <dave dot korn dot cygwin at googlemail dot com>
- To: cygwin-developers at cygwin dot com
- Date: Sun, 28 Jun 2009 14:47:24 +0100
- Subject: [RFC] Cygwin libstdc++ plan (operator new/delete replacement)
Hi everybody,
Here's a way to implement C++ replacement of the C++ operators new, delete,
etc., etc. (hereafter ONDEE). This is a simpler approach than going the whole
hog with load-time dynamic relinking, and may be a better one for that reason;
on the other hand, it's a little more fragile than having all the support at
the lower end of the toolchain, as it relies on the g++ driver specs being
used when compiling and linking C++. That's probably not a serious downside,
as not using the driver to link is already a "don't do this unless you really
know what you're doing" kind of activity, but I mention it so it can be taken
into consideration.
The heart of the trick is to use LD's --wrap option to redirect all
references to the replaceable ONDEE functions. The cygwin DLL then exports
these wrapper functions, so every reference to an ONDEE function in either
DLLs or EXEs gets redirected to the cygwin DLL.
Meanwhile, we add a struct in _cygwin_crt0_common.cc that records the real
values of whichever ONDEE symbols are referenced in the final link so we know
what overrides were in effect at that time.
We keep a pointer in the DLL's internal per_process struct to the current
set of overrides. At process startup time, this points to the DLL's own
private internal versions of the ONDEE functions; as the application's DLLs
are loaded and perform their crt0 common init, they inherit any defaults that
they don't have overrides for into their own list of overrides, and then
register the resulting set as the global set of overrides.
The application does likewise with its own overrides, so by the time we're
ready to start executing, the full final set of overrides are in place, and
this all happens before static c-tors get called in __main.
The rest of the patch contains little of note. There's a new module,
libstdcxx_wrapper.cc, which contains the wrapper functions; they trivally jump
through the function pointers in the override struct. There are exports for
the wrappers in cygwin.din.
I've added a bit of makefile hackery so that we can selectively remove the
-nostdinc and -nostdincxx flags when compiling particular source files. I
think it's probably important to #include the real "new" header file where
we're doing this and get all the definitive types and definitions from the
compiler's headers. (I still had to go and copy and paste the declarations
into _cygwin_crt0_common.cc because there's no other way to apply the
necessary attributes to them.)
One omission that I'd have to rectify before this could be a submission
rather than an RFC is that I haven't remembered to bump the abi version number
in this patch. There's also some rather long lines I expect I'd better wrap.
Apart from that this is the final shape of a working solution to the
problem. I think it's wholly backward compatible: old applications and DLLs
won't know about or use the redirectors or wrappers. Statically-linked apps -
which are the only apps where ONDEE replacement has ever worked - won't
change. Old dynamically-linked apps will continue to import directly from
libstdc++ or to use any overrides that they linked at final-link time, and
libstdc++ will continue to not be interposed in this case. I do not expect
there to be any requirement for new DLLs or EXEs to be compatible with older
versions of the DLL.
There might just be a corner case with dynamically-linked apps that link
against a mix of new and old DLLs and where the old DLLs supplied overrides.
But since none of this has actually worked correctly ever anyway, I think it's
reasonable to require a recompile. We're breaking ABIs with abandon right now :)
From the GCC side of things, I need do very little except add something like
this
#define CXX_WRAP_SPEC "\
--wrap _Znwj \
--wrap _Znaj \
--wrap _ZdlPv \
--wrap _ZdaPv \
--wrap _ZnwjRKSt9nothrow_t \
--wrap _ZnajRKSt9nothrow_t \
--wrap _ZdlPvRKSt9nothrow_t \
--wrap _ZdaPvRKSt9nothrow_t "
to the linker spec. If we agree this is a reasonable approach to take, I
could rush out a gcc4-4.3.2-3 update to add this very quickly.
So, would everyone be happy to do it this way?
cheers,
DaveK
? cygwin-libstdc-dll.diff
? newlib/autom4te.cache
Index: winsup/Makefile.common
===================================================================
RCS file: /cvs/src/src/winsup/Makefile.common,v
retrieving revision 1.55
diff -p -u -r1.55 Makefile.common
--- winsup/Makefile.common 11 Oct 2005 18:17:59 -0000 1.55
+++ winsup/Makefile.common 28 Jun 2009 13:15:16 -0000
@@ -130,9 +130,9 @@ ifeq (,${findstring $(gcc_libdir),$(CFLA
GCC_INCLUDE:=${subst //,/,-I$(gcc_libdir)/include}
endif
-COMPILE_CXX=$(CXX) $c $(nostdincxx) $(nostdinc) $(ALL_CXXFLAGS) $(GCC_INCLUDE) \
- -fno-rtti -fno-exceptions
-COMPILE_CC=$(CC) $c $(nostdinc) $(ALL_CFLAGS) $(GCC_INCLUDE)
+COMPILE_CXX=$(CXX) $c $(if $($(*F)_STDINCFLAGS),,$(nostdincxx) $(nostdinc)) \
+ $(ALL_CXXFLAGS) $(GCC_INCLUDE) -fno-rtti -fno-exceptions
+COMPILE_CC=$(CC) $c $(if $($(*F)_STDINCFLAGS),,$(nostdinc)) $(ALL_CFLAGS) $(GCC_INCLUDE)
vpath %.a $(cygwin_build):$(w32api_lib):$(newlib_build)/libc:$(newlib_build)/libm
Index: winsup/cygwin/Makefile.in
===================================================================
RCS file: /cvs/src/src/winsup/cygwin/Makefile.in,v
retrieving revision 1.225
diff -p -u -r1.225 Makefile.in
--- winsup/cygwin/Makefile.in 12 Apr 2009 04:14:31 -0000 1.225
+++ winsup/cygwin/Makefile.in 28 Jun 2009 13:15:16 -0000
@@ -145,9 +145,9 @@ DLL_OFILES:=assert.o autoload.o bsdlib.o
fhandler_termios.o fhandler_tty.o fhandler_virtual.o fhandler_windows.o \
fhandler_zero.o flock.o fnmatch.o fork.o fts.o ftw.o getopt.o glob.o \
glob_pattern_p.o globals.o grp.o heap.o hookapi.o inet_addr.o inet_network.o \
- init.o ioctl.o ipc.o kernel32.o localtime.o lsearch.o malloc_wrapper.o \
- minires-os-if.o minires.o miscfuncs.o mktemp.o mmap.o msg.o mount.o \
- net.o netdb.o nfs.o nftw.o ntea.o passwd.o path.o pinfo.o pipe.o \
+ init.o ioctl.o ipc.o kernel32.o libstdcxx_wrapper.o localtime.o lsearch.o \
+ malloc_wrapper.o minires-os-if.o minires.o miscfuncs.o mktemp.o mmap.o msg.o \
+ mount.o net.o netdb.o nfs.o nftw.o ntea.o passwd.o path.o pinfo.o pipe.o \
poll.o posix_ipc.o pthread.o random.o regcomp.o regerror.o regexec.o \
regfree.o registry.o resource.o rexec.o rcmd.o scandir.o sched.o \
sec_acl.o sec_auth.o sec_helper.o security.o select.o sem.o \
@@ -268,6 +268,7 @@ fhandler_windows_CFLAGS:=-fomit-frame-po
fhandler_zero_CFLAGS:=-fomit-frame-pointer
flock_CFLAGS:=-fomit-frame-pointer
grp_CFLAGS:=-fomit-frame-pointer
+libstdcxx_wrapper_CFLAGS:=-fomit-frame-pointer
malloc_CFLAGS:=-fomit-frame-pointer
malloc_wrapper_CFLAGS:=-fomit-frame-pointer
miscfuncs_CFLAGS:=-fomit-frame-pointer
@@ -285,6 +286,10 @@ sysconf_CFLAGS:=-fomit-frame-pointer
uinfo_CFLAGS:=-fomit-frame-pointer
endif
+_cygwin_crt0_common_STDINCFLAGS:=yes
+libstdcxx_wrapper_STDINCFLAGS:=yes
+cxx_STDINCFLAGS:=yes
+
.PHONY: all force dll_ofiles install all_target install_target all_host install_host \
install install-libs install-headers
Index: winsup/cygwin/cxx.cc
===================================================================
RCS file: /cvs/src/src/winsup/cygwin/cxx.cc,v
retrieving revision 1.4
diff -p -u -r1.4 cxx.cc
--- winsup/cygwin/cxx.cc 3 Jan 2009 05:12:20 -0000 1.4
+++ winsup/cygwin/cxx.cc 28 Jun 2009 13:15:16 -0000
@@ -11,9 +11,14 @@ details. */
#if (__GNUC__ >= 3)
#include "winsup.h"
+#include "cygwin-cxx.h"
+
+/* These implementations of operators new and delete are used internally by
+ the DLL, and are kept separate from the user's/libstdc++'s versions by
+ use of LD's --wrap option. */
void *
-operator new (size_t s)
+operator new (std::size_t s)
{
void *p = calloc (1, s);
return p;
@@ -26,7 +31,7 @@ operator delete (void *p)
}
void *
-operator new[] (size_t s)
+operator new[] (std::size_t s)
{
return ::operator new (s);
}
@@ -37,6 +42,34 @@ operator delete[] (void *p)
::operator delete (p);
}
+/* Nothrow versions, provided only for completeness in the fallback array. */
+
+void *
+operator new (std::size_t s, const std::nothrow_t &)
+{
+ void *p = calloc (1, s);
+ return p;
+}
+
+void
+operator delete (void *p, const std::nothrow_t &)
+{
+ free (p);
+}
+
+void *
+operator new[] (std::size_t s, const std::nothrow_t &nt)
+{
+ return ::operator new (s, nt);
+}
+
+void
+operator delete[] (void *p, const std::nothrow_t &nt)
+{
+ ::operator delete (p, nt);
+}
+
+
extern "C" void
__cxa_pure_virtual (void)
{
@@ -52,4 +85,21 @@ extern "C" void
__cxa_guard_release ()
{
}
+
+/* These routines are made available as last-resort fallbacks
+ for the application. Should not be used in practice. */
+
+struct per_process_cxx_malloc default_cygwin_cxx_malloc =
+{
+ &(operator new),
+ &(operator new[]),
+ &(operator delete),
+ &(operator delete[]),
+ &(operator new),
+ &(operator new[]),
+ &(operator delete),
+ &(operator delete[]),
+};
+
+
#endif
Index: winsup/cygwin/cygwin-cxx.h
===================================================================
RCS file: winsup/cygwin/cygwin-cxx.h
diff -N winsup/cygwin/cygwin-cxx.h
--- /dev/null 1 Jan 1970 00:00:00 -0000
+++ winsup/cygwin/cygwin-cxx.h 28 Jun 2009 13:15:16 -0000
@@ -0,0 +1,43 @@
+/* cygwin-cxx.h
+
+ Copyright 2009 Red Hat, Inc.
+
+This file is part of Cygwin.
+
+This software is a copyrighted work licensed under the terms of the
+Cygwin license. Please consult the file "CYGWIN_LICENSE" for
+details. */
+
+#ifndef _CYGWIN_CXX_H
+#define _CYGWIN_CXX_H
+
+#ifdef __cplusplus
+
+/* Files including this header must override -nostdinc++ */
+#include <new>
+
+/* This is an optional struct pointed to by per_process if it exists. */
+struct per_process_cxx_malloc
+{
+ void* (*oper_new) (std::size_t);
+ void* (*oper_new__) (std::size_t);
+ void (*oper_delete) (void *);
+ void (*oper_delete__) (void *);
+ void* (*oper_new_nt) (std::size_t, const std::nothrow_t &);
+ void* (*oper_new___nt) (std::size_t, const std::nothrow_t &);
+ void (*oper_delete_nt) (void *, const std::nothrow_t &);
+ void (*oper_delete___nt) (void *, const std::nothrow_t &);
+};
+
+extern struct per_process_cxx_malloc default_cygwin_cxx_malloc;
+
+#else /* !__cplusplus */
+
+struct per_process_cxx_malloc
+{
+ PVOID dummy[8];
+};
+
+#endif /* __cplusplus */
+
+#endif /* _CYGWIN_CXX_H */
Index: winsup/cygwin/cygwin.din
===================================================================
RCS file: /cvs/src/src/winsup/cygwin/cygwin.din,v
retrieving revision 1.211
diff -p -u -r1.211 cygwin.din
--- winsup/cygwin/cygwin.din 31 Mar 2009 09:42:58 -0000 1.211
+++ winsup/cygwin/cygwin.din 28 Jun 2009 13:15:17 -0000
@@ -1806,3 +1806,11 @@ y1 NOSIGFE
y1f NOSIGFE
yn NOSIGFE
ynf NOSIGFE
+__wrap__ZdaPv NOSIGFE
+__wrap__ZdaPvRKSt9nothrow_t NOSIGFE
+__wrap__ZdlPv NOSIGFE
+__wrap__ZdlPvRKSt9nothrow_t NOSIGFE
+__wrap__Znaj NOSIGFE
+__wrap__ZnajRKSt9nothrow_t NOSIGFE
+__wrap__Znwj NOSIGFE
+__wrap__ZnwjRKSt9nothrow_t NOSIGFE
Index: winsup/cygwin/globals.cc
===================================================================
RCS file: /cvs/src/src/winsup/cygwin/globals.cc,v
retrieving revision 1.2
diff -p -u -r1.2 globals.cc
--- winsup/cygwin/globals.cc 24 Mar 2009 12:18:34 -0000 1.2
+++ winsup/cygwin/globals.cc 28 Jun 2009 13:15:17 -0000
@@ -97,7 +97,8 @@ extern "C"
/* calloc */ calloc,
/* premain */ {NULL, NULL, NULL, NULL},
/* run_ctors_p */ 0,
- /* unused */ {0, 0, 0, 0, 0, 0, 0},
+ /* unused */ {0, 0, 0, 0, 0, 0},
+ /* cxx_malloc */ &default_cygwin_cxx_malloc,
/* UNUSED forkee */ 0,
/* hmodule */ NULL,
/* api_major */ CYGWIN_VERSION_API_MAJOR,
Index: winsup/cygwin/libstdcxx_wrapper.cc
===================================================================
RCS file: winsup/cygwin/libstdcxx_wrapper.cc
diff -N winsup/cygwin/libstdcxx_wrapper.cc
--- /dev/null 1 Jan 1970 00:00:00 -0000
+++ winsup/cygwin/libstdcxx_wrapper.cc 28 Jun 2009 13:15:17 -0000
@@ -0,0 +1,84 @@
+/* libstdcxx_wrapper.cc
+
+ Copyright 2009 Red Hat, Inc.
+
+ Originally written by Dave Korn <dave.korn.cygwin@gmail.com>
+
+This file is part of Cygwin.
+
+This software is a copyrighted work licensed under the terms of the
+Cygwin license. Please consult the file "CYGWIN_LICENSE" for
+details. */
+
+
+/* We provide these stubs to call into a user's
+ provided ONDEE replacement if there is one - otherwise
+ it must fall back to the standard libstdc++ version.
+*/
+
+#include "winsup.h"
+#include "cygwin-cxx.h"
+#include "perprocess.h"
+
+// We are declaring asm names for the functions we define here, as we want
+// to define the wrappers in this file. GCC links everything with wrappers
+// around the standard C++ memory management operators; these are the wrappers,
+// but we want the compiler to know they are the malloc operators and not have
+// it think they're just any old function matching 'extern "C" _wrap_*'.
+extern void *operator new(std::size_t sz) throw (std::bad_alloc) __asm__ ("___wrap__Znwj");
+extern void *operator new[](std::size_t sz) throw (std::bad_alloc) __asm__ ("___wrap__Znaj");
+extern void operator delete(void *p) throw() __asm__ ("___wrap__ZdlPv ");
+extern void operator delete[](void *p) throw() __asm__ ("___wrap__ZdaPv");
+extern void* operator new(std::size_t sz, const std::nothrow_t &nt) throw() __asm__ ("___wrap__ZnwjRKSt9nothrow_t");
+extern void* operator new[](std::size_t sz, const std::nothrow_t &nt) throw() __asm__ ("___wrap__ZnajRKSt9nothrow_t");
+extern void operator delete(void *p, const std::nothrow_t &nt) throw() __asm__ ("___wrap__ZdlPvRKSt9nothrow_t");
+extern void operator delete[](void *p, const std::nothrow_t &nt) throw() __asm__ ("___wrap__ZdaPvRKSt9nothrow_t");
+
+extern void *
+operator new(std::size_t sz) throw (std::bad_alloc)
+{
+ return (*user_data->cxx_malloc->oper_new) (sz);
+}
+
+extern void *
+operator new[](std::size_t sz) throw (std::bad_alloc)
+{
+ return (*user_data->cxx_malloc->oper_new__) (sz);
+}
+
+extern void
+operator delete(void *p) throw()
+{
+ (*user_data->cxx_malloc->oper_delete) (p);
+}
+
+extern void
+operator delete[](void *p) throw()
+{
+ (*user_data->cxx_malloc->oper_delete__) (p);
+}
+
+extern void*
+operator new(std::size_t sz, const std::nothrow_t &nt) throw()
+{
+ return (*user_data->cxx_malloc->oper_new_nt) (sz, nt);
+}
+
+extern void*
+operator new[](std::size_t sz, const std::nothrow_t &nt) throw()
+{
+ return (*user_data->cxx_malloc->oper_new___nt) (sz, nt);
+}
+
+extern void
+operator delete(void *p, const std::nothrow_t &nt) throw()
+{
+ (*user_data->cxx_malloc->oper_delete_nt) (p, nt);
+}
+
+extern void
+operator delete[](void *p, const std::nothrow_t &nt) throw()
+{
+ (*user_data->cxx_malloc->oper_delete___nt) (p, nt);
+}
+
Index: winsup/cygwin/winsup.h
===================================================================
RCS file: /cvs/src/src/winsup/cygwin/winsup.h,v
retrieving revision 1.228
diff -p -u -r1.228 winsup.h
--- winsup/cygwin/winsup.h 7 Apr 2009 12:13:37 -0000 1.228
+++ winsup/cygwin/winsup.h 28 Jun 2009 13:15:17 -0000
@@ -183,6 +183,9 @@ extern "C" int dll_dllcrt0 (HMODULE, per
extern "C" int dll_noncygwin_dllcrt0 (HMODULE, per_process *);
void __stdcall do_exit (int) __attribute__ ((regparm (1), noreturn));
+/* libstdc++ malloc operator wrapper support. */
+extern struct per_process_cxx_malloc default_cygwin_cxx_malloc;
+
/* UID/GID */
void uinfo_init ();
Index: winsup/cygwin/include/sys/cygwin.h
===================================================================
RCS file: /cvs/src/src/winsup/cygwin/include/sys/cygwin.h,v
retrieving revision 1.79
diff -p -u -r1.79 cygwin.h
--- winsup/cygwin/include/sys/cygwin.h 16 Jan 2009 12:17:28 -0000 1.79
+++ winsup/cygwin/include/sys/cygwin.h 28 Jun 2009 13:15:17 -0000
@@ -194,6 +194,8 @@ enum
class MTinterface;
#endif
+struct per_process_cxx_malloc;
+
struct per_process
{
char *initial_sp;
@@ -236,7 +238,8 @@ struct per_process
/* non-zero of ctors have been run. Inherited from parent. */
int run_ctors_p;
- DWORD unused[7];
+ DWORD unused[6];
+ struct per_process_cxx_malloc *cxx_malloc;
/* Non-zero means the task was forked. The value is the pid.
Inherited from parent. */
Index: winsup/cygwin/lib/_cygwin_crt0_common.cc
===================================================================
RCS file: /cvs/src/src/winsup/cygwin/lib/_cygwin_crt0_common.cc,v
retrieving revision 1.16
diff -p -u -r1.16 _cygwin_crt0_common.cc
--- winsup/cygwin/lib/_cygwin_crt0_common.cc 3 Jan 2009 05:12:22 -0000 1.16
+++ winsup/cygwin/lib/_cygwin_crt0_common.cc 28 Jun 2009 13:15:17 -0000
@@ -10,6 +10,23 @@ details. */
#include "winsup.h"
#include "crt0.h"
+#include "cygwin-cxx.h"
+
+// Weaken these declarations so the references don't pull in C++ dependencies unnecessarily.
+#define WEAK __attribute__ ((weak))
+
+// Use asm names to bypass the --wrap that is being applied to redirect all other
+// references to these operators toward the redirectors in the Cygwin DLL; this
+// way we can record what definitions were visible at final link time but still
+// send all calls to the redirectors.
+extern WEAK void *operator new(std::size_t sz) throw (std::bad_alloc) __asm__ ("___real__Znwj");
+extern WEAK void *operator new[](std::size_t sz) throw (std::bad_alloc) __asm__ ("___real__Znaj");
+extern WEAK void operator delete(void *p) throw() __asm__ ("___real__ZdlPv ");
+extern WEAK void operator delete[](void *p) throw() __asm__ ("___real__ZdaPv");
+extern WEAK void* operator new(std::size_t sz, const std::nothrow_t &nt) throw() __asm__ ("___real__ZnwjRKSt9nothrow_t");
+extern WEAK void* operator new[](std::size_t sz, const std::nothrow_t &nt) throw() __asm__ ("___real__ZnajRKSt9nothrow_t");
+extern WEAK void operator delete(void *p, const std::nothrow_t &nt) throw() __asm__ ("___real__ZdlPvRKSt9nothrow_t");
+extern WEAK void operator delete[](void *p, const std::nothrow_t &nt) throw() __asm__ ("___real__ZdaPvRKSt9nothrow_t");
/* Avoid an info message from linker when linking applications. */
extern __declspec(dllimport) struct _reent *_impure_ptr;
@@ -25,6 +42,14 @@ int main (int, char **, char **);
int _fmode;
void _pei386_runtime_relocator ();
+struct per_process_cxx_malloc __cygwin_cxx_malloc =
+{
+ &(operator new), &(operator new[]),
+ &(operator delete), &(operator delete[]),
+ &(operator new), &(operator new[]),
+ &(operator delete), &(operator delete[])
+};
+
/* Set up pointers to various pieces so the dll can then use them,
and then jump to the dll. */
@@ -33,16 +58,16 @@ _cygwin_crt0_common (MainFunc f, per_pro
{
/* This is used to record what the initial sp was. The value is needed
when copying the parent's stack to the child during a fork. */
- DWORD newu;
+ per_process *newu = (per_process *) cygwin_internal (CW_USER_DATA);
int uwasnull;
if (u != NULL)
uwasnull = 0; /* Caller allocated space for per_process structure */
- else if ((newu = cygwin_internal (CW_USER_DATA)) == (DWORD) -1)
+ else if (~0u == (DWORD) newu)
return 0;
else
{
- u = (per_process *) newu; /* Using DLL built-in per_process */
+ u = newu; /* Using DLL built-in per_process */
uwasnull = 1; /* Remember for later */
}
@@ -84,6 +109,27 @@ _cygwin_crt0_common (MainFunc f, per_pro
u->realloc = &realloc;
u->calloc = &calloc;
+ /* Likewise for the C++ memory operators - if any. */
+ if (newu && newu->cxx_malloc)
+ {
+ /* Inherit what we don't override. */
+#define CONDITIONALLY_OVERRIDE(MEMBER) \
+ if (!__cygwin_cxx_malloc.MEMBER) \
+ __cygwin_cxx_malloc.MEMBER = newu->cxx_malloc->MEMBER;
+ CONDITIONALLY_OVERRIDE(oper_new);
+ CONDITIONALLY_OVERRIDE(oper_new__);
+ CONDITIONALLY_OVERRIDE(oper_delete);
+ CONDITIONALLY_OVERRIDE(oper_delete__);
+ CONDITIONALLY_OVERRIDE(oper_new_nt);
+ CONDITIONALLY_OVERRIDE(oper_new___nt);
+ CONDITIONALLY_OVERRIDE(oper_delete_nt);
+ CONDITIONALLY_OVERRIDE(oper_delete___nt);
+ }
+
+ /* Now update the resulting set into the global redirectors. */
+ if (newu)
+ newu->cxx_malloc = &__cygwin_cxx_malloc;
+
/* Setup the module handle so fork can get the path name. */
u->hmodule = GetModuleHandle (0);