This is the mail archive of the cygwin-apps mailing list for the Cygwin project.

Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Re: RFC: cygport cross-compiling APIs

On 7/12/2010 6:19 AM, Yaakov (Cygwin/X) wrote:
I think I'm getting close to nailing down cross-compiling support in

Great! (Is your current state checked in to git master or some other branch?)

Throughout, the prefix=/usr assumption has been removed; *-*-mingw*
hosts use /mingw, everything else is /usr for now.

Are we sure about this, given mingw64's express desire to avoid this $prefix -- specifically because "the other mingw" has historically used it, and "the other mingw" has had its preferences enshrined in the upstream gcc source code?

Do the guys at #mingw64 have any comment on the "proper" prefix that cross compilers should use underneath a sysroot?

(Anyone know any
specific systems where that's not the case?)  Cross-compiled packages
are properly strip(1)ed with the correct binutils.  Libtool fixup puts
non-native DLLs into $CROSS_HOST/prefix/bin

I ask the following question for clarification only:

So by DLL you mean, explicitly, PE/COFF Dynamic Libraries for win32, and NOT the generic "dynamically loaded library" category that can, loosely speaking, include also ELF .so's.

I assume so, because that's the only interpretation that doesn't conflict with this statement:

if not already there, and
doesn't move non-PE libs

Now, for the next bit:

but still removes static modules.

You mean that if a (shared) library -- whether ".dll" or ".so" -- is built using libtool's -module flag (e.g. shouldnotlink=yes in the .la file), then the .a's (and .dll.a's) are removed?

I suppose the "linktime symbolic links" often used with ELF shared libraries aren't even created by 'libtool --mode=install' for modules, so that's a non-issue.

I ask the preceding question for clarification only.

package dependencies for mingw* and linux* cross-compiles are supported.
While I was at it, some initial work was done to make adapting to a
possible future x86_64-pc-cygwin easier as well.


Here's what the API looks like so far:

toolchain.cygclass: for building binutils/gcc/gdb (cross-)compilers
* TOOLCHAIN_TARGET: the target triplet (defined in .cygport BEFORE
"inherit toolchain")
* toolchain_compile(): runs cygconf with --build/host/target and
--with-sysroot, followed by cygmake
* src_compile(): just toolchain_compile; no cygautoreconf (these
packages don't autoreconf easily)

Sounds good.

cross.cygclass: for cross-compiling
* CROSS_HOST: the host triplet (defined in .cygport BEFORE "inherit
* CROSS_SYSROOT: the host sys-root (== /usr/${CROSS_HOST}/sys-root)
* CC/CXX/F77/FC are $host-prefixed
* PKG_CONFIG_LIBDIR and PKG_CONFIG_SYSROOT_DIR are set for pkg-config
* cygconf automatically adds --build/host/target

What about $prefix? Is this handled automatically by cygport (e.g. always /usr, unless $host=mingw* then /mingw?) and can't be overridden?

I'd really like to be able to override $prefix in general, because it would make the gcc-tools-* packages a lot easier.

Also, particularly with respect to the mingw* cross compilers, I'm unsure that all three should all use sys-root/mingw/. Again, what do the #mingw64 folks recommend in this situation?

I implemented the cygconf hook instead of a separate cross_compile() so
that other cygconf-based cygclasses would benefit.  For example, a
mingw64-x86_64-glib2.0 would "inherit cross gnome2", where
gnome2_compile calls cygconf.

I see.

Right now my dilemma is with the cross.cygclass install functions.  I'm
not sure whether cyginstall/do*/new* should just flip personality to
install under $CROSS_SYSROOT (much like cygconf does), or whether to
implement separate docross{bin,include,lib,...} functions.  My reason
for considering the former is that it would reduce duplicate code and
avoid easy mistakes (e.g. installing cross-compiled stuff into native
directories).  It would also make it easier to take a native .cygport,
rename it, define CROSS_HOST and inherit cross, and build.

Another argument is similar to the one above: what about other .cygclasses that use these functions? If I'm required to use docrossbin, I can easily do that in my own .cygport -- but not if I inherit another .cygclass in addition to 'cross', and THAT .cygclass uses the (original) dobin.

As far as the package .cygport calling dodoc...I think we'll have to take that as it goes. The NOCROSS idea below might help here.

OTOH I'm not
sure whether the more manually-controlled dodir, exe/insinto, doexe/ins,
and newexe/ins should also imply $CROSS_SYSROOT or not:

Well, you know me: I'm always in favor of giving the end user of cygport more power, and less authoritarian control by cygport itself. Boat-on-water with channel markers, rather than car-on-road with concrete guardrails.

However, in THIS case...see below.

IOW, is there
any case where a cross-compiled package would legitimately install
anything outside of $CROSS_SYSROOT (besides the standard docs).  I also
wonder if that would itself be confusing.

Well, there's the mingw-runtime stuff, which is still used/needed by gcc3's -mno-cygwin. Plus there's the particular version of w32api that is used by cygwin's own gcc (and gcc3's -mno-cygwin) and is installed into /usr/include/w32api + /usr/lib/w32api.

I assume this version would NOT be compiled using the i686-pc-mingw32 toolchain, but instead by the cygwin one (e.g. its cygport -- if it is actually built USING cygport -- wouldn't inherit cross), so that's probably not an issue.

Other than that, the only "mingw-ish" stuff we actually ship are, IIRC:

     -- special case
  the mingw-{zlib,bzip2,xz,libgcrypt,libgpg-error} stuff
     -- should be handled by the new cross cygclass with no problems
  the tools in the cygwin package, like strace.exe and cygcheck.exe
     -- no changes; handled by cygwin gcc with -nostdlib? and not
        built using cygport anyway.

So...there aren't that many cross-compiled packages that are part of the cygwin distribution -- and few would use any additional cygclasses, other than 'cross', so...manual docross* is not that difficult.

However, cygport is supposed to make "doing it right" easy. So...

I think, on balance, it'd be better to have the existing do* scripts change their behavior when 'cross' has been inherited, rather than requiring the use of a separate set of functions.

The best of all worlds would be to have a local override, that can be set per-command:

inherit cross gtk


   # goes in /usr/$target/sys-root/$prefix/something
   dodir something

   # goes in  /usr/$target/sys-root/$prefix/bin
   dobin cross_app1

   # goes in /$other_prefix/bin
   # e.g. $other_prefix might be /usr, while the
   #      'cross.cygclass'-enforced "$prefix" is actually
   #      /mingw
   NOCROSS=$other_prefix dobin special_app

   # goes in  /usr/$target/sys-root/$prefix/bin
   # NOCROSS gets unset by previous command?
   dobin cross_app2

This scheme can't address the issues related to OTHER inherited cygclasses which use the do* commands. But then, we probably don't care about that very much. .cygclasses are supposed to be black boxes that 'just work'(tm); their internal operation shouldn't be second guessed.

As for the package naming scheme -- which is not being enforced by
cygport itself -- my thoughts are OS[-CPU]-PACKAGE, IOW:



Remember that in mingw64-speak, the "mingw64-crt-headers" is a completely separate package than "mingw64-crt-lib" -- there's no "top-level" configure typing them together, AFAIR; so it'd be rather difficult to munge them together into a single package named "mingw64-*-runtime".

-headers contains all of the .h's for both what we call "mingw-runtime" as well as "w32api". -lib contains all of the static, import, and DLLs for both, as well -- but no headers.

So, rather than mingw64-$CPU-runtime, I'd suggest sticking with mingw64's own nomenclature, and using:


Also, you've made no provision here for allowing to install the runtime DLLs (from mingw64-$CPU-crt-lib and from mingw64-$CPU-gcc itself) as separate packages. I know, you say this is a deliberate decision; see below.

Or, for that matter, splitting mingw64-$CPU-gcc into separate languages. I *really* don't want to have to download Fortran, ObjC, (and perhaps Ada and java?) when all I really want is C++.

Finally, you appear to be deliberately precluding any POSSIBILITY of providing a multilib gcc. I know cygport can't yet support it, but we might find it necessary or desirable down the road to add that support. However, this naming scheme won't EVER support that -- unless (e.g.) this hypothetical, future multilib version of


ACTUALLY included the $CPU stuff as well as the $OTHER-CPU (or $OTHER-ABI) stuff in */lib32 or */lib64 (or: */o64 or */n64). Maybe that's the right thing to do, but somehow it just seems odd.

Aside, and thinking-out-loud:

This is another reason for all three of our mingw-ish cross compilers to use a $prefix OTHER than /mingw underneath sys-root: that way, the following would -- I think -- work (see HERE:)

  +- usr
      +- bin   - Links to cross compiler toolchain
      +- i686-pc-mingw32   ( stuff)
      |   |
      |   +- sys-root  - root for cross compiled binaries
      |       |
      |       +- mingw32
      |           |
      |           +- bin     - cross-compiled binaries & runtime DLL
      |           +- doc     - documentation
      |           +- include - include files for cross compiled libs
      |           +- lib     - cross-compiled static and link libraries
      |           |   |
      |           |   +- pkgconfig
      |           |
      |           +- share
      |               |
      |               +- man
      +- i686-w64-mingw32   (mingw64 32-bit stuff)
      |   |
      |   +- sys-root  - root for cross compiled binaries
      |       |
      |       +- mingw64-32
      |       |   |
      |       |   +- bin     - cross-compiled binaries & runtime DLL
      |       |   +- doc     - documentation
      |       |   +- include - include files for cross compiled libs
      |       |   +- lib     - cross-compiled static and link libraries
      |       |   |   |
      |       |   |   +- pkgconfig
      |       |   |
      |       |   +- share
      |       |       |
      |       |       +- man
      |       |
HERE: |       +- mingw64 -> symlink to
      |                     /usr/x86_64-w64-mingw32/sys-root/mingw64
      +- x86_64-w64-mingw32   (mingw64 64bit stuff)
          +- sys-root  - root for cross compiled binaries
              +- mingw64
              |   |
              |   +- bin     - cross-compiled binaries & runtime DLL
              |   +- doc     - documentation
              |   +- include - include files for cross compiled libs
              |   +- lib     - cross-compiled static and link libraries
              |   |   |
              |   |   +- pkgconfig
              |   |
              |   +- share
              |       |
              |       +- man
HERE:         +- mingw64-32 -> symlink to

Since nobody is actually using the hardcoded /mingw "$prefix" there'd be no conflict in this scenario between the -m32 and -m64 mode of a hypothetical future multilib compiler.

I don't know if the $(MULTILIBDIR) stuff would cause any problems in this scenario; right now, libtool doesn't support that (everything goes into $prefix/lib, regardless) although there have been a few patches posted to libtool-patches in the last week.

If libtool were better at MULTILIB, then perhaps the symlinking would need to occur at a "deeper" level:

      +- i686-w64-mingw32   (mingw64 32-bit stuff)
      |   |
      |   +- sys-root  - root for cross compiled binaries
      |       |
      |       +- mingw64-32
      |       |   |
      |       |   +- bin     - cross-compiled binaries & runtime DLL
      |       |   |
      |       |   +- bin64   - symlink to
      |       |            /usr/x86_64-w64-mingw32/sys-root/mingw64/bin
      |       |
      |       |   +- doc     - documentation
      |       |   +- include - include files for cross compiled libs
      |       |   +- lib     - cross-compiled static and link libraries
      |       |   |   |
      |       |   |   +- pkgconfig
      |       |   |
      |       |   +- lib64   - symlink to

In *that* case, (a) the "conflicting /mingw $prefix" problem goes away, so all three cross compilers could use the same sys-root $prefix, but (b) there are, as you've discovered, additional complications. Like setting PKGCONFIG_DIR appropriately, and invoking the correct bin[32|64]/foo-config scripts during configure...These questions all get more complicated in this "deep linking" scenario.

Maybe THAT is yet another reason to defer multilib until later (if ever); apparently libtool still needs some help with the concept.

Which is why I say, *defer* this until later -- don't try to solve these multilib problems right now. But also, don't deliberately preclude various possible solutions, by package nomenclature whose that isn't expressive enough, or poorly chosen sys-root $prefix values.

Again, any idea how the #mingw64 guys recommend approaching sys-root $prefixes, as distinct from the install prefix of native compilers and compiled libs/apps?


That [leave room for multilib in the nomenclature, even if current toolchains are single-lib] was the thinking behind the original nomenclature JonY and I were trying to develop, here:

Whether this specific "-tc64-/-tc32-" + "-m64-/-m32-" nomenclature is used, or a some different one sufficiently expressive to allow for multilib, doesn't matter to me.

I just wonder if we shouldn't leave some place in the nomenclature for distinguishing between the "normal" $CPU of the toolchain, and all the stuff related to that, and stuff related to any other -mNN mode (e.g. "sufficiently expressive") Not just for mingw*, but also perhaps for other cross toolchains *down the road*, if/when we decide that cygport and *our packaging/naming policies* should allow multilib.

Say, a cygwin-$hosted solaris-$target cross compiler with 32/64 bit multilib support. Or a cygwin-$hosted IRIX-$target one with o64 and n64 ABI support.

Now, as I said above, maybe the correct answer is "don't try to split it up: the multilib "mingw64-crt-lib" should include whatever is necessary in both sys-root/prefix/lib and sys-root/prefix/lib$MULTI; ditto sys-root/prefix/bin and ...bin$MULTI, all in one happy package. Ditto for the other "core" components of a toolchain.

If you do that, then mingw64-$CPU-$PKG would work fine (but might be a little confusing, since it could contain $OTHER-CPU stuff).

But this is all "cygwin policy" (e.g. "how should I name my packages that I ITP'd") and not cygport-specific. So package *naming* doesn't directly impinge on the design or implementation of cygport(1) -- only on how JonY's ultimate cygport(5) scripts are written.

linux-i686-glibc (not built yet; borrowed from Fedora for bootstrapping)
linux-i686-kernel-headers (ditto)

(none of these are built yet)

Right, well, I'll obviously handle the stuff I currently maintain: the mingw-{zlib,bzip2,xz,libgcrypt,libgpg-error} stuff. And I assume DaveK would do the mingw-{binutils,gcc} ones, since he's already volunteered for that (unless he's changed his mind?).

And Chris S. would probably handle the mingw-runtime [1] (and mingw-w32api [2], plus (cygwin) w32api [3]) as needed -- since those are "his" packages right now.

Naturally Chris S. and DaveK would need to coordinate; after they are done I can do "mine".

[1] where the "new" mingw-runtime (derived from winsup/mingw)
    would go in
    and not
       /usr/bin, /usr/include/mingw, /usr/lib/mingw

[2] where mingw-w32api (derived from winsup/w32api) would go in
    and not
       /usr/include/w32api, /usr/lib/w32api

[3] where this w32api (derived from winsup/w32api) would go in
       /usr/include/w32api and /usr/lib/w32api
    just like our current version.

And so on.  For simplicity, none of these are multilib, and the
conflicting binutils/gcc locales/info/etc. are removed, for reasons
stated in earlier discussions.

Ack. For now, and maybe forever wrt $target and mingw64 $target(s). Probably, tho, the no-multilib "policy" will be hard to maintain if cygwin starts shipping cross compilers targeting other $hosts like Solaris, IRIX, etc. Not that I have any burning desire to ship or maintain those myself -- but somebody out there might volunteer in the future.

I just don't like deliberately hardcoding a possibly-subject-to-change policy discussion into a difficult-to-reverse nomenclature.

Note that I'm *not* breaking out separate lib/devel packages, because if
you think about the reasons for these splits with native packages,
within the context of cross-compiling they doesn't make sense.

No, I disagree with this. mingw-bzip2 provides, in toto, the DLL, headers, static/link libs, and utility apps.

Nobody who ONLY wants to build setup.exe using the new cross compiler cares about the DLLs (because setup.exe uses -static) nor do they care about the apps. OTOH, those elements ARE part of the cross compiled "sys-root" that, if someone were to 'repackage' for independent deployment selected contents of sysroot/$prefix/bin, they would need. Since we ship a mingw-bzip2 complex of packages, why should (a) the first group have to download and install everything, if they only need the headers and static libs but everything were bundled into one package, or (b) the second group have to recompile mingw-bzip2 themselves if we strip the "unnecessary" parts from a single mingw-bzip2 package?

The same argument holds with regards to the gcc language runtimes. Maybe somebody is interested not in cross compiling anything themselves, but using (from a private, or corporate) setup.exe repo some native win32 tools that are meant to be installed into a cygwin-hosted mingw sysroot. Shouldn't they be able to get our libgcc_s-1-sjlj.dll without installing the entire cross compiler?

> Comments welcome, but let's not bikeshed this.

Well, nomenclature and installation locations are probably the ultimate bikeshed topic; but getting it RIGHT is important, if we don't want to have to revisit this topic again and again every year or two.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]