Re: RFC: cygport cross-compiling APIs

[Charles Wilson <cygwin at cwilson dot fastmail dot fm>]

On 7/12/2010 6:19 AM, Yaakov (Cygwin/X) wrote:
> I think I'm getting close to nailing down cross-compiling support in
> cygport.

Great! (Is your current state checked in to git master or some other 
branch?)

> Throughout, the prefix=/usr assumption has been removed; *-*-mingw*
> hosts use /mingw, everything else is /usr for now.

Are we sure about this, given mingw64's express desire to avoid this 
$prefix -- specifically because "the other mingw" has historically used 
it, and "the other mingw" has had its preferences enshrined in the 
upstream gcc source code?

Do the guys at #mingw64 have any comment on the "proper" prefix that 
cross compilers should use underneath a sysroot?

> (Anyone know any
> specific systems where that's not the case?)  Cross-compiled packages
> are properly strip(1)ed with the correct binutils.  Libtool fixup puts
> non-native DLLs into $CROSS_HOST/prefix/bin

I ask the following question for clarification only:

So by DLL you mean, explicitly, PE/COFF Dynamic Libraries for win32, and 
NOT the generic "dynamically loaded library" category that can, loosely 
speaking, include also ELF .so's.

I assume so, because that's the only interpretation that doesn't 
conflict with this statement:

> if not already there, and
> doesn't move non-PE libs

Now, for the next bit:

> but still removes static modules.

You mean that if a (shared) library -- whether ".dll" or ".so" -- is 
built using libtool's -module flag (e.g. shouldnotlink=yes in the .la 
file), then the .a's (and .dll.a's) are removed?

I suppose the "linktime symbolic links" often used with ELF shared 
libraries aren't even created by 'libtool --mode=install' for modules, 
so that's a non-issue.

I ask the preceding question for clarification only.

> Also,
> package dependencies for mingw* and linux* cross-compiles are supported.
> While I was at it, some initial work was done to make adapting to a
> possible future x86_64-pc-cygwin easier as well.

Neato.

> Here's what the API looks like so far:
>
> toolchain.cygclass: for building binutils/gcc/gdb (cross-)compilers
> * TOOLCHAIN_TARGET: the target triplet (defined in .cygport BEFORE
> "inherit toolchain")
> * toolchain_compile(): runs cygconf with --build/host/target and
> --with-sysroot, followed by cygmake
> * src_compile(): just toolchain_compile; no cygautoreconf (these
> packages don't autoreconf easily)

Sounds good.

> cross.cygclass: for cross-compiling
> * CROSS_HOST: the host triplet (defined in .cygport BEFORE "inherit
> cross")
> * CROSS_SYSROOT: the host sys-root (== /usr/${CROSS_HOST}/sys-root)
> * CC/CXX/F77/FC are $host-prefixed
> * PKG_CONFIG_LIBDIR and PKG_CONFIG_SYSROOT_DIR are set for pkg-config
> * cygconf automatically adds --build/host/target

What about $prefix?  Is this handled automatically by cygport (e.g. 
always /usr, unless $host=mingw* then /mingw?) and can't be overridden?

I'd really like to be able to override $prefix in general, because it 
would make the gcc-tools-* packages a lot easier.

Also, particularly with respect to the mingw* cross compilers, I'm 
unsure that all three should all use sys-root/mingw/.  Again, what do 
the #mingw64 folks recommend in this situation?

> I implemented the cygconf hook instead of a separate cross_compile() so
> that other cygconf-based cygclasses would benefit.  For example, a
> mingw64-x86_64-glib2.0 would "inherit cross gnome2", where
> gnome2_compile calls cygconf.

I see.

> Right now my dilemma is with the cross.cygclass install functions.  I'm
> not sure whether cyginstall/do*/new* should just flip personality to
> install under $CROSS_SYSROOT (much like cygconf does), or whether to
> implement separate docross{bin,include,lib,...} functions.  My reason
> for considering the former is that it would reduce duplicate code and
> avoid easy mistakes (e.g. installing cross-compiled stuff into native
> directories).  It would also make it easier to take a native .cygport,
> rename it, define CROSS_HOST and inherit cross, and build.

Another argument is similar to the one above: what about other 
.cygclasses that use these functions?  If I'm required to use 
docrossbin, I can easily do that in my own .cygport -- but not if I 
inherit another .cygclass in addition to 'cross', and THAT .cygclass 
uses the (original) dobin.

As far as the package .cygport calling dodoc...I think we'll have to 
take that as it goes. The NOCROSS idea below might help here.

> OTOH I'm not
> sure whether the more manually-controlled dodir, exe/insinto, doexe/ins,
> and newexe/ins should also imply $CROSS_SYSROOT or not:

Well, you know me: I'm always in favor of giving the end user of cygport 
more power, and less authoritarian control by cygport itself. 
Boat-on-water with channel markers, rather than car-on-road with 
concrete guardrails.

However, in THIS case...see below.

> IOW, is there
> any case where a cross-compiled package would legitimately install
> anything outside of $CROSS_SYSROOT (besides the standard docs).  I also
> wonder if that would itself be confusing.

Well, there's the mingw-runtime stuff, which is still used/needed by 
gcc3's -mno-cygwin.  Plus there's the particular version of w32api that 
is used by cygwin's own gcc (and gcc3's -mno-cygwin) and is installed 
into /usr/include/w32api + /usr/lib/w32api.

I assume this version would NOT be compiled using the i686-pc-mingw32 
toolchain, but instead by the cygwin one (e.g. its cygport -- if it is 
actually built USING cygport -- wouldn't inherit cross), so that's 
probably not an issue.

Other than that, the only "mingw-ish" stuff we actually ship are, IIRC:

  setup.exe
     -- special case
  the mingw-{zlib,bzip2,xz,libgcrypt,libgpg-error} stuff
     -- should be handled by the new cross cygclass with no problems
  the tools in the cygwin package, like strace.exe and cygcheck.exe
     -- no changes; handled by cygwin gcc with -nostdlib? and not
        built using cygport anyway.

So...there aren't that many cross-compiled packages that are part of the 
cygwin distribution -- and few would use any additional cygclasses, 
other than 'cross', so...manual docross* is not that difficult.

However, cygport is supposed to make "doing it right" easy. So...

I think, on balance, it'd be better to have the existing do* scripts 
change their behavior when 'cross' has been inherited, rather than 
requiring the use of a separate set of functions.


The best of all worlds would be to have a local override, that can be 
set per-command:

inherit cross gtk

src_install()
{
   cyginstall

   # goes in /usr/$target/sys-root/$prefix/something
   dodir something

   # goes in  /usr/$target/sys-root/$prefix/bin
   dobin cross_app1

   # goes in /$other_prefix/bin
   # e.g. $other_prefix might be /usr, while the
   #      'cross.cygclass'-enforced "$prefix" is actually
   #      /mingw
   NOCROSS=$other_prefix dobin special_app

   # goes in  /usr/$target/sys-root/$prefix/bin
   # NOCROSS gets unset by previous command?
   dobin cross_app2
}

This scheme can't address the issues related to OTHER inherited 
cygclasses which use the do* commands.  But then, we probably don't care 
about that very much.  .cygclasses are supposed to be black boxes that 
'just work'(tm); their internal operation shouldn't be second guessed.

> As for the package naming scheme -- which is not being enforced by
> cygport itself -- my thoughts are OS[-CPU]-PACKAGE, IOW:
>
> mingw64-i686-binutils
> mingw64-i686-gcc
> mingw64-i686-pthreads
> mingw64-i686-runtime
> mingw64-i686-zlib
>
> mingw64-x86_64-binutils
> mingw64-x86_64-gcc
> mingw64-x86_64-pthreads
> mingw64-x86_64-runtime
> mingw64-x86_64-zlib

Remember that in mingw64-speak, the "mingw64-crt-headers" is a 
completely separate package than "mingw64-crt-lib" -- there's no 
"top-level" configure typing them together, AFAIR; so it'd be rather 
difficult to munge them together into a single package named 
"mingw64-*-runtime".

-headers contains all of the .h's for both what we call "mingw-runtime" 
as well as "w32api".  -lib contains all of the static, import, and DLLs 
for both, as well -- but no headers.

So, rather than mingw64-$CPU-runtime, I'd suggest sticking with 
mingw64's own nomenclature, and using:

mingw64-$CPU-crt-headers
mingw64-$CPU-crt-lib



Also, you've made no provision here for allowing to install the runtime 
DLLs (from mingw64-$CPU-crt-lib and from mingw64-$CPU-gcc itself) as 
separate packages.  I know, you say this is a deliberate decision; see 
below.

Or, for that matter, splitting mingw64-$CPU-gcc into separate languages. 
I *really* don't want to have to download Fortran, ObjC, (and perhaps 
Ada and java?) when all I really want is C++.


Finally, you appear to be deliberately precluding any POSSIBILITY of 
providing a multilib gcc. I know cygport can't yet support it, but we 
might find it necessary or desirable down the road to add that support. 
However, this naming scheme won't EVER support that -- unless (e.g.) 
this hypothetical, future multilib version of

   mingw64-$CPU-crt-lib

ACTUALLY included the $CPU stuff as well as the $OTHER-CPU (or 
$OTHER-ABI) stuff in  */lib32 or */lib64 (or: */o64 or */n64).  Maybe 
that's the right thing to do, but somehow it just seems odd.

<aside>
Aside, and thinking-out-loud:

This is another reason for all three of our mingw-ish cross compilers to 
use a $prefix OTHER than /mingw underneath sys-root: that way, the 
following would -- I think -- work (see HERE:)

[root]
  +- usr
      |
      +- bin   - Links to cross compiler toolchain
      |
      +- i686-pc-mingw32   (mingw32.org stuff)
      |   |
      |   +- sys-root  - root for cross compiled binaries
      |       |
      |       +- mingw32
      |           |
      |           +- bin     - cross-compiled binaries & runtime DLL
      |           +- doc     - documentation
      |           +- include - include files for cross compiled libs
      |           +- lib     - cross-compiled static and link libraries
      |           |   |
      |           |   +- pkgconfig
      |           |
      |           +- share
      |               |
      |               +- man
      |
      +- i686-w64-mingw32   (mingw64 32-bit stuff)
      |   |
      |   +- sys-root  - root for cross compiled binaries
      |       |
      |       +- mingw64-32
      |       |   |
      |       |   +- bin     - cross-compiled binaries & runtime DLL
      |       |   +- doc     - documentation
      |       |   +- include - include files for cross compiled libs
      |       |   +- lib     - cross-compiled static and link libraries
      |       |   |   |
      |       |   |   +- pkgconfig
      |       |   |
      |       |   +- share
      |       |       |
      |       |       +- man
      |       |
HERE: |       +- mingw64 -> symlink to
      |                     /usr/x86_64-w64-mingw32/sys-root/mingw64
      |
      +- x86_64-w64-mingw32   (mingw64 64bit stuff)
          |
          +- sys-root  - root for cross compiled binaries
              |
              +- mingw64
              |   |
              |   +- bin     - cross-compiled binaries & runtime DLL
              |   +- doc     - documentation
              |   +- include - include files for cross compiled libs
              |   +- lib     - cross-compiled static and link libraries
              |   |   |
              |   |   +- pkgconfig
              |   |
              |   +- share
              |       |
              |       +- man
              |
HERE:         +- mingw64-32 -> symlink to
                               /usr/i686-w64-mingw32/sys-root/mingw64-32

Since nobody is actually using the hardcoded /mingw "$prefix" there'd be 
no conflict in this scenario between the -m32 and -m64 mode of a 
hypothetical future multilib compiler.

I don't know if the $(MULTILIBDIR) stuff would cause any problems in 
this scenario; right now, libtool doesn't support that (everything goes 
into $prefix/lib, regardless) although there have been a few patches 
posted to libtool-patches in the last week.

If libtool were better at MULTILIB, then perhaps the symlinking would 
need to occur at a "deeper" level:

      +- i686-w64-mingw32   (mingw64 32-bit stuff)
      |   |
      |   +- sys-root  - root for cross compiled binaries
      |       |
      |       +- mingw64-32
      |       |   |
      |       |   +- bin     - cross-compiled binaries & runtime DLL
      |       |   |
      |       |   +- bin64   - symlink to
      |       |            /usr/x86_64-w64-mingw32/sys-root/mingw64/bin
      |       |
      |       |   +- doc     - documentation
      |       |   +- include - include files for cross compiled libs
      |       |   +- lib     - cross-compiled static and link libraries
      |       |   |   |
      |       |   |   +- pkgconfig
      |       |   |
      |       |   +- lib64   - symlink to
                           /usr/x86_64-w64-mingw32/sys-root/mingw64/lib

In *that* case, (a) the "conflicting /mingw $prefix" problem goes away, 
so all three cross compilers could use the same sys-root $prefix, but 
(b) there are, as you've discovered, additional complications. Like 
setting PKGCONFIG_DIR appropriately, and invoking the correct 
bin[32|64]/foo-config scripts during configure...These questions all get 
more complicated in this "deep linking" scenario.


Maybe THAT is yet another reason to defer multilib until later (if 
ever); apparently libtool still needs some help with the concept.



Which is why I say, *defer* this until later -- don't try to solve these 
multilib problems right now.  But also, don't deliberately preclude 
various possible solutions, by package nomenclature whose that isn't 
expressive enough, or poorly chosen sys-root $prefix values.

Again, any idea how the #mingw64 guys recommend approaching sys-root 
$prefixes, as distinct from the install prefix of native compilers and 
compiled libs/apps?

</aside>

That [leave room for multilib in the nomenclature, even if current 
toolchains are single-lib] was the thinking behind the original 
nomenclature JonY and I were trying to develop, here:
http://cygwin.com/ml/cygwin-apps/2010-06/msg00208.html

Whether this specific "-tc64-/-tc32-" + "-m64-/-m32-" nomenclature is 
used, or a some different one sufficiently expressive to allow for 
multilib, doesn't matter to me.

I just wonder if we shouldn't leave some place in the nomenclature for 
distinguishing between the "normal" $CPU of the toolchain, and all the 
stuff related to that, and stuff related to any other -mNN mode (e.g. 
"sufficiently expressive")  Not just for mingw*, but also perhaps for 
other cross toolchains *down the road*, if/when we decide that cygport 
and *our packaging/naming policies* should allow multilib.

Say, a cygwin-$hosted solaris-$target cross compiler with 32/64 bit 
multilib support.  Or a cygwin-$hosted IRIX-$target one with o64 and n64 
ABI support.



Now, as I said above, maybe the correct answer is "don't try to split it 
up: the multilib "mingw64-crt-lib" should include whatever is necessary 
in both sys-root/prefix/lib and sys-root/prefix/lib$MULTI; ditto 
sys-root/prefix/bin and ...bin$MULTI, all in one happy package. Ditto 
for the other "core" components of a toolchain.

If you do that, then mingw64-$CPU-$PKG would work fine (but might be a 
little confusing, since it could contain $OTHER-CPU stuff).


But this is all "cygwin policy" (e.g. "how should I name my packages 
that I ITP'd") and not cygport-specific. So package *naming* doesn't 
directly impinge on the design or implementation of cygport(1) -- only 
on how JonY's ultimate cygport(5) scripts are written.

> linux-i686-binutils
> linux-i686-gcc
> linux-i686-glibc (not built yet; borrowed from Fedora for bootstrapping)
> linux-i686-kernel-headers (ditto)
> linux-i686-zlib
>
> (none of these are built yet)
> mingw-binutils
> mingw-bzip2
> mingw-gcc
> mingw-runtime

Right, well, I'll obviously handle the stuff I currently maintain: the 
mingw-{zlib,bzip2,xz,libgcrypt,libgpg-error} stuff. And I assume DaveK 
would do the mingw-{binutils,gcc} ones, since he's already volunteered 
for that (unless he's changed his mind?).

And Chris S. would probably handle the mingw-runtime [1] (and 
mingw-w32api [2], plus (cygwin) w32api [3]) as needed -- since those are 
"his" packages right now.

Naturally Chris S. and DaveK would need to coordinate; after they are 
done I can do "mine".

[1] where the "new" mingw-runtime (derived from winsup/mingw)
    would go in
       /usr/i686-pc-mingw32/sys-root/mingw*/{include,lib,bin}
    and not
       /usr/bin, /usr/include/mingw, /usr/lib/mingw

[2] where mingw-w32api (derived from winsup/w32api) would go in
       /usr/i686-pc-mingw32/sys-root/mingw*/{include,lib,runtime}
    and not
       /usr/include/w32api, /usr/lib/w32api

[3] where this w32api (derived from winsup/w32api) would go in
       /usr/include/w32api and /usr/lib/w32api
    just like our current version.

> And so on.  For simplicity, none of these are multilib, and the
> conflicting binutils/gcc locales/info/etc. are removed, for reasons
> stated in earlier discussions.

Ack. For now, and maybe forever wrt mingw.org $target and mingw64 
$target(s).  Probably, tho, the no-multilib "policy" will be hard to 
maintain if cygwin starts shipping cross compilers targeting other 
$hosts like Solaris, IRIX, etc.  Not that I have any burning desire to 
ship or maintain those myself -- but somebody out there might volunteer 
in the future.

I just don't like deliberately hardcoding a possibly-subject-to-change 
policy discussion into a difficult-to-reverse nomenclature.

> Note that I'm *not* breaking out separate lib/devel packages, because if
> you think about the reasons for these splits with native packages,
> within the context of cross-compiling they doesn't make sense.

No, I disagree with this.  mingw-bzip2 provides, in toto, the DLL, 
headers, static/link libs, and utility apps.

Nobody who ONLY wants to build setup.exe using the new cross compiler 
cares about the DLLs (because setup.exe uses -static) nor do they care 
about the apps.  OTOH, those elements ARE part of the cross compiled 
"sys-root" that, if someone were to 'repackage' for independent 
deployment selected contents of sysroot/$prefix/bin, they would need. 
Since we ship a mingw-bzip2 complex of packages, why should (a) the 
first group have to download and install everything, if they only need 
the headers and static libs but everything were bundled into one 
package, or (b) the second group have to recompile mingw-bzip2 
themselves if we strip the "unnecessary" parts from a single mingw-bzip2 
package?

The same argument holds with regards to the gcc language runtimes. 
Maybe somebody is interested not in cross compiling anything themselves, 
but using (from a private, or corporate) setup.exe repo some native 
win32 tools that are meant to be installed into a cygwin-hosted mingw 
sysroot.  Shouldn't they be able to get our libgcc_s-1-sjlj.dll without 
installing the entire cross compiler?

> Comments welcome, but let's not bikeshed this.

Well, nomenclature and installation locations are probably the ultimate 
bikeshed topic; but getting it RIGHT is important, if we don't want to 
have to revisit this topic again and again every year or two.

--
Chuck