This is the mail archive of the cygwin-apps mailing list for the Cygwin project.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Re: cygport cross-compiling beta1


On 7/23/2010 1:54 AM, Yaakov (Cygwin/X) wrote:
> On Thu, 2010-07-22 at 22:41 -0400, Charles Wilson wrote: 
>> OK, so the libtool stuff is still percolating. Paolo just posted his
>> patch series earlier today, so I haven't had a chance to test it out
>> yet. However, it APPEARS after a cursory glance to work like this:
>>
>> 1) you configure stuff with --prefix=${CROSS_SYSROOT}${prefix}
> 
> Not according to his sysroot.at (patch 5/8).  He adds a new configure
> option, so you configure with --with-sysroot=${CROSS_SYSROOT} --prefix=
> ${prefix}.

Ah, you're right.  I'm sure this will make more sense once I get a
chance to actually test it.  Looks like Sunday afternoon.

>> The reason I think this is, I'm concerned about *hardcoding* the
>> $sysroot$prefix values into the compiled binaries! (e.g. 'strings
>> foo.exe | grep $sysroot' might return non-empty, and that would be bad,
>> when foo.exe is deployed on the $host system in $prefix.
> 
> If $sysroot needs to be prepended to $prefix, then that strengthens the
> argument for considering cross-compiling-for-sysroot and
> cross-compiling-for-host two separate use cases.

Ah, but I was wrong, so my incorrect analysis can hardly be said to
strengthen anything.

What does gentoo do with cross compilers and sysroots?  I know
ebuild/emerge supports them; do they treat them strictly as support for
cross compiles, or as installable images on the intended $host?  I
suspect the latter...


However, I take your point.  Looking at the fedora-mingw-cross
distribution, it does appear that they intend to provide a cross
compilation environment *only*; if you intend to package something for
deployment to the $host (e.g. to win32), then that's a separate build
process, using something other than rpm-build.

We could go that route as well; it just strikes me as a shame to "give
up" since win32 is /right there/ underneath cygwin all along...

OTOH, even on fedora-x86, it'd be nice to be able to map $sysroot$prefix
to some specific wine path (identity mount "inside" wine, as $prefix?),
and "run" the binaries "in place".  That would let you, in some cases,
run package test suites and the like.

>> One thing I thought of; if you want to run foo-config scripts from
>> /usr/i686-pc-mingw32/sys-root/mingw/bin during configure, then you
>> pretty much have to set $PATH so that $sysroot$prefix/bin precedes your
>> own /usr/bin -- otherwise, you'd get /usr/bin/foo-config.
>>
>> But...what if you're creating a REAL, full sysroot.  Like, with
>> coreutils and everything, over there in $sysroot$prefix/bin.  Do you
>> REALLY want the cross-$host version of 'uname' to precede your own?
> 
> No, but packages such as coreutils which only provide programs don't
> belong in a sysroot.  The purpose of a sysroot is for libraries which
> you use to cross-compile other software which depend on them.

See, my experience with "sys-roots" have been the "place where I install
all the crap I will turn into the system image on embedded-platform-X".
 And that includes $sysroot$prefix/bin/busybox, etc.

> But for this very reason, adding $sysroot$prefix/bin to PATH is
> unhelpful at best and maybe even dangerous.  

Well, yeah, that was kinda my point.  On cygwin-x-mingw, I might want to
do that for limited times and purposes, but not ordinarily.  And doing
it routinely just so that "configure" will "work" is...fraught with peril.

> If a package's configure
> depends on a foo-config script, then it should provide a way to specify
> the path to that script, by passing either a --with-foo-config=$sysroot
> $prefix/bin option or FOO_CONFIG=$sysroot$prefix/bin value to configure.

Right, that's a better approach.  However, you'd probably still need to
patch foo's foo-config script itself, so that it also accepts a
--sysroot option, right?  Otherwise, even
	$sysroot$prefix/bin/foo-config --cflags
will return something like -I$prefix/include/foo, when you really need
-I$sysroot$prefix/include/foo.

OTOH, a patch to foo-config to support sysroot is quite likely to be
accepted upstream.  (I know, you don't like this; see below).

And, the configure.ac for packages that are clients of libfoo, may also
need to be patched to support --with-foo-config=a-path if they don't
already. But again, that would be acceptable upstream.  Here's how vlc
does it, for dvdnav (where VLC_ADD_CFLAGS and VLC_ADD_LIBS do what you
would expect; adds the argument to CFLAGS and LIBS):

dnl
dnl  libdvdnav plugin
dnl
AC_ARG_ENABLE(dvdnav,
  [  --enable-dvdnav         dvdnav input module (default enabled)])
if test "${enable_dvdnav}" != "no"
then
  dnl prepend -ldvdcss on OS that need it
  AS_CASE(["${SYS}"], [mingw32|darwin|beos], [VLC_ADD_LIBS([dvdnav],
[-ldvdcss])])

  DVDNAV_PATH="${PATH}"
  AC_ARG_WITH(dvdnav-config-path,
    [  --with-dvdnav-config-path=PATH dvdnav-config path (default search
in \$PATH)],
    [ if test "${with_dvdnav_config_path}" != "no"
      then
        DVDNAV_PATH="${with_dvdnav_config_path}:${PATH}"
      fi ])
  AC_PATH_PROG(DVDNAV_CONFIG, dvdnav-config, no, ${DVDNAV_PATH})
  if test "${DVDNAV_CONFIG}" != "no"
  then
    VLC_ADD_PLUGIN([dvdnav])
    VLC_ADD_CFLAGS([dvdnav],[`${DVDNAV_CONFIG} --cflags`])
    VLC_ADD_LIBS([dvdnav],[`${DVDNAV_CONFIG} --libs`])
    AC_CHECK_LIB(dvdnav, dvdnav_get_video_resolution,
                AC_DEFINE(HAVE_DVDNAV_GET_VIDEO_RESOLUTION, 1, [Define
if you have dvdnav_get_video_resolution.]),
                [], [${LIBS_dvdnav}])
  fi
fi


> And no, I'm not about to start inventing paths or moving files around
> either.
> 
>> I think that's probably easier than patching every package that ships a
>> foo-config, so that foo-config accepts a -sysroot option.  But...a patch
>> adding a -sysroot option to a foo-config would probably be accepted by
>> the upstream maintainers of foo.
> 
> Not necessarily, and requiring a patch for every single package which
> ships a foo-config is less than ideal.

True, but foo-config + sysroot is broken. That's not ideal either.
However: how many packages, that either provide or use foo-config
scripts, are *WE* planning to build/support using the cygwin cross
toolchains, immediately?

If that number is relatively small, then...maybe having a "sysroot
clean" environment is a "good", that outwieghs the minor "bad" of having
to patch a *small* number of packages. Especially as those patches would
be, IMO, welcomed upstream.

> This is, as I noted earlier, the other area which makes configuring
> without $sysroot in $prefix difficult.  They are prone to adding -I and
> -L flags which will point to the native root instead of the sysroot,
> which will break the build at link time at least, if not earlier.

Well, if the config scripts support a --sysroot option, then that would
be ok. (or env var $SYSROOT, see below).

> If the libtool patch is accepted (and that's an if, given upstream's
> track record on patch review),

Ah, but --sysroot affects more than just us poor redheaded windows
stepchildren.  It's important also on (cue choirs of angels) linux. I
reckon the sysroot patch(es) will get expedited review.

Or, what passes for expedited review @ libtool.

> and this remains the only other issue to
> using --prefix=$prefix, then perhaps adding a cross_sysrootize()
> function which will do the trick:
> 
> src_install() {
>    cd ${B}
>    cyginstall
> 
>    cross_sysrootize ${D}${CROSS_SYSROOT}/usr/bin/foo-config
> }
> 
> where cross_sysrootize essentially does:
> 
> for f in $@; do sed -i -e "s|\([^t]\)$prefix|\1$CROSS_SYSROOT$prefix|g"
> $f; done
> 
> (the leading [^t] to avoid paths which somehow already had sys-root)
> 
> AFAICS from the *-config scripts I have in /usr/bin, that would appear
> to do the trick w/o breaking anything, but it would need some testing.

Hmm, hrrmmm, hummmm....so, I wanted to test your script to see what it
would do.  The first one I picked was 'freetype-config'.  Guess what?

It already supports SYSROOT, by the simple expedient of prepending all
output paths with the environment variable $SYSROOT (--libs sysroot
support uses some additional logic):

$ /usr/bin/freetype-config --cflags
-I/usr/include/freetype2

$ /usr/bin/freetype-config --libs
-lfreetype -lz

$ SYSROOT=/usr/i686-pc-mingw32 /usr/bin/freetype-config --cflags
-I/usr/i686-pc-mingw32/usr/include/freetype2
-I/usr/i686-pc-mingw32/usr/include

$ SYSROOT=/usr/i686-pc-mingw32 /usr/bin/freetype-config --libs
-L/usr/i686-pc-mingw32/usr/lib -lfreetype -lz



What about using this, for your cross-sysrootize (note the backslash
preceeding $CROSS_SYSROOT? (or, $SYSROOT)

for f in $@; do
  sed -i -e "s|\([^t]\)$prefix|\1\$CROSS_SYSROOT$prefix|g" $f
done

>> Now, as far as using the cross.cygclass to build non-core stuff.  Using
>> my original mingw-{zlib,bzip2,xz,gpg-error,gcrypt} cygports and Yaakov's
>> versions, I've created local versions of all those packages using the
>> new tools and the i686-pc-mingw32 compiler.
>>
>> It all seemed to go pretty smoothly; my cygports are still uglier than
>> Yaakov's, but that's partly because I was trying to maintain strict
>> one-for-one compatibility with the old versions (mgwbz2_1.dll names, and
>> all).  Later on, we might think about breaking backwards compat on those
>> packages (esp. the DLLs, since nobody uses them yet) and make "ours"
>> more normal, and the cygports more clean.  But not yet.
> 
> Those "mgw" prefixes need to GO AWAY.  Period.  NOW is the time to fix
> these DLL names, when were anyway completely changing mingw support,
> because they were wrong to begin with. 

Wrong? There is no "wrong" in this context; it's only software, and they
each performed their function as designed.  They are just different than
what you expect.


You know, every time I think I'm too opinionated and judgmental, all I
have to do is read one of your emails, and remind myself that I'm
practically a laissez-faire anarchist by comparison.


The mgw*dlls were done that way specifically so as NOT to step on
mingw.org's toes, since "ours" were stuck at gcc-3.4.4 and most likely
patched differently than "theirs".

You'd rather we blunder about duplicating dll names but not ABIs,
creating DLL hell for other distributions, because We're Just Mean?

It was actually a lot of work to ensure that we AVOIDED doing that to
other platforms (and, we have had ZERO complaints about stomping on
them, to date. That's worth something).

Also, with respect to zlib, the zlib.net people distribute their own
zlib1.dll which is built using MSVC6.0:

$ unzip -l ../zlib123-dll.zip
Archive:  ../zlib123-dll.zip
  Length      Date    Time    Name
---------  ---------- -----   ----
    18325  10-11-2004 04:15   DLL_FAQ.txt
        0  07-19-2005 06:00   include/
     9876  05-28-2005 08:40   include/zconf.h
    67545  07-18-2005 04:26   include/zlib.h
        0  07-19-2005 06:00   lib/
     6109  07-18-2005 17:25   lib/zdll.exp
    10590  07-18-2005 17:25   lib/zdll.lib
      981  11-08-2003 21:47   lib/zlib.def
     1528  07-19-2005 06:00   README.txt
        0  07-19-2005 06:00   test/
     8704  07-18-2005 17:37   test/example_d.exe
     4608  07-18-2005 17:37   test/minigzip_d.exe
     5120  07-18-2005 17:37   test/testzlib_d.exe
     7168  07-18-2005 17:37   test/untgz_d.exe
     3114  10-05-2004 02:00   USAGE.txt
    59904  07-20-2005 11:48   zlib1.dll
---------                     -------
   203572                     16 files

And they explicitly request that any DLL using the name 'zlib1.dll' be
exactly compatible with that one.  As it happens, a DLL built
using zlib-1.2.x/win32/Makefile.gcc -- named 'zlib1.dll' -- isn't
necessarily compatible with the official MSVC6.0-built "zlib1.dll",
because that makefile doesn't use -mms-bitfields (among other reasons).


Now, if we ensured that we did use -mms-bitfields (and the default
-static-libgcc), then maybe "our" mingw-ish zlib1.dll would be similar
enough (e.g. the pre-compiled test programs in the distro package above
actually work with the "imposter" dll), that we could get away with it.

But even at mingw.org, their mingw-ish zlib package doesn't pretend to
be "the" zlib1.dll. Instead, Keith builds it as "libz-1.dll" which is
neither "zlib1.dll", nor "mgwz.dll".

If I were to break our existing back-compat NOW, I'd probably attempt to
be inline with the libraries distributed by mingw.org, and try to ensure
that our version and theirs were built exactly the same. (And not
attempt ABI compatibility with zlib.net's "zlib1.dll")



However...


The primary purpose, until now, of the mingw-{zlib,...gcrypt} libraries
has been to support setup.exe and setup.exe alone.  I want to make the
MINIMUM changes in these packages, until we are sure that each
additional change doesn't break setup.exe.

Now, I don't OBJECT to doing so; in fact, I'll probably progress that
way before the mingw-gcc packages are officially released.  But not
today, and not by next Tuesday.  And now that I have an initial version
that DOES allow setup.exe to build correctly, I'll change ONE THING at a
time, testing setup.exe at each step, to ensure compatibility throughout.

These were the *initial* ports, the first attempt at migrating the
existing packages to the new toolchain, with the barest minimum changes.
 That way I could isolate any setup.exe build breakages if necessary.


> Cross-compiled sysroot libraries
> must always be compatible with the native versions in name and ABI,
> otherwise you completely defeat the purpose.

Right, and ours were not, so they avoided the native version names.  I
plan to EVENTUALLY migrate toward your goal, but FIRST I have to satisfy
my PRIMARY customer, setup.exe.

First, Do No Harm.

>> 2) For ALL of the mingw-{zlib,bzip2,xz,gpg-error,gcrypt} packages, I
>> made sure that each was compiled with -mms-bitfields.  All of the
>> *-mingw32-* packages shipped by mingw.org are built that way, even
>> though it is not the default for the compiler.
> 
> If mingw.org is using it for its packages then so must we; that's part
> of ABI compatibility after all.
> 
>> I really think it ought to BE the default for *mingw32*, but that train
>> has long left the station.
> 
> What about adding it to mingw*-gcc specs?  If not, then I'll have to add
> it automatically to CFLAGS and friends.

Well, isn't that "changing the default"? It seems that we have two
conflicting issues here: (1) maintaining compatibility of the
mingw.org-cross *compiler* with the mingw.org's own native compiler, AND
(2) maintaining compatibility of the sysroot inhabitants with the
corresponding ones @ mingw.org.

This is complicated by the fact that the mingw.org products use
-mms-bitfields, but the mingw.org compiler doesn't do that by default.
This means that any change in the compiler might surprise people who
have existing libraries they compiled themselves, using the compiler
default settings.

The same "surprise" might occur here: if somebody has previously built
mingw-ish stuff using 'gcc -mno-cygwin' but without -mms-bitfields, the
new (cross) compiler shouldn't make ABI changes. (Well, unless we flag
it as an ABI break; but that would make our compiler different from
mingw.org's!)

I think we are stuck with the same conflict as mingw.org, with our
compiler: the compiler defaults to GNU bitfields, but the products are
all provided with -mms-bitfields.

So...I think the place to fix this is NOT in the mingw-gcc's specs file,
but instead in cygport's cross.cygclass[case:i686-pc-mingw32].


If mingw.org ever changes their default ABI to -mms-bitfields, then we
would naturally follow suit.


> JonY, what about mingw-w64?  Are you using -mms-bitfields as well?
> 
>> 3) I also *linked* all of the DLLs and EXEs in these packages using
>> -shared-libgcc.  This has no effect on the setup.exe test, but I mention
>> it for thoroughness.
> 
> BTW, isn't that supposed to be the default?  Why isn't libgcc showing up
> as a dependency of anything C?

No, cygwin defaults to -shared-libgcc.  mingw uses -static-libgcc by
default for C, shared for all other languages:

* Shared libgcc: If all modules are linked with -shared-libgcc,
  exceptions can be thrown across DLL boundaries.  Note that this is
  the default for all languages other than C. To disable this, use
  -static-libgcc.

>> 4) I linked setup.exe with -static-libgcc, naturally.
> 
> Using -Wc,-static instead of -Wl,-static (part of my patch) takes care
> of that automatically.

Ah.

>> 6) I used Yaakov's setup-gcc45.patch to start with, but in order to
>> ensure that setup's .o objects were all compiled with -mms-bitfields, I
>> had to extend his changes (see attached; apply after setup-gcc45.patch).
>>
>>
>> Now, with all of those changes, setup.exe worked.  However, if I
>> compiled setup.exe using i686-pc-mingw32-gcc WITHOUT -mms-bitfields,
>> then I got an 0xc0000005 error.
> 
> Which is worse then I got doing everything w/o -mms-bitfields but using
> autoload.  But that's not surprising: once you start building with
> -mms-bitfields then *everything* needs to use it.

I see. As I said, I thought that ms-bitfields may have been a red herring.

--
Chuck


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]