[ITP] mingw-w64

[JonY]

On 7/1/2010 00:36, Charles Wilson wrote:
> On 6/29/2010 1:13 PM, JonY wrote:
>> On 6/30/2010 00:10, Charles Wilson wrote:
>>> Now, I thought you wanted to use the w64 prefix as a "project origin"
>>> indicator, and assumed that "-mingw64-" would be the "target bitdepth"
>>> indicator. However, given "w64-mingw64-pthreads-devel32" and
>>> "w32-mingw64-pthreads-headers32" I'm not sure that's the case. It seems
>>> NOW that you want to use 'mingw64' as the "project origin" indicator,
>>> and "w64"/"w32" as the target bit depth (and I'm sure the trailing '32'
>>> in these two anomalies is unnecessary).
>>>
>>> So, I think I'd put the project origin first, followed by the bit depth.
>>>
>>
>> Yes, I was using w64/w32 as bitness indicator, as with some releases
>> made with the buildbot on sf, mingw64 to differentiate it from mingw.org.
>
> Ah. Well, in that case I would think that the origin should come first,
> rather than the bitdepth. Our existing mingw-runtime package is from
> (effectively) mingw.org, after all (*).
>
> More below.
>
> (*) Well, both mingw.org mingwrt and cygwin's mingw-runtime come from a
> common repository: the winsup/mingw directory on sourceware's CVS
> server. [note that the "mingwrt" and "w32api" parts of "mingw.org" are
> hosted by sourceware; all the rest of "mingw.org" including other source
> code repos are hosted by sourceFORGE. The reason for the split is
> historical.]
>

I understand, they do use sourceware's combined configure tree.

> I don't know much about sf's buildbot; I assume that if you can get a
> cygport to DTRT on your home PC without human intervention except for
> kicking off the build, then you can convince the buildbot to do it for you?
>

Right now, human intervention still needed. cygport messes up the target 
dll locations by moving them around and trying to fix libtool files. Its 
also using cygwin strip(1) to strip 64bit dlls, it fails but doesn't 
lead to the cygport halting.

>> The trailing 32/64 is to indicate which toolchain the pthreads implibs
>> are for, it is too possible to setup a 32bit default multilib setup, the
>> current toolchain is defaulting to 64bit. So w64*devel32 means 64bit
>> implib for 32bit default toolchain that has yet to bet setup.
>>
>> For the current multilib toolchain example, users would want w32*devel64
>> and w64*devel64 pthreads packages, 32bit and 64bit implib for 64bit
>> default toolchain (the tarballs have the same installation path to the
>> 64bit libdir).
>
>
> Hmm. So, big picture, we have possibly three different mingw-ish
> compilers, and you're currently attempting to shepherd the first one,
> while being mindful of future issues related to simultaneous
> installation of both of the first two:
>
> (1) mingw64-derived, multilib, default 64bit
> (2) mingw64-derived, multilib, default 32bit
> (3) mingw.org-derived, non-multilib, only 32bit
>
> In the first two cases, they each must deliver some components in both
> 32 and 64 bit form, while some components (within a single case 1 or 2)
> are shared. However, NO components are shared BETWEEN any of the three
> cases, not even between (1) and (2). (**)
>
> (**) except for a possible sharing of target runtime DLLs with the same
> target bitdepth, but that's a small matter, discussed at the end of this
> email.
>
> Correct so far?
>

Correct, I am attempting the first.

> Now, SOME of the components that cannot be shared between (1) and (2)
> will be IDENTICAL in content, EXCEPT that they get installed into
> different locations. This would include, for instance, the runtime DLLs
> for, say, 32bit libgcc1 (UNLESS the 'share the target runtimes between
> (1) and (2)' approach is taken, but assume for the moment that we do
> not; wait until the end of this email for that discussion).
>
>
> So, we need a naming scheme that accounts for:
> 1) origin
> 2) default bit depth of the tool chain
> 3) bit depth of the target
>
> I think I would drop 'w64' since that's abiguous. "windows 64? is that
> tool chain default, so I've got to use -m32 if I want 32bit, or is that
> the actual bit depth of the contents, like "import libraries for linking
> 64bit target executables?" (Aside: I pity the organizational nightmare
> that you have to deal with over at mingw64.org, because you've got yet
> another set of 32/64 bifurcations: what is the bit depth of the
> toolchain EXE's themselves: is ld.exe a 32bit or 64bit executable...at
> least cygwin is just...cygwin.)
>

Well, Windows is special. It has Cygwin, MinGW, Interix, mingw-w64 and 
etc, all possibly running on the same time, while Linux is just Linux. 
It takes some nerve to get them to cooperate :)

> In fact, for complete clarity, I think I'd use:
> -tc64- or -tc32- == toolchain default bitdepth 64 or 32
> -m64- or -m32- == bit depth 64 or 32(use "m" as a mnemonic
> for which "-mNN" option it corresponds to)
>
> (And, returning to the aside, that 'frees up' the 'w64'/'w32' tag for
> use in distinguishing the bit depth of native windows ld.exe/gcc.exe:
> w64-gcc4 contains a binary gcc4.exe that runs as a 64bit native windows
> application. Not that you guys need or want to use an elaborate package
> naming scheme like it appears we need here. You don't have setup.exe to
> worry about.)
>
>
> If something is common to BOTH, then just drop that element. E.g. both
> the -m32 and -m64 mingw headers are provided by the same package, so
> that package would have only "mingw64-tc64-". OTOH, if you split the
> mingw import libs for the defaults-to-64bit toolchain between the
> "normal" ones (for that toolchain, e.g. 64bit) and the lib32/ ones, then
> you'd have "mingw64-tc64-m64" and "mingw64-tc64-m32"
>

Yes, this is a nice plan, clear and concise naming.

>
> The, obviously, the mingw.org compiler -- if cygwin ever got one --
> would prefix all of its elements with "mingw-" and no "toolchain
> bitdepth" or "target bitdepth" because, right now, it supports only
> 32bits. So everything would be -tc32-m32- which is rather silly.
>
> [1] These two packages [1a] and [1b] contain a DLL that is identical,
> but "lives" in a different location buried under the compiler's private
> lib area:
> mingw64-tc64-m64-libgcc1 : */lib/gcc/x86_64-w64-mingw32/4.6.0/
> mingw64-tc32-m64-libgcc1 : */lib/gcc/i686-w64-mingw32/4.6.0/64/
>
> If we had a /opt/mingw64/bin64 and /opt/mingw64/bin32/ "standard" then
> maybe the DLLs would go there, and you'd consolidate these two DLL
> packages into a single common package that just installs one copy of
> this specific DLL into
> /opt/mingw64/bin64/
> But we don't yet, so for now "buried" -- e.g. two separate identical
> copies in separate non-conflicting locations -- is good.
>
> [2] Ditto
> mingw64-tc64-m64-libstdc++6 :
> */lib/gcc/x86_64-w64-mingw32/4.6.0/libstdc++-6.dll
> mingw64-tc32-m64-libstdc++6 :
> */lib/gcc/i686-w64-mingw32/4.6.0/64/libstdc++-6.dll
> vs.
> /opt/mingw64/bin64/
>
> [3] Ditto, /opt/mingw64/bin32/
> mingw64-tc64-m32-libgcc1 : */lib/gcc/x86_64-w64-mingw32/4.6.0/32/
> mingw64-tc32-m32-libgcc1 : */lib/gcc/i686-w64-mingw32/4.6.0/
> vs.
> /opt/mingw64/bin32/
>
> [4] Ditto
> mingw64-tc64-m32-libstdc++6 :
> */lib/gcc/x86_64-w64-mingw32/4.6.0/32/libstdc++-6.dll
> mingw64-tc32-m32-libstdc++6 :
> */lib/gcc/i686-w64-mingw32/4.6.0/libstdc++-6.dll
> vs.
> /opt/mingw64/bin32/
>
>
>> The pthreads naming is a bit confusing (to myself as well). I should
>> change it to something easier. Ideas welcome.
>
> Well, if the ideas above make sense, then it should be pretty
> straightforward to extend them to libpthread2.
>

 From the packaging point of view, having them buried in toolchain is a 
lot easier than trying to move them around at postinstall.

>>> Which leads to...use a different prefix as the only viable choice, I
>>> think (I'd go with /opt/mingw64/). Or asking to use a non-standard
>>> --datarootdir -- and HOPE that the only conflicts we ever discover are
>>> these /usr/share ones.
>>>
>>>
>>
>> I'm also not too sure how to proceed, --datarootdir sounds good.
>
> Well, putting everything from mingw64 into /opt/mingw64 would also work.
> So the exe's (for both the default-to-64bit toolchain and the
> default-to-32bit toolchain) would go in
> /opt/mingw64/bin/
> as well as the typical "cross-compiler" copies of those exe's in
> /opt/mingw64/x86_64-w64-mingw32/bin/ and
> /opt/mingw64/i686-w64-mingw32/bin/
> As cygwin apps, these would likely depend, themselves, on DLLs in
> /usr/bin -- but that just means that /usr/bin/ must be in $PATH when
> launching these tools. But that's ALWAYS true for cygwin apps.
>
> BUT, you'd have the added freedom of establishing also
> /opt/mingw64/bin32/ and
> /opt/mingw64/bin64/
> specifically for the target DLLs (libgcc1_sjlj.dll, libstdc++6.dll,
> libpthread2.dll, etc) for the two separate bitdepths. This would allow
> you to consolidate the duplicate DLL packages [1a] == [1b], [2a] ==
> [2b], [3a] == [3b], [4a] == [4b], etc, above.
>
>

OK, I will use --prefix=/opt/mingw64 and 
--with-sysroot=/opt/mingw64/{x86_64,i686}-w64-mingw32 to avoid stuff 
clashing with /usr/share.

Target DLLs will go in /opt/mingw64/bin{32,64}. Users will have a 
consistent place to look for them.

To consolidate the DLLs, only one copy for 32bit and 64bit will be 
packaged from one of the compilers and placed in 
/opt/mingw64/bin{32,64}. For example, packaging 32bit libgcc DLL from 
the 64bit toolchain, but leaving out the one from the 32bit toolchain 
since they're similar.

This leads to the next question, how do I add the /opt/mingw64/bin to 
the user's $PATH after installation?

>
> As for the rest of your reply...
>
> OK, ignore my silly comments about extra symlinks.
>
>>> LIBGCC RUNTIME DLL
>>> ===================================
>>> w64-mingw64-gcc4-libgcc1-4.6.20100619-1.tar.bz2
>>>
>>> (First, no need to include 'gcc4' in the library package name).
>>>
>>
>> I'm not sure how to do this with cygport. Any ideas?
>
> In the cygport file, you can specify exactly the name of all
> sub-packages. Here's the relevant bit from gettext's cygport
>
> PKG_NAMES="${PN} ${PN}-devel libgettextpo0 libasprintf0 libintl8"
> PKG_HINTS="${PN} ${PN}-devel libgettextpo0 libasprintf0 libintl8"
>

No idea why I haven't thought of removing ${PN} for the subpackages.

> The contents of the different packages in this case are controlled by
> the following files, created by the .cygwin.patch:
>
> gettext-0.17/CYGWIN-PATCHES/gettext-devel.list | 36 ++
> gettext-0.17/CYGWIN-PATCHES/gettext.list | 33 ++
> gettext-0.17/CYGWIN-PATCHES/libasprintf0.list | 1
> gettext-0.17/CYGWIN-PATCHES/libgettextpo0.list | 3
> gettext-0.17/CYGWIN-PATCHES/libintl8.list | 1
>
> as long as the *.list name matches the value in PKG_NAMES, it will be
> use. A more common approach is (from ncurses cygport):
>
> PKG_NAMES="${PN} lib${PN}${NCURSES_ABI} lib${PN}-devel ${PN}-demo terminfo"
> PKG_HINTS="${PN} lib${PN}${NCURSES_ABI} lib${PN}-devel ${PN}-demo terminfo"
>
> PKG_CONTENTS[0]="--exclude=usr/bin/*.dll --exclude=usr/bin/*-config
> usr/bin usr/share/doc/${PN} usr/share/doc/Cygwin usr/share/man/man1
> usr/share/man/man5 usr/share/man/man7"
>
> PKG_CONTENTS[1]='usr/bin/*.dll'
>
> PKG_CONTENTS[2]='--exclude=usr/lib/ncurses etc usr/bin/*-config usr/lib
> usr/include usr/share/man/man3'
>
> PKG_CONTENTS[3]='usr/lib/ncurses'
>
> PKG_CONTENTS[4]='usr/share/terminfo usr/share/tabset'
>
>
> Where the element of the PKG_CONTENTS array is used when creating the
> tarball for the corresponding PKG_NAME.
>

I am currently using this approach, this leads to long and complicated 
--exclude lines. Does the list file allow \n delimited list of files to 
include in the tarball release?

>>
>> Ok, right now, I should make all the dlls package common to 32bit and
>> 64bit defaulting toolchain, but I'm not too sure how to do it with
>> cygport. Any ideas?
>
> Well, I *assume* that you build the default-to-64bit gcc *completely*
> separately from the default-to-32bit? That is, there's a
>
> something-something-gcc-VER-REL-src.tar.bz2 for the default-to-32bit
> toolchain, and a something-something-gcc-VER-REL-src.tar.bz2 for the
> default-to-64bit one? And each -src tarball contains a separate
> something-something-gcc-VER-REL.cygport?
>

That was my first attempt, but I concluded its better to have them 
shared according to your replies. There will still be 2 sources (each 
for 32bit default and 64bit default), but the runtime target dlls won't 
get duplicated.

> IF you wanted to ship only a single copy of the appropriate runtime DLL,
> to be used by both toolchains, FIRST you need to decide where it should
> go. If it's to be shared, then it can't go under
> */x86_64-w64-mingw32/* anywhere, nor
> */i686-w64-mingw32/*
> because then they wouldn't be "sharing". One toolchain would have to
> "know" about the other somehow. Ick.
>
> However, you can't simply stick them in /usr/bin (or /opt/mingw64/bin,
> whatever), because the true 64bit version and the 32bit version both
> have the same name. Also Ick.
>

I tried fixing up libtool upstream to use "w64" and "w32" prefix instead 
of "lib" prefix like the cygwin "cyg" prefix, but maintainers weren't 
too welcoming of the idea.

> That's why I suggest a "top-level" */bin32 and */bin64 specifically for
> these target DLLs, so that they can be shared by both the default-to-64
> and default-to-32 toolchains. However, cygwin package standards don't
> allow additional top-level dirs under / or /usr. So... /bin32 and
> /usr/bin32 are out.
>
> Hence, the advantages of you putting everything under /opt/mingw64/.
> You'd OWN that directory tree. Do whatever you like under there.
>
> Once you make your decision about WHERE the target DLLs should go,
> ensuring you only have one version is easy: both cygports will build and
> create all the appropriate packages.
>
> The default-to-64 cygport would create (for example) under the "rules"
> above
> mingw64-tc64-m64-libgcc1-VER-REL.tar.bz2
> containing
> /opt/mingw64/bin64/libgcc1_sjlj.dll
>
> while the default-to-32 cygport would create (for example)
> mingw64-tc32-m64-libgcc1-VER-REL.tar.bz2
> ALSO containing
> /opt/mingw64/bin64/libgcc1_sjlj.dll
>
> Ideally, these two DLLs would be identical. Assuming that is the case,
> then you rename one of those two tarballs
> mingw64-m64-libgcc1-VER-REL.tar.bz2
> (e.g. drop the -tc64-). Ditto for
> mingw64-m32-libgcc1-VER-REL.tar.bz2
> and all the other toolchain runtime libs (incl. libpthread2 ?)
>
> As a final shortcut, you tweak both cygports to create
> non-tc*-differentiated tarballs for the target runtime dlls to begin
> with, so in the end, both cygports create
>
> mingw64-m64-libgcc1-VER-REL.tar.bz2 <<< gen by -tc64- cygport
> mingw64-m32-libgcc1-VER-REL.tar.bz2 <<< gen by -tc64- cygport
> mingw64-m64-libgcc1-VER-REL.tar.bz2 <<< gen by -tc32- cygport
> mingw64-m32-libgcc1-VER-REL.tar.bz2 <<< gen by -tc32- cygport
>
> rather than
>
> mingw64-tc64-m64-libgcc1-VER-REL.tar.bz2 <<< gen by -tc64- cygport
> mingw64-tc64-m32-libgcc1-VER-REL.tar.bz2 <<< gen by -tc64- cygport
> mingw64-tc32-m64-libgcc1-VER-REL.tar.bz2 <<< gen by -tc32- cygport
> mingw64-tc32-m32-libgcc1-VER-REL.tar.bz2 <<< gen by -tc32- cygport
>
> and you simply upload only one of each pair.
>

Setup won't fudge over similarly named packages? I thought of doing just:
mingw64-m64-libgcc1-VER-REL.tar.bz2 <<< gen by -tc64- cygport
mingw64-m32-libgcc1-VER-REL.tar.bz2 <<< gen by -tc64- cygport

and having them totally independent of the actual toolchain that 
generated them, from setup.exe's package dependency point of view.