[general] some ideas & request for comments (LONG)

Michael Ring m.ring@ndh.net
Tue May 16 14:53:00 GMT 2000


On Tue, 16 May 2000 00:14:05 -0400, you wrote:

>Over the next several months I'm planning to rebuild many of the
>packages that are offered on CygUtils so that they will fit into the new
>sourceware distribution. At minimum, this means:
>
>--prefix=/usr
>--host=i686-pc-cygwin
>
>But, as I go, I'm also trying to clean up the ports somewhat so that
>they build more cleanly, without as much manual intervention. At best,
>configure/make/make -install. At worst, patch/configure/[ edit result
>:-( ]/make/make-install. In addition, I hope that all the library
>packages (like zlib, libtiff, ncurses, etc) will build as a static
>library and as a dll+import lib[* see below] in a single pass. The idea
>is that dll's will be used by default; static libs provided as a
>convenience only.

Thank you for that fantastic idea! I am also of the oppinion that as
much as possible should be available as shared libraries. This makes
updating & bugfixes a hole lot easier.

>I've got my asbestos suit zipped up, so flame ON!
>

No need to do that, lots of what you are writing makes sense to me...

flame OFF !!

>=========================================
>1) naming and location convention for libs/implibs/dll's
>
>First I'll present a few options, and then my comments. I'll use zlib as
>the example. I'll also assume that you need slightly different .h files
>to use the static lib or the dll/importlib. This is not true in the case
>of zlib, but other libraries may have restrictions like that, so let's
>assume the worst.
>
>a) /usr/lib/libz.a-static (static library)
>   /usr/lib/libz.a-dll    (import library for dll)
>   /usr/lib/libz.a -> libz.a-dll (symlink; dll by default)
>   /usr/bin/libz.dll
>   /usr/include/zlib.h        (headers for dll)
>   /usr/include/zconf.h
>   /usr/include/static/zlib.h (headers for static library)
>   /usr/include/static/zconf.h
>
>packages can link with dll OOB
>linking with static lib requires -I/usr/include/static and changing the
>symlink

I do not like the symlink approach; problem is that you cannot decide
at build-time if you want to link static or dynamic.

Example:

With my Unix-buildscript I build rpm with shared-libs to make files
small. The next thing I build is a completly static linked version of
rpm for our production environment because this cannot be harmed by
any updates / superusers going crazy / etc...

To be able to do this with this approach would mean to change links on
the fly.. Nah, don't like that..... 

>
>b) /usr/lib/libz.a        (import library)
>   /usr/lib/static/libz.a (static library)
>   /usr/bin/libz.dll
>   /usr/include/zlib.h, zconf.h
>   /usr/include/static/zlib.h, zconf.h
>
>packages can link with dll OOB
>linking with static lib requires -I/usr/include/static and
>-L/usr/lib/static, but no filesystem (renames or symlink) changes

YES, looks much better. Funny thing is, I had the same discussion on
the phone with Corinna a few days ago. This approach makes it easy to
change between static/dynamic linking. 

>
>c) /usr/lib/libz.a      (import library)
>   /usr/lib/libz_s.a    (static library)
>   /usr/bin/libz.dll
>   /usr/include/zlib.h, zconf.h
>   /usr/include/static/zlib.h, zconf.h
>
>packages can link with dll OOB
>linking with static lib requires -I/usr/include/static and changing the
>target package's makefile -- replace '-lz' with '-lz_s'
>

Not my favorite, better than a) but not as good as b) If you think
about automating the process of search & replace in the makefile
things are not that easy because you will have to considder that there
exist static-only libraries for which nothing should be changed. The
problem is not impossible to solve but it is not an easy no-brainer as
b)


>d) /usr/lib/libz.a     (static library)
>   /usr/lib/libz.dll -> /usr/bin/libz.dll
>   /usr/bin/libz.dll
>   /usr/include/zlib.h, zconf.h
>   /usr/include/static/zlib.h, zconf.h
>
>gcc/binutils can link directly to a dll without an import lib. (Try it
>-- put the full pathname of a dll on the link line: 
>   'gcc -o foo.exe foo.o /usr/local/bin/libz.dll'
>and it works just fine. The only problem is binutils & gcc don't know to
>search for "libz.dll" when they see '-lz' on the link line.
>
>I've got a few very minor patches I'm submitting to the binutils and gcc
>lists tonight that allow the following behavior:
>
>gcc -o foo.exe foo.o -lbar 
>  ---> will link foo.o with bar.dll or libbar.dll or libbar.a (in that
>order of preference) without using an import lib (unless, of course,
>neither bar.dll nor libbar.dll are found in the library search path, and
>libbar.a *is* an import lib)
>
>gcc -static -o foo.exe foo.o -lbar
>  ---> ignores bar.dll & libbar.dll, will only link to libbar.a (this
>doesn't *guarantee* a statically linked foo.exe, since libbar.a may be
>an importlib)
>
>So, assumming these patches get in (at least to the cygwin distro of
>those tools) then where should libfoo.dll go? It needs to be in the
>windows PATH so that the windows subsystem can do the runtime dynamic
>linking. It needs to be in gcc's library search path so that gcc can
>find it during the compile-time linking. If libfoo.dll goes in /usr/lib,
>then you have to add /usr/lib to the PATH (actually, /lib since windows
>doesn't grok the cygwin mount points); symlinks won't work because
>windows doesn't understand them. 
^^^^^^^ No! ^^^^^^^^
vvvvvvv Yes vvvvvvvv
OR, put libfoo.dll in /usr/bin ( ==
>/bin which is already in the windows PATH), and symlink from
>/usr/lib/libfoo.dll to /usr/bin/libfoo.dll ---> /usr/lib is already in
>the gcc library search path, and cygwin-gcc groks symlinks.
>

Sounds like the best solution if your patches get accepted. 

On NT, the .dll should be in the same directory as .exe; this makes it
possible to start an application even if there is no path set to the
directory. 

/lib or /usr/lib should not be used because Windows98 users could run
in severe out-of-environment-space problems because they would have to
add /usr/lib /usr/X11R6/lib and /usr/local/lib to path
(O.K. I am making up arguments, the real reason is on my point of view
that it simply looks horrible)



>Sometimes, the import library provides information that can't be
>generated automatically on-the-fly by ld/gcc during the link step. For
>instance, libcygwin.a renames some of the functions in cygwin1.dll. In
>those rare cases where the import lib provides extra information, then
>funky things can be done like:
>  /usr/lib/libfoo.a (import lib)
>  /usr/lib/libfoo_s.a (static lib)
>  /usr/bin/libfoo.dll
>  (NO libfoo.dll symlink in /usr/lib )
>
>packages can link with dll OOB
>packages can link with static lib, but it depends on whether you NEED to
>use an explicit import library (I expect this to be very rare). 
>  (1) implicit import lib: change the link command by adding '-static'
>option, and put -I/usr/include/static on the compile line. No filesystem
>changes are needed.
>  (2) explicit import lib: change the link command by using '-lz_s'
>instead of '-lz', and put -I/usr/include/static on the compile lines. No
>filesystem changes are needed.

What about this idea:

link foo.o with libbar_import.a (import library, if exists) bar.dll
libbbar.dll then libbar.a

This would keep filenames more consistent.

instead of renaming the static lib, simply add a renamed version for
the import_library.
 As you stated, import libraries are not always needed. With your
approach, the name of the static lib must be changed in this seldom
case. This could confuse people because they cannot follow simple
rules like:

.a is static
.import.a is import-library
.dll is dynamic. 

With your approach .a sometimes means dynamic. No need for that
confusion if you use my approach.

>
>+++++++++++++++
>The following drawback is shared by (a), (b), and (d): no granularity.
>Your target program is either completely staticly linked (other than the
>windows dlls and cygwin1.dll, of course), or totally dynamically linked.
>With option (c) you can explicitly link libfoo (static), libbar
>(dynamic) and libbaz (static): '-lfoo_s -lbar -lbaz_s'

Do we really need that ??? At this point I cannot imagine a scenario
where you will want to go for a mix. I am thinking about the work that
has to be done for porting an existing application. b) or d) are the
versions that promise an easy port.

>Assuming binutils/gcc accept my patches (or at least the maintainers of
>the cygwin ports of those packages), then I vote for option (d).

Make it two votes! (and a half vote for b) ;-)

>
>=========================================
>2) location of these 'extra' packages on Cygnus' sourceware ftp site.
>
>Not everybody wants to download rpm or perl or even libz as part of the
>initial setup of cygwin1.1.1. Yet, it would be nice to consolidate many
>of the Cygwin ports to sourceware, so that folks can find them easier. 
>

YESYESYESYESYESYESYESYESYESYESYESYESYESYESYESYES

>Now, DJ & I've already stirred up the hornets' nest a bit by putting
>zlib and libpng into the main 'latest' directory. By default, setup will
>download that to all new users. To work around this problem, Chris
>Faylor offered to create a 'cygutils' directory on sourceware for "my"
>ports. However, I think that's too specific; perhaps 'extras' or
>'add-ons' or something would be better. Wouldn't it be great if
>Corinna's OpenSSL and OpenSSH ports were in there? So, let's call it the
>'extras' directory for the purposes of this discussion.

name it contrib, because it means: no support, use at your own risk,
don't blame corinna, dj or cgf if it does not work.

>
>Should 'extras' be a child of 'latest' or a sibling?

I think iut should be kept independent of the latest dir. the apps
there have nothing to do with the official release. 

>
>I think most folks would not see it, if it was a sibling, and as a child
>of 'latest' it will be obvious to folks who are just browsing the
>downloads. If 'latest/extras' contains only directories, and no
>.tar.gz's, then setup.exe will by default ignore it. 
>
>So, how about:
>
>  /pub/cygwin/latest/extras
>  /pub/cygwin/latest/extras/zlib/
>  /pub/cygwin/latest/extras/OpenSSL/

I would say:

/pub/cygwin/contrib/latest/zlib

>
>etc.?
>
>=========================================
>3) how setup.exe handles 'extra' packages
>
>I would like for folks to be *able* to use setup.exe to download/upgrade
>these 'extras'. However, setup.exe will only access the 'latest'
>directory, at present. Whether 'extras' is a child or sibling of
>'latest', I propose a new option to setup.exe that tells setup to
>download extra stuff.
>   setup --extras (and setup -u --extras) ?
>   setup --optional-packages ?
>Each of this options would tell setup.exe to not only search within
>'latest' for dirs ONE level down that contain .tar.gz's, but also to
>look in the 'latest/extras' directory one MORE level down for additional
>.tar.gz's.

Yup, sounds good. Perhaps one should also implement something like
setup --contrib --show

to receive a listing of what is available. 

>
>Or perhaps 'setup --search-dirs='latest:latest/extras' is more flexible,
>where '--search-dirs=latest' is the assumed default.
>
>I propose these things, but I don't know enough about the innards of
>setup.exe to actually add the functionality. However, do ya'll think
>it's a good idea?
>

Yes, it is in my point of view.

>=========================================
>4) what should be included in the package tarballs (especially
>-src.tar.gz) on Cygnus' sourceware ftp site.
>
>I see four options:
>---------------
>(1) the way I did it at first (sorry, guys...)
>
>o zlib-<pkgversion>-<cygwinver>-src.tar.gz
>      === identical to official zlib-<pkgversion>.tar.gz
>
>o zlib-<pkgversion>-<cygwinver>-patch.tar.gz 
>      === contains patches necessary for clean building on cygwin to
>produce:
>
>o zlib-<pkgversion>-<cygwinver>.tar.gz
>      === the binary package
>
>problems: user must download two packages to build; setup.exe unpacks
>all the -patch.tar.gz's into the root directory automatically (Oops.)
>Setup behavior is easy to fix, but... 

Still my favorite, why, see later...

>
>----------------
>(2)
>
>o zlib-<pkgversion>-<cygwinver>-src.tar.gz
>      contains the same files as the official zlib-<pkgversion>.tar.gz,
>but also contains the required patches and stuff for cygwin (user must
>apply the patches, then build the package)
>
>o zlib-<pkgverson>-<cygwinver>.tar.gz
>
>headache for maintainers
>
>----------------

Hate that! I think that source-packages should always be kept in their
original state. If something needs to be changed, put aside a patch
file, but do not include it; It is better as (3) but still a lot worse
than (1) 

>(3)
>
>o zlib-<pkgversion>-<cygwinver>-src.tar.gz
>      contains the source code for zlib-<pkgversion>, but already
>patched up to build cleanly with cygwin. Also contains the necessary
>patches, so that the cygwin-specific changes can be reversed by 'patch
>-R < cygwin-patch1' etc.
>
>o zlib-<pkgversion>-<cygwinver>.tar.gz
>
>headache for maintainers
>
>-----------------

AAAARRGGGHHHHH....... NoNoNo, NEVER....

(sorry)

I am working a lot with rpm and rpm works with patch files.
downloading a file, extracting it, undoing patches, moving it to a
save place and then unpacking source again, deleting the two patch
files, only to generate the 'right' patch file and then tar the
'unpatched' version does not sound like fun to me.

>(4)
>
>o zlib-<pkgversion>-<cygwinver>-src.tar.gz
>      contains the source code for zlib-<pkgversion>, but already
>patched up to build cleanly with cygwin. NO extra documentation or
>patches.
>
>o zlib-<pkgversion>-<cygwinver>.tar.gz
>
>Could get ugly. Keeping the code in an evolving cygwin-apps CVS
>repository can help, but this amounts to forking the code. Good? Bad?

BAD! This makes upgrading to newer versions like hell because after a
while you loose sync to the original distribution. If you have ever
tried to make sense out of a diff of make from cygwin against make
from GNU you will know what I mean. Especially if peaple do things
like libtoolize or autoheader before packaging things get ugly (and
patchfiles get LARGE

>
>------------------
>In both (2) and (3), the patchset should probably reside in a uniform
>directory in every package, whose name will probably not conflict with
>pre-existing directories in any package. 'CYGWIN-PATCHES' ?
>
>I vote for (3). It's clean, it allows our patches to eventually be
>merged back into the main codebase if they're clean enough and don't
>break other platforms. It's a little bit of a pain for maintainers, but
>it's similar to what I've been doing with CygUtils for a long time now.
>Finally, personal CVS repositories or sourceware-based CVS can make it
>not quite so painful.

I give my clear vote to (1) 

I am using rpm to build software and the rpm-based version of
cygwin-1.1.1 will hit the beta-road soon. They Way that rpm works
(because a lot of developers decided to implement it that way) is that
it takes the original distribution found at the author's site and then
adds patches to the source. So checksums and everything keep intact
and you can always be sure that you are really compiling the original
code and not code that was hacked to be a trojan-horse by someone.

The setup.exe problem could be solved very easy: simply replace
.tar.gz with .src.tar.gz; then setup will leave the files alone.

>
>===========================================
>The End. Whew.
>
>--Chuck

OOPs, been writing for almost an hour now, but it was fun!

Hope my 10 Cents help!

Michael Ring


More information about the Cygwin-apps mailing list