[ANNOUNCEMENT] Server Test 22
Charles S. Wilson
cwilson@ece.gatech.edu
Fri May 4 01:03:00 GMT 2001
Executive Summary:
Each point is discussed more thoroughly below. I am replying with this
HUGE message because (a) I'm not on the cygwin-xfree mailing list, so
individual replies are tricky and don't show up right in the archives,
and (b) there was a lot of repitition in the discussion, I'll just reply
in one spot, thank you... Each of the following six points is addressed
in greater detail below. Search for "-----SECTION".
1) I have no objections to moving the official libz package from contrib
to latest, and will do so at the earliest opportunity.
2) I'd prefer to relinquish Xpm support to the cygwin-xfree project, but
the noX version of Xpm still needs a home, and **needs cooperation**
from cygwin-xfree since both X-based and noX-based libs share xpm.h.
noX-Xpm is *required* for the cygwin, native-windowing version of
XEmacs.
3) The cyg<package><maj-ver>.dll naming convention was adopted after
MUCH discussion last summer, and a hell of a lot of coding in
binutils/gcc. Packaging a binary compiled so that it creates a dll with
that name is not a fork. (And, in most cases, I submit my changes back
to the original maintainers of the packages; sometimes they're adopted,
sometimes not). In any case, the cygwin-xfree developers chose to
ignore that discussion, so cygwin-xfree dll's do not follow the
convention.
4) Suhaib's proprietary attitude toward zlib and xpm is misplaced. The
Xfree folks copied the zlib code into their codebase a long time ago,
true -- but that doesn't make XFree's version official
(Xfree/xc/zlib/README still says 1.0.8 BTW, not 1.1.3). xpm was an
independent entity for YEARS (xpm-3.4k, anyone?) before finally being
adopted into the xfree codebase at 4.0.0. However, in that case,
xfree's version was officially recognized as the official xpm. But zlib
is a system library (you don't see libz.so inside
xfree86-4.0.3-X-i386.rpm, do you?)
5) I hereby pledge not to provide a freetype library. (Thank God. I
was dreading it. I also didn't realize that it is now a part of XFree.
It wasn't in x11r6.4) Suhaib was right -- I *was* planning to migrate
my port. But now I won't. And that makes me happy. :-)
6) cygIPC. I recommend against building cygwin/XFree86 with this
dependency. There is already work underway to add IPC functionality to
the cygwin1.dll, although full IPC support will require a "cygwin
daemon". Once this capability is added to the official cygwin dist,
then it will be actively supported by the cgywin developers. However,
CygIPC (by which I mean the specific package at
http://www.neuro.gatech.edu/users/cwilson/cygutils/V1.1/cygipc/ ) will
NEVER be added to the official dist, for licensing reasons. Therefore,
CygIPC is not supported by the list, but by one person. Me. And I've
got really limited cycles; I'm afraid that hundreds of new cygwin-xfree
users will overwhelm my ability to support. The end result: people
upset with me, people upset with cygwin-xfree, and really really
confused newbies. But it's your decision.
--------------------------------------
-----SECTION 1
--------------------------------------
zlib moving to latest. What does "contrib" vs. "latest" mean? A bit of
history.
my dll-ized version of zlib first appeared in cygwin/latest on Sat, 13
May 2000.
It is true that cygwin-xfree contained a zlib library at that time.
However, zlib is required for many other packages and it was IMO silly
to force people who wanted libpng (which depends on libz) to download a
21Meg package (xfree86-bin.tar.bz2 was 21Meg, at the time) in order to
get a 50k dll that is, in fact, a separate project. libz has its own
webpage, and its own (okay, halted) development. But the source code
inside xfree/zlib/ is NOT the official repository of zlib.
http://www.info-zip.org/pub/infozip/zlib/ is.
At best, we have two separate forks of zlib. Not "Xfree's official
non-forked zlib" and that scuzball cwilson's evil fork. XFree forked
the zlib code at 1.0.8 and included the code within its own codebase. I
took 1.1.3 and applied the patches necessary to get it to build as a dll
on cygwin, and follow the cygwin conventions. xfree *may* have been
tracking the official zlib code changes from 1.0.8--1.1.3, but I'm not
sure. I do know that cygwin-xfree's version of libz.dll does not follow
the cygwin naming conventions. So which one is the fork? Which one
*should* be official? (answer left as an exercise for the student). The
main difference between the two libz dll's is the __declspec()
decorations in zlib-1.1.3-6's cygz.dll, and no declspec decorations in
cygwin-xfree's libz.dll. Also, the name cygz.dll vs. libz.dll. Not
counting the actual dll's, zlib-1.1.3-6 provides /usr/lib/libz.dll.a
(import lib) AND /usr/lib/libz.a (static lib). xfree provides only
/usr/X11R6/lib/libz.a (import lib). NO possibility of static linking
with the xfree version of libz. I'll point out, also, that the official
cygwin/windows dll naming scheme requires cygz.dll and libz.dll.a names
for dll and import lib.
Anyway, I contributed zlib as a dll and static lib to cygwin in May of
2000. It was always my hope that the cygwin-xfree folks would start
using it instead of rolling their own, but that never happened (until
now, maybe). I'm not sure why, but whatever the reason, it's not
important.
I provided the zlib package as a service to cygwin users -- including
xfree-folks -- and did NOT intend it as an insult or slight to the
cygwin-xfree people. I also don't consider it a fork. I submitted my
changes, which enable zlib to build as a dll on cygwin following the
appropriate conventions, back to Greg Roelofs. He has placed them at
http://www.info-zip.org/pub/infozip/zlib/contrib/ However, he is
reluctant to release a 1.1.4 with any additional patches without the OK
from Jean-loup Gailly -- and HE doesn't really want to release ANY
updates that aren't strictly bugfix. Thus, any new platform support
(e.g. a dll-ized cygwin zlib) can only be as official as the
infozip/zlib/contrib directory on the info-zip.org website. Until
Jean-loup and Mark Adler decide to fix any bugs. :-(
Mea culpa: I haven't submitted my changes to Greg since
cygwin-zlib-1.1.3-2. We're now at 1.1.3-6. The library name change
happened eight months ago at 1.1.3-5, in order to follow the official
dll naming convention for cygwin. I'm terribly sorry about the
oversight. I'll correct that immediately and submit my latest patch to
Greg.
Anyway, back to cygwin-zlib. In June 2000, at zlib-1.1.3-3, zlib was
moved to the newly created cygwin/contrib directory. (Remember,
zlib-1.1.3-1,2 both lived in cygwin/latest). This was because, at the
time, the "official" packages (e.g. from DJ, Chris, and Corinna) would
be in "latest" and EVERYTHING else would go in "contrib". We even
talked about porter-specific contrib dirs, but decided against it. Note
that the distinction between latest and contrib was based on SOURCE (who
submitted it), not on a distinction between "required" and "optional" --
although, in the end, that's the way it has worked out.
Right now, though, setup.exe -- the REQUIRED method of installing cygwin
-- makes no distinction at ALL between "latest" and "contrib". zlib
could be in /cygwin/dont-install-me/zlib/ and setup.exe would still
happily display it as an option. So, currently, the "contrib" directory
is meaningless except at the meta-level as a contributer-ID
classification.
My understanding of the "protocol" was that once an "official" package
-- e.g. one in /latest/ -- developed a dependency on a "contrib"
package, then either (a) remove the dependency, or (b) move the package
into latest. This happened recently with ncurses -- the inetutils
(talk) package acquired a dependency, so we moved ncurses from contrib
to latest. So far, to my knowledge, no package currently in /latest/
depends on zlib, so I haven't moved zlib into latest. But I will do so
now.
Eventually, it would be nice if setup (a) understood dependencies (b)
knew what was "optional" and what was "required" -- but it doesn't. And
even then, the "latest" vs. "contrib" is meaningless. "inetutils" is in
"latest" but is hardly required -- most folks use sshd/ssh not
telnetd/telnet or rtalk, etc.
Finally, the zlib package installs itself into /usr/lib, /usr/bin, and
/usr/include (along with documentation elsewhere). The fact that the
archive originates in /cygwin/contrib/zlib/ is largely meaningless.
To summarize:
zlib-1.1.3-1: my first dll-ized zlib contribution, May 2000. in
latest.
zlib-1.1.3-3: zlib moved to contrib (a new subdir), Jun 2000.
zlib-1.1.3-5: the great name change of '00. "cygz.dll" Oct 2000.
current: zlib-1.1.3-6
xfree's copy of zlib sources is not the official one.
(neither is mine for that matter, but it's as close as we can get with
current zlib project status (cf. Jean-loup Gailly, Mark Adler, Greg
Roelofs) Alternative: no dll.
did not intend insult to cygwin-xfree folks.
setup.exe required install method for cygwin, "contrib"/"latest"
meaningless.
I'll move zlib to "latest" anyway, if it makes people happy.
--------------------------------------
-----SECTION 2
--------------------------------------
xpm. XEmacs. noX.
I'd love to turn over xpm support to the cygwin-xfree folks. There's
just a few problems, but we can probably find solutions. First, though,
I'd like to reiterate: xpm existed as an independent package for many
years. It was not an integral part of xfree until 4.0.0. Suhaib stated
that he thought xpm was part of the various Xfree packages distributed
in the past for B19 and B20.1 -- those packagers MAY have included the
compiled lib, but only as a convenience. X11R6.3 and X11R6.4 source
archives did not contain xpm themselves. In any case, when I built
libxpm-3.4k for cygwin-pre21/ v1.1, there were no other precompiled
binary packages for cygwin-pre21/v1.1 that included it.
When it came time to migrate the xpm package from the cygutils external
site into the official dist, I went looking for updated source. All I
found was the newly assimiliated version inside XFree86-4.0.0, but
discovered it was identical to the original xpm-3.4k. I considered
stopping, then -- if xfree would provide it, why should I step on their
turf? However, the Xpm dll in the cygwin-xfree project required an
Xserver. If you built an app and linked to cygwin-xfree's Xpm library,
you couldn't run your app without an Xserver. So, I grabbed the code
and built the noX version. Also, I patched it so that it would build a
dll *according to the official cygwin dll naming convention* unlike the
cygwin-free one. This had the beneficial result that there was no name
conflict with what was, at that time, "Suhaib's version".
So, my intention with the xpm package was to provide an Xless version --
specifically for use by the xless-rxvt and by native-ms-windowing
XEmacs. However (and this was discussed on cygwin-apps) I also included
an X version of the library as well, with symlinks and whatnot to allow
the user to choose which one to link against at compile time. This was
perhaps a mistake, for which I apologize.
That's all water under the bridge. I'd like to remove the xpm package
and let you guys have it. But...
a) all currently compiled binaries that depend on cygXpm-X.dll or
cygXpm-noX.dll will break. They must be recompiled to look for the
cgywin-xfree libXpm.dll (symlinks won't work. copying libXpm.dll to
cygXpm-X.dll won't work -- dll's contain their own name reference)
I am not sure how many packages this would affect. The precompiled
cygwin XEmacs at http://www.xemacs.org/ would be one. That's very
bad....
b) it will no longer be possible to build, OOB, an executable that uses
libXpm to interpret and decode Xpm formatted images/icons, but that
doesn't display them or that uses Windows GDI to draw its own display --
unless you have an Xserver running. Particularly, this means XEmacs.
Perhaps the best thing to do is for me to modify the "xpm" package to
ONLY include the noX version. However, since both the X version and the
noX version should use the same header (with different #define options,
certainly) this would require some modification to your official xpm.h
to allow both to work together. Is that acceptable?
To summarize:
cygwin-xfree should distribute the official X version of libXpm.dll
for cygwin
the "xpm" package should contain the noX version, only
header files should be harmonized
the "xpm" package should be renamed "xpm-noX" ?????
how can setup deal with files present in two packages...should we just
insure that they are identical and let setup overwrite each with the
other?
many apps will break -- incl. the "official" XEmacs -- unless
cygXpm-noX.dll is retained.
--- VERY BAD.
many apps may break if cygXpm-X.dll is replaced by libXpm.dll
--- possibly ok?
--------------------------------------
-----SECTION 3
--------------------------------------
dll naming convention. how the linker works.
See the cygwin-apps mailing list for the full discussion. I summarize
the results here:
dll's: /usr/bin/cyg<package><maj-ver>.dll
import libs: /usr/lib/lib<package>.dll.a
static libs: /usr/lib/lib<package>.a
For whatever reason, cygwin-xfree chose not to follow this protocol.
That was their decision -- and it had the serendipitous side effect that
my Xpm dll and their Xpm dll didn't conflict. (until now?)
At least one poster claimed that native windows tools build "foo.dll"
and "foo.lib", not "libfoo.dll". Not entirely true; for example, the
MikTeX distribution of netpbm ( http://www.miktex.org/ ,
http://www.ctan.org/tex-archive/systems/win32/miktex/util/netpbm/ ) uses
libppm.dll, libpng.dll, libtiff.dll (and the exception that proves the
rule, zlib.dll)
There seemed to be some confusion in this thread as to how cygwin's
linker finds libraries and dlls, I'll summarize here:
The linker, when given the argument "-lfoo" will look for the library in
the following order:
1. In the first directory in the library search path,
a. libfoo.dll.a
b. foo.dll.a
c. libfoo.a
d. cygfoo.dll (*)
e. libfoo.dll
f. foo.dll
(*) this target is added because gcc's spec file calls ld with the
"--dll-search-prefix=cyg" option. Note that this only works for
versionless dll's, or if the version is explicitly specified (-lfoo5 ==
cygfoo5.dll).
2. go to next directory in search path and repeat.
Note that the linker will first search for "import libs", then the
static lib, and then as a fallback it will attempt to use DJ's magic
"generate virtual import lib" stuff inside binutils and link directly to
the dll. This virtual import lib stuff is NOT supposed to be used in
general, except as an emergency fallback. You're supposed to provide
import libs with your packages.
Anyway, most of my "forks" are merely attempts to
a) make the library build as a dll and as a static lib -- updated
makefiles, .def files, etc. This requires more patching than I would
like because until VERY recently, libtool didn't play nicely with the
cygwin dll-building process
b) add cygwin-specific documentation.
One area where my dll's differ from the cygwin-xfree project: I
explicitly use __declspec() on exported functions and variables, but
cgywin-xfree does not. My way can lead to "__imp__symbol not found"
errors unless you're careful with your headers (*), but is more
"correct" on pei386. (also, you MUST use __declspec() on exported
global variables; I'm unsure how cygwin-xfree avoids that issue, unless
XFree86 libs don't export any global vars)
(*) by default, the headers corresponding to all libraries I've dll-ized
and contributed to cygwin, are set up to be "careful" and allow proper
linking to the dll with no extra action by the user except #include the
header as normal and "-lfoo" as normal.
In any case, I almost always submit my changes back to the maintainers
(libpng has adopted them completely, for instance). Some maintainers
incorporate them, others do not. What am I supposed to do when they
don't accept them? Say, "Sorry cygwiners but Sam Smith of the foobar
project rejected my patch. I'm giving up; no dll for libfoobar. You'll
have to live with the static lib if it builds at all." In some cases
the maintainer is MIA. What then?
I really resent this criticism about "forking" when there really doesn't
seem to be any alternative AND the "fork" is minimal in nature: minimum
changes necessary to build a working shared lib with a BRAIN-DAMAGED
Microsoft-induced format. I really really resent the psuedo-ownership
asserted, in which I'm somehow not ALLOWED to provide a port of an
independent package as a service to the entire community, merely because
the proprietor of a HUGE 21MEG project wants to bundle it inside his own
mega project.
This is not to belittle cygwin-xfree; it's a great project and I use it
myself daily (okay, I use the libs and client progs. I'm not brave
enough for the server yet; I use X-Win32). I just disagree when smaller
but useful-by-themselves projects like zlib are assimilated, borglike,
into the collective.
About my other "forks"
libpng -- all changes submitted (and accepted by) png-develop
gettext -- all changes submitted, but maintainer decided to wait for
autoconf improvements to support dlls on cygwin, rather than use my
changes. However, it's been a long time and that STILL hasn't
happened. Meanwhile, cygwin has a gettext shared lib (which wouldn't be
possible without the "fork")
ncurses -- changes submitted back to maintainer
jpeg -- changes NOT submitted (why? because there will never be
another jpeg release without major rewrite for jpeg2000 support. jpeg-6b
is frozen, there will be no jpeg-6c. jpeg2000 support may be in a
future library, but it won't be "libjpeg".)
jbigkit -- changes NOT submitted back to maintainer. I probably
should, but development seems dead.
readline -- changes submitted back to maintainer.
In any case, most of the changes I've made to these packages are the
BARE minimum to (a) get them to compile with full functionality on
cygwin (b) install and name themselves in the "official" cygwin way (c)
dll-ize themselves.
In many cases, maintainers are reluctant to absorb the changes necessary
to support cygwin/windows' whacked mechanism of supporting dlls. Your
choice: "fork" w/ shared libs, or no "fork" and static lib only (or no
lib at all, in some cases).
--------------------------------------
-----SECTION 4
--------------------------------------
zlib and xpm belong to XFree. That Charles person is a jerk for
stealing "our" packages and releasing them separately.
zlib: not true. never has been true. zlib is its own package. XFree
copied the code into its codebase long ago, but even then Xfree's
version was not the official one, anymore that the one inside newlib is
official.
xpm: NOW it is true, but it was not true back when I first ported xpm to
cygwin-pre21/v1.1. those were the days of X11R6.4 which did NOT include
xpm. Now the xpm sources inside XFree-4.0.x are considered canon,
however.
'nuff said. This is a dead horse.
--------------------------------------
-----SECTION 5
--------------------------------------
freetype.
back in the days of cygwin-pre21/v1.1, I decided to port all of Andy
Piper's usrlocal tarball (for cygiwn B20.1) to the new cygwin
snapshots. Because of internal changes to cygwin, ALL libraries had to
be recompiled, so I jumped in as a service to the community. My
usr-local tarball for v1.1 was 35.1M uncompressed, and contained
autoconf-2.13
automake-1.4
bzip2-0.9.5d
cvs-1.10
>> freetype-1.3
gdbm-1.8.0
gettext-0.10.35
groff-1.11.1
inetutils-1.3.2
jbig-1.0
jpeg-6b
libcrypt
libpng-1.0.5
login
man-1.5g
ncurses-5.0
readline-4.0
rxvt-2.6.1
tar-1.12+bzip2
texinfo-3.12
tiff-v3.5.2
unzip-5.32
vim-5.5 no GUI
>> xpm-3.4k (includes two libXpm's: one is not dependent on X)
zip-2.2
zlib-1.1.3
Many of these original components have been obsoleted by later
contributions to the official cygwin distro -- most by others, but some
I have ported, recompiled, packaged up, and added. Anyway, freetype-1.3
was NOT part of X11R6.4 (nor was xpm-3.4k); also, for the most part, the
freetype libraries didn't depend on X at all, and only a few of the
freetype executables did. Anyway, the fourth release of my usr-local
tarball contains only:
automake-1.4 ("recently" OBSOLETED)
freetype-1.3 (static lib only)
ncftp-3.0.1 ("recently" OBSOLETED)
rxvt-2.6.2
unzip-5.41 ("recently" OBSOLETED)
wget-1.5.3 ("recently" OBSOLETED)
zip-2.3 ("recently" OBSOLETED)
leaving only rxvt and freetype. Since Steve O. is soon to contribute
his X-less rxvt, I WAS planning to dll-ize freetype (perhaps moving to
freetype2). And then usr-local would be extinct. But, I've been
putting that off for a long time since it was going to be a very
difficult job...I'm **OVERJOYED** to be able to cross freetype off of my
list, and make my usr-local dist obsolete NOW.
BTW, I *think* freetype is still an independent project (like zlib, but
unlike xpm). I don't think it is correct for cygwin-xfree to be
"proprietary" toward the freetype library. (e.g. Suhaib's threat *)
HOWEVER, for the purposes of the CYGWIN port, we can, I think, all agree
that the official freetype dll/lib/headers for cygwin will be provided
by the cgywin-xfree team and not by some external effort.
For my part, I agree not put forth that external effort. :-)
(*) Suhaib said, "[if] libraries which are already being used by
Cygwin/XFree86, like libfreetype, fonts, and xpm, start showing up in
contrib I will rather take off" This is silly. XFree86 doesn't own
libfreetype or zlib (it only recently assimilated xpm). My contribution
of zlib and xpm was not intended to "creating headache for developers on
this list" -- but to help all cygwin users, many of whom needed zlib and
xpm support but did not own or want or need an Xserver; recall that the
cygwin-xfree Xserver, at the time, was non-functional. However, in the
interests of harmony, I personally, and I hope everyone else, will agree
that cygwin-xfree will be the final arbiter when it comes to
libfreetype.
--------------------------------------
-----SECTION 6
--------------------------------------
cygIPC
cygIPC can't be added to cygwin officially (for copyright reasons; I
don't own the copyright, and the original authors have dropped off the
net). OTOH, cygwin-python relies on cygIPC -- they just point people
to the CygUtils site for it, and support the python users of CygIPC
themselves. (There's good news coming, though: work on including SysV
IPC mechanisms into cygwin1.dll itself is ongoing...but it must be a
"clean room" implementation. Nobody who has looked at cygIPC code can
contribute to that effort :-( )
cygwin/xfree is much more important an application than is python, IMO.
I'd hate to see usage and acceptance of cygwin/xfree slowed by
unncessary and confusing-to-newbies reliance on an external dependency
(cygIPC).
--Chuck Wilson
cygwin zlib maintainer
cygwin xpm maintainer (not for much longer...?)
CygUtils site owner: http://www.neuro.gatech.edu/users/cwilson/cygutils/
CygIPC maintainer
extremely overworked, cycle-limited grad student.
More information about the Cygwin-xfree
mailing list