cygwin patches integrating back into standard gnu

[cwilson]

Edward S. Peschko wrote:
>>See, to build a shared lib, you really really need to use libtool-devel, 
>>which is libtool-1.5, and which requires automake > 1.5.0 and autoconf > 
>>2.50.  However, those packages are just now -- after 1.5 years -- coming 
>>into widespread use, because

> that's just silly. Gnu builds are uniformly prefix-friendly, and there
> was a simple way to 'make a development environment' for any given platform,
> then you could have your autoconf-2.13 (old) environment, and your autoconf-2.5
> (new) environment based on path. And you would hack on one or the other 
> based on what environment you were in. It'd probably be easier to 
> port between the two, too.

HA HA HA HA HA!  You're living in a dream world.

Sure, I can build autoconf-2.13 to install into, say, 
/usr/autotool/stable, and automake-2.5x to install into, for instance, 
/usr/autotool/devel.  No problem.  [Taken a look at your cygwin 
installation lately? Hmm?]

And then, those packages will always and forever look in 
/usr/autotool/*/share/aclocal[-VER]/ for .m4 and .am macros.  (btw, the 
[-VER] thing is a fairly recent [~ 1yr] put in place to help automake 
coexist with itself)

But other packages -- gettext, libiconv, etc -- don't put their m4 
macros there.  No, they put their macros into their OWN 
${prefix}/share/aclocal/ directory -- usually /usr/share/aclocal.  But 
my two autoconf's (extend to automake, libtool) won't look there.

Unless I then, for every package that I develop, set my PATH 
appropriately AND remember to aclocal/autoconf with the 
-I/usr/share/aclocal flag whenever I re-autotool.  OR, I could add a 
feature to standard autoconf/automake/etc so that it looks in a 
site-specifiable list of directories, in addition to 
/usr/autotool/*/share/aclocal[-VER].

Oh wait.  I did that.  Two years ago.  And got the patch accepted into 
official automake-1.5/6.x.  What were you doing then?  Basking in how 
well all GNU packages are prefix-friendly?

Oh, and one other thing: autoconf, automake, and libtool -- in the past 
-- all needed to be installed with the same prefix.  So if you had two 
intallations of autoconf with prefixA and prefixB, then you also need to 
install automake into both prefixA and prefixB, ditto libtool.  This 
restriction has been slightly relieved recently...but if you've got old 
enough versions then you still have to deal with this issue.  That's why 
cygwin provides
   autoconf-2.13 + automake-1.4p6 + libtool-1.4.3
and
   autoconf-2.5x + automake-1.7.x + libtool-1.5

> Which - I might add, cygwin does *not* lend itself to given that each
> build.sh script that I saw had hard-coded paths in it for building 
> to /usr/bin, non-standard steps for install, etc. 

And your favorite RPM .spec files do what, exactly?

the build scripts are cygwin's version of .spec files; their JOB is to 
specify, according to some policy, those things that GNU packages allow 
one to specify.  Like --prefix.  At some point, somebody somewhere has 
to specify those values, or nothing actually gets compiled.  We do it in 
the build script, and follow a policy we have here hashed out.  Debian 
does it in their rules files, following their social contract policy. 
Red Hat does it in .spec files, following some internal corporate [now 
external Fedora-project] policy.

Don't like the policy?  Then manually modify the spec file. Or rules 
file.  Or build.sh file.

> If it built clean in whatever prefix you desired, you could do a 
> little substitution trick, to get binary builds for *any* path:
> 
>     1) substitute (in binary files) any prefix with a different 
>        prefix, and pad it with null characters ('/tmp/very/long/prefix' 
>        becomes '/my/prefix...........' where . is \0)
> 
>     2) substitute (in text files) any prefix with a different prefix
>        minus the null padding.
> 
> and hence have more than one cygwin environment per cygwin.dll. 

This is so fraught with error and misconception that I don't even know 
where to begin.  Plus, it's not even related to the original point of 
this thread: getting patches back upstream, and why are some cygwin 
patches so big.  So I'm not going there.

> It'd probably be wise to integrate this with setup.exe so you would 
> have the option to install packages in a non-standard place. 

Oh god no.  "Click here to automatically fsck-up all the binaries I just 
installed"  THAT's a GREAT idea!  I nominate you to support that nightmare.

> Ok, one question... If I'm going to temporarily maintain these patches, 

Huh?  Who said *you* were going to maintain them?  Aren't the cygwin 
package owners doing that already?

> it
> sounds like I'm going to need to only apply them with cygwin builds, right?
> And that they may not fit with older gnu builds?

No, not at all.  There's nothing very special about the cygwin versions 
of the new autotools.  They're just NEW(er).  (automake-devel and 
autoconf-devel are stock.  So is libtool-devel as of the 1.5 release) 
Thus, they are by definition cross-platform.

The thing is, popt (for instance) has to take the plunge.  They must 
decide "We shall now begin using modern autotools".  Fortunately for 
them, Nicholas has already done the work, if any modifications were 
required to source files (configure.ac, Makefile.am, etc) in order to 
cope with the new autotool idiosyncracies.  Just apply that portion of 
his patch, and then run the appropriate, cross-platform, NEW, autotools 
on the result.

In fact, for the most part, you could do this with a cygwin package, on 
linux or solaris or irix or whatever (and I HAVE, in fact, done 
precisely this for a number of my packages, to ensure that my patches do 
not break stuff -- usually I do this just prior to submitting my patch 
upstream for "official" inclusion):

untar -src package
./build-script prep
   -- which unpacks the actual source, AND patches it with the 
"cygwin-specific" (but not really cygwin-specific, see above) patch.

cd <the source dir>
./configure <my favorite linux options>
make
make install

and it would work.  For the most part, cygwin ports -- especially those 
that differ only in autotool-related ways -- are AHEAD of their upstream 
distributions, not forked AWAY from them.  The upstream folks will 
"catchup" when they decide to start using modern autotools.  And they'll 
probably re-solve all of the issues that I and Nicholas and Corinna and 
... have already solved.  Or they could take (the non-autotool-generated 
part of) our patches.  But it's their decision, and they'll make it in 
their own time.

>>  3) Things are slowly getting better.  Some platforms are now finally 
>>providing mechanisms where both autoconf-2.13 and autoconf-2.5x can 
>>coexist.  (Cygwin has been doing this for years, but Red Hat et al took 
>>a little longer)   Plus, every week that goes by, another upstream 
>>maintainer "takes the plunge" -- opening the way for our patches to go 
>>back.  This trend is now (finally) accelerating.
> 
> 
> well, its nice to see that there are formal mechanisms, but the method 
> for doing this (two environments based on multiple paths) has been around
> for a while..

Yes, and that is exactly what we have -- and have had "for some time" -- 
on cygwin.  /usr/autotool/*/ -- with one added wrinkle (now shared by 
Mandrake and Red Hat).  We also provide a wrapper version of each tool 
in /usr/bin -- which analyzes the configure.in file and determines the 
"correct" version of the underlying tool to invoke.  That way, you often 
don't even have to set your path to force "devel" or "stable".  And it 
Just Works(tm).

But it is not as simple, with the autotools, as "compile and install 
autoconf/automake/libtool with different prefixes".  There needs to be 
some additional support because autotool-controlled packages have 
"maintainer mode" -- where they may re-invoke the autotools during build 
phase...please take the time to think about this, and the possible 
pitfalls (I have, and painful experience led to the current solution.  I 
YEARN for the day when it is no longer necessary.  Say, when we're 
having this discussion about autoconf-3.4 and autoconf-4.0.  But as long 
as autoconf-2.13 is part of the picture, it DOES NOT have the necessary 
hooks...and I can't wait until it dies.)

We ("the cygwin platform") solved these problems years ago -- but few 
individual developers on other platforms were motivated to do so.  So it 
became one more hurdle to overcome when "the popt people" were 
considering whether to upgrade to new autotools (let's face it: cygwin 
is a little fish in a VERY big pond.  What Linux developers want (keep 
the same autotools) matters more than what a few, or one, cygwin 
developer wants (upgrade popt to official new autotools).  Because there 
were no point-n-click solutions to the compatibility issues, "the popt 
people" faced (linux)developer backlash.  Which slowed adoption of the 
new autotools.  Which means that the cygwin patches for many packages 
languished, waiting for the upstream maintainers to become confident 
enough...to take the plunge.

The compatibiltiy -- and PATH/M4PATH/-m4-path/--include-path -- issues 
are really something that should have been solved at the platform or 
tool level.  That is, by (take your pick)

a) the autotool people themselves
    -- there were a few tentative steps in this direction.  They 
accepted my dirlist patch.  They added the [-VER] suffix to aclocal 
directories and ${bindir} binaries.  But this took time.  We (cygwin) 
couldn't wait -- or we'd still be using mostly static libraries for KDE 
and everything else.  And even if there were solutions straight from the 
autotool developers for all three autotools, it wouldn't help TODAY -- 
it would help tomorrow, when we're fighting over autoconf3.6 and 4.0.

But they didn't really pursue this too strongly -- instead, they focused 
on attempting to make the transition smooth ('autoupdate', etc).  They 
ignored the network-stasis effects (essentially, a classic 'deadlock' 
problem: you first, no you first...)

b) distro packagers -- Red Hat, Mandrake, Debian, et al.  Again, they 
slowly have begun doing this -- but only slowly.  It's just now gotten 
to the point that many individual developers are willing to install both 
autoconf_213 and autoconf_25x RPMs/DEBs/etc on their system, and "switch 
hit" for different projects.

Read the gcc and binutils mailing list archives over the past six months 
for evidence of this progress -- and the difficulties involved, and the 
backlash even a cautious approach can generate.

>>Oh god no.  Automated patch-spam to mailing lists?  I can't think of a 
>>better way to ensure that our patches are rejected.
> 
> 
> I think you misunderstand me. You would:
> 
[snip]
> 
> So its not a 'dumb patch-spammer'; its a 'grand central station' for patch
> submission (and it probably would have utility outside of cygwin).

It's a time sink for some poor soul, and waste of my time (why go thru a 
central switching station -- Chicago O'Hare -- when I can fly direct 
from MyTown to UpstreamPackageMailingListTown faster and with less effort?)

> It more directly fits my needs, as someone who casually works with 
> environments, finds things that I need, don't like, or want to add, yet
> don't want to go through the extra work of subscribing to lots 
> of lists and find it easier to maintain my own patches instead. I doubt
> I'm the only one who fits this category.

So you want someone else to manage all of this, for all of those 
packages, SEPARATE from (a) the originator of the patch and (b) the real 
mailing list for that package.

So that YOU don't have to?  (And wait.  What are the current cygwin 
maintainers doing so badly that you want to take over and redo their 
jobs for them?)

Look.  Patches should go back to the upstream package. There should not 
be a bunch of extant, uncommitted patches laying about -- for ANY 
platform.  IF there are, it *means* something:

1) the patch just isn't ready for prime time (so how would a pumpking 
help -- unless he's also a superdeveloper who speaks many computer 
languages and is supremely knowledgable about all 7000 freshmeat 
packages -- so that she can fix 'em up.  Not bloody likely.

2) The patch just isn't that valuable to the current maintainers.  If 
the experts on a given package aren't interested in a patch, why should 
your pumpking care about it?  Obviously the "Real" maintainers are more 
knowledgable about their package than the pumpking jack-of-all-trades 
who is managing patches for 7000 packages...

3) The patch has been rejected by the current maintainers.  Ditto.

4) The current maintainers have been hit by a bus.  Or Real Life 
Happened.  Abandonware.  Whatever.  In this case, a pumpking isn't 
needed -- a fork, or a new maintainer, is needed.

5) The originator of the patch hasn't sent it upstream.  Perhaps she 
believes it isn't ready, or isn't cross-platform enough.  Or perhaps he 
just doesn't want to bother.  In that case, see SUGGESTION below.

And FINALLY, we already have several people/groups who, despite all my 
arguments above, ARE DOING WHAT YOU WANT.   You know who they are?

Distribution maintainers.

That is, Red Hat has a bunch of hired hands and volunteers who marshall 
patches and manage build recipes (spec files) for 3000 packages, so that 
they build cleanly on a half-dozen architectures.

Ditto ditto Mandrake.

Ditto ditto ditto Debian.

Ditto every one of the other 500 GNU/Linux distros out there.

And ditto cygwin.

Instead of re-do-ing our (cygwin package owners) jobs, why don't you 
just let us DO our jobs.  If you're antsy to see our patches go upstream 
-- well, so are we, for the most part.


Here's a SUGGESTION

Instead of proposing grand new layers of bureaucracy requiring time, 
dedication and server resources and which PLACE BARRIERS between the 
patch originator and the people who might actually integrate them into 
the official dists, simply contact a cygwin maintainer about a package 
you're concerned about, and offer to be a patch-shepherd for them. 
Offer to help them navigate their patches -- the parts that are 
cross-platform, at least, or are merely autotool related -- thru the 
rocky shoals that are the upstream mailing lists.

Or be patient, and let us do our jobs.

--
Chuck



--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/