This is the mail archive of the cygwin-apps@cygwin.com mailing list for the Cygwin project.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Re: -src package standard: proposal #5 and #5a


Robert Collins wrote:

 >
 > Ok, this sounds reasonable... how about this script though. ====
 > #!/bin.sh
 > echo "unpack the source archive, apply the patch, and then optionally"
 > echo "build the package for you using the script found in "
 > echo "<srcdir>/CYGWIN-PATCHES/ to build this package."
 > echo "Call this script like \"cygbuild packagenameandversion\""
 > echo "add --build to cause the package to be built" ...

If for simple packages only, fine.  It's really no different than I was
saying, except that for "simple" packages, setup.exe doesn't do the
unpack/patch thing; 'cygbuild pkgnameversion' does it.  As before, for
simple packages, the "real" work -- configure, build, pack, etc -- is done
by <src>/CYGWIN-PATCHES/<some-script>.  Very debian.

But this .sh file is functionally unnecessary -- as was mine, for simple
packages.  I merely put it there to show *that it wasn't necessary*. And
because the instant FAQ would be "I tried to build foo from -src, but
where's the .sh file?" if ANY other package -- a more complicated package
-- contained a real, non-window-dressing .sh file.

However, if you intend the above script only for simple packages -- those
that are easily built by a simple "unpack,patch,run
CYGWIN-PATCHES/buildscript" sequence -- then your use of cygbuild is a mere
replacement for the functionality I assumed was to be put into setup.exe.
So, no meaningful differences there.
However, you still don't address (e.g. agree or disagree or counterpropose)
the issue of more complicated packages.  Therefore, I must assume that you
intend for that script above to be used both for simple AND complex
packages, including using "cygbuild" to handle the initial unpacking and
patching.

If I'm wrong, then the rest of this message is probably of academic 
interest only.  However, I don't think I am.  I believe you want a 
window-dressing .sh file for ALL packages, instructing builders of ALL 
packages to use 'cygbuild pkgnameversion'.

If you intend the above "script" for both simple and complex packages --
that is, tell users to use 'cygbuild pkgnameversion' for complex packages
as well -- then there are problems (the same ones I've previously
mentioned) -- unless you resort to shar and meta patches.  However, EVEN
for simple packages, you'll have problems with your "--build" option if the
pristine source archive isn't named "correctly" or packed "correctly".
These naming problems can't be fixed without either (a) repacking or
renaming the pristine source archive, or (b) providing additional metadata
somehow, or (c) add a lot more smarts to cygbuild, and keep the original
-src package around (because cygbuild will have to parse its content 
listing "What was the name of the tarball inside "foo-VER-REL-src.tar.bz2? 
Oh! foosrc.vVER.tar.gz").

You assume that the only meta-data necessary is completely contained in
"packagenameandversion".  Not so.  For instance, "jpeg-6b" comes in the
following tarball:
  "jpegsrc.v6b.tar.gz" (no, I am not kidding)
which unpacks into
  "jpeg-6b/" (but not jpegsrc.v6b/)

A human can tell that those two are probably related, but I doubt even a
human -- without a_priori knowledge of the naming rule -- will be able to
derive the .tar.gz name from the unpacked dirname, or vice versa.  A
generic computer program (like cygbuild) can't either -- without some other
source of metadata.  Unless: you're gonna parse the -src tarball 'tar tvf'
output to determine the pristine tarball's name, and then parse the
pristine tarball 'tar tvf' output to determine the srcdir name...this
implies that you keep the -src tarball around, but previously you indicated
that the -src tarball was never stored on disk, and would be unpacked
on-the-fly by setup.exe

Or you can just explicitly include the meta data in the -src tarball!  With
rpm's, meta data is stored in the spec file.  My proposal puts this meta
data in an external build script -- if and only if necessary.  It really
seems like your scheme -- and debian -- are trying to jump thru a HELL of a
lot of hoops, in the pursuit of an unnecessary goal: "I want two files and
two files only: pristine source and one big patch".

The "one big patch" leads you to (a) using shar to generate binary files
(b) including meta-patches in your "one big patch" -- the big patch creates
secondary patches and shar files...

BLEAGH.

Sure, I see WHY you want to do this -- so that all port-specific stuff is
selfcontained, and can be completely separated from the upstream "pristine"
source -- even downloaded separately.  There's another reason why you'd
want to do this -- but that can wait until I put my ranting hat on.

I don't think that goal is worth the hoops and hassles.

 > <not real script>
 > dirname = sed rule on $1
 > packagever = sed rule on $1 to remove cygwin version

Again, not gonna work with jpeg.

 > if [test -d $dirname ]; then
 >   mv dirname dirname.old
 > fi

hmm..good thought.  I should but this in my version -- but you have to
insure that dirname.old doesn't already exist...

 > tar xjf ${packagever}-src.tar.(bz2|gz)
 > cd dirname

dirname is not reliable...

 > patch -p1 ../$1.patch
 > cd ..
 > if (--build was passed)
 >  ${dirname}/CYGWIN-PATCHES/${1}.sh all
 > fi
 > ===
 >
 > That could be put in /bin by a package - call it cyghelper - and
 > called something like cygbuild, and is global for all packages. And
 > setup.exe doesn't need to extract the inner source, it can just look
 > for cygbuild and if it's there invoke it appropriately (ie without
 > --build).

Actually, with your script as is, you'd even have problems with jpeg just
unpacking and patching the source (cd dirname breaks).  That's easy to fix
(do a patch -p0 from the parentdir instead of patch -p1 from inside srcdir)
-- but accesing <srcdir>/CYGWIN-PATCHES/buildscript is not easy to fix
without additional smarts or external metadata.

Sure -- all of this is *possible* to do with the "one pristine src and one
patch only" method -- provided you jump thru enough hoops:
   shar to create binaries (.jpg files, additional tarballs)
   "one big patch" creates secondary patches, along with the buildscript
   cygbuild parses the tarballs to extract dir and filenames rather than
assuming that they follow a sane pattern (which jpeg *doesn't*)

Or, you drive it all with one external package-specific script that KNOWS
about the peculiarities of the package (contains the metadata itself).
RPM+package.spec.  bash+package.sh.  package.spec/package.sh are *custom*
to the package, and KNOW about its wackiness if any.

As you've probably guessed, I've created a jpeg-6b-5 example at
http://www.neuro.gatech.edu/users/cwilson/cygutils/packaging/ for you to
take a crack at.  (FYI: This will NOT be my real jpeg-6b-5 release; there
are other changes I want to make to this package...like autoimport and
stuff...before I make a "real" new release)

jpeg is a 'fully ported' dllized library -- it doesn't require "after
building the static lib, do a make clean, then reconfigure and apply 
<dllpatch> and then build the dll" mumbo-jumbo.
ncurses and readline DO -- which makes them harder.
But they will be hard for ANY automated build system; the right thing to
do, of course, is to complete the port so that ncurses builds both the
statlib and dll all in one step, like jpeg here, and not waste time trying
to cram the half-ported round peg into the automated build square hole.
Completing the ncurses or readline port is a job for (the new) libtool...

OTOH, if we had been following that logic -- no "mostly ported" packages
allowed -- then we wouldn't have had an ncurses or readline library for the
past two years (I'm including the old cygutils days)

<rant>
I'm starting to get a bit pissed off here.  I've spent hours writing these
emails, trying to point out the shortcomings in the debian-centric
proposal.  Each of which is answered by "oh yeah, we can work around that
using..."  Work around?!  E.g. the debian scheme isn't powerful enough on
its own, so we have to work around...

I've spent hours creating examples that demonstrate the various proposals
-- I chose a simple package (mktemp) at first because it was easy, but
unfortunately it didn't demonstrate some of the difficulties with the
debian scheme, although I have pointed out those difficulties in my emails.

Then I suggested a compromise where simple packages -- MOST packages --
could be done the debian way, but still leave enough flexibility for more
complicated ones.  This was countered by cygbuild -- which doesn't exist
yet (I hate vapor), and can't derive the requisite metadata without yet 
MORE complicated hoops (tarball parsing, etc).
[And don't get me started on the partial ports like ncurses...]

Now I've spent yet more hours creating an example using jpeg, which does
exhibit some of the problems I have warned about, and I await Robert's
brilliant use of shar and patches-that-both-patch-and-create-other-patches
to show me how it should be done.  (I'll point out that even if I WAS
inclined to use shar, it wasn't an official cygwin tool until Saturday
anyway).  But anyway, IMNSHO, that's a recipe for maintainer's migraine.  A
patch that creates a shar file that when unshared creates a tarball that
gets unpacked revealing a .ppm, a .jpg, and -- YES -- *another* patch,
which can then be applied.  (This is the lossless jpeg example).

but what if my cygwin patch should be applied AFTER the semi-official
lossless jpeg patch? (e.g. I have to patch files created by the ljpeg 
patch)?  Oops, I can't do that, because there's only one
"real" patch, and it was already applied.  So, my "real" patch, in addition
to creating the shar file mentioned previously, must also create a
secondary "apply after the ljpeg patch" cygwin-specific patch.

Umm...how do I regenerate this automatically?  @^!#$*  Now we're back to
the "two cygwin-specific patches" problem that is what makes the ncurses
and readline ports in their current two-step non-libtoolized form 
difficult.  The Debian Way makes jpeg have the same problem -- unnecessarily!

I realize, jpeg is a corner case.   But that's my point: the 
one-src+one-patch method isn't powerful enough to handle THIS corner case 
without heroic, non-maintainer-friendly efforts.

But WHY must we jump thru these hoops?  Apparently, to avoid the terrible,
horrifying possibility that we might have something OTHER than a single
pristine source tarball and a one big patch in our -src archives, because
debian is God's gift to package distribution, and anything they have done
MUST be the optimum.

Robert's compromise seems to be "okay, everybody can also put this useless
.sh file in there too (even for complex packages [jpeg,
ncurses-non-new-libtool]), which will tell you to use cygbuild."  Even when
cygbuild won't work (with complex packages) without hellacious
tarball-parsing ugliness.

I really suspect an ulterior motive, here, Robert.  When will your port of
dpkg be ready?  Since you seem intent on forcing this process to specify
The Debian Way as the official -src packaging standard for cygwin, I can
only assume that the reason is you want to use dpkg eventually for all 
cgywin-building. I don't think that's going to be very popular with the
Red Hat brass, tho.

I'm ready to be proven wrong.  Please, without repacking or renaming the 
original jpeg source tarball, show me how simple it really is to do it The 
Debian Way, with one source tarball and one patch.  Show me a generic 
cygbuild script that can handle it, lossless jpeg patch and all.  I'm sure 
it is possible -- I just really really doubt that it will be simple, or 
maintainer-friendly.
</rant>

I'm tired of this.  I'm tired of spending so much useless time
unproductively.  I'm tired of being at loggerheads here -- with the
exception of an early contribution from Corinna (which I compromised away),
and recent contributions from Gareth, this has been largely a two person
conversation -- and I'm tired of THAT, too.  I had hoped we could reach
some sort of sanity by Thanksgiving, and I had planned to spend some of my
time next weekend developing new ports.

No longer.  Unless there is some new blood in this conversation, I'm out of
it until after Thanksgiving (although I will *read* with interest Robert's
shar-and-metapatch jpeg dist)

--chuck


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]