[CT-NG] WIP - status

["Yann E. MORIN" <yann dot morin dot 1998 at anciens dot enib dot fr>]

Hello All!

The repository has been quiet lately, apart for some bug fixes and a few
contributed improvements. But this does not mean developement has stalled!

I'm currently working on three important things. The first impacts the
canadian cross compilation, the second touches the companion libraries,
while the third is about the C libraries. (In order of importance in my
point of view, does not imply any ordering on arrival!)

-------------------
TL;DR:
- canadian cross overhaul:
  - need only for existing build->target cross-toolchain
  - build the rest
- companion libraries;
  - build our own, or rely entirely on up-to-date distros
- C library:
  - simpler process, less steps
  - reunite glibc and eglibc
- testing tree will be pushed shortly

-------------------
On the canadian-cross side, we currently need to have two pre-exosting
cross-toolchains:
  - one to build the tools proper: runs on build, generates code for host
  - one to build the C library   : runs on build, generates code for target

Also, in case of canadian-cross, we avoid both of the core cc passes. But
what are the core cc good for, in the end? Building the C library for
the target, and nothing else.

So, by avoiding those two core cc, we end up having to provide a complete
toolchain just for the prupose of building the C library. That's indeed a
little bit deficient, as we already have code just for this!

So I'm rewiring the stuff internally so that we only need the first toolchain
(ie. build -> host), and use the core cc to build the target C library. So
the core cc are always running on build, and generating for target, whatever
the configuration, while the final cc is always running on host (which may
be the same as build!) and generating for target, whatever the configuration.

On the other hand, the core cc are curently used to issue the final compiler
when we are targetting bare-metal. This means that the above is not entirely
true, and the exception is bare-metal. But the current wrappers to pass-1 and
pass-2 makes it easy enough to decide on whether running on build vs. on host.

Still, we are missing yet a single part of the toolchain, the binary utilities
(namely: GNU binutils, elf2flt and sstrip) that we still have to handle to
generate the final toolchain. As we have to build the target C library, we
need binutils to run on build and generate for target, but as part of building
the toolchain, we also have to build them to run on host and generate for
target.

This last part means we have to build them twice in the canadian case. This is
quite easy, as the three binary utilities are quite easy to cross-configure
and cross-build. But it should be done with care, to avoid duplication the
build process.

Also, while we are at it, the build tools are currently wrapped into
${CT_PREFIX_DIR}/buildtools, and removed after the toolchain is built. This
is defintely not their place, and they get moved into a sub-dir of
${CT_WORK_DIR}/${CT_TARGET} (the place where the toolchain is built). Maybe
it could/should even go into ${CT_BUILD_DIR}. Anyway, we have to be able to
save and restore them (as part of the build-restart feature).

Although that is an almost independent issue, I decided to treat it in the
same run, as the canadian stuff already touches PATH, and maybe also the
place we install the core cc. So it seemed to fit.

To sum up, here is what is changing on the canadian front:
- move kernel headers just before libc headers
- move the buildtools from ${CT_PREFIX_DIR} to ${CT_WORK_DIR}
  a- in a variable CT_BUILDTOOLS_DIR still pointing to ${CT_PREFIX_DIR}/buildtools
  b- move CT_BUILDTOOLS_DIR below ${CT_WORK_DIR}
  c- move comptools to ${CT_BUILDTOOLS_DIR}
- add binutils frontend/backend
  - call the binutils backend with build -> host  -> target (as done today)
- implement the new canadian way:
  - add binutils frontend for      build -> build -> target
  - always put core_cc in PATH
    - core_binutils always do      build -> build -> target
    - core_cc always does          build -> build -> target
      - take care about cross/canadian bare-metal!
  - remove target tuple selection from the toolchain menu

-------------------
Next, the companion libraries are evolving. Distributions are catching up
with the strictly required versions (they have to if they want to bundle
the latest gccs), so this will shortly become a non-issue, and building our
own will no longer be required.

Nonetheless, there exists quite a bunch of existing stable production
machines for which it is not sensible to upgrade the system, and we still
have to support that. So building our own versions will still be needed.

So, either the distributions are up-to-date, and provide all the required
(or optional) companion libraries, and we use that, or the distribution does
not provide all of them, and we build all of them. This is an all-or-nothing
kind of choice. I do not believe there are sane cases where a distribution
has only part of the complibs, and not the others.

The complete migration process is not yet fully slated, but I have a few
ideas already. Yet, the canadian case makes it complex, as we can't use the
build system's complibs. So either we require them to be already built for
running on host, or we still build our own versions.

To sum up, here what is changing on the companion libraries front:
- decorelate complibs support vs. use from gcc
  - configuration-wise
  - code wise: remove required complibs prefix, make it optional
- add option to choose which to use
  - build our own, or
  - use those from the distribution (or am existing host 'sysroot')

-------------------
And finally, the C libraries. As of today, building a C library requires
four steps: libc_headers, libc_start_files, libc, and libc_finish.

There is no step between libc_headers and libc_start_files, so those two
can be folded into one single step. uClibc and eglibc already does this.
The mingw libc does nothing for start files, while newlib does nothing for
headers. Only glibc has two separate, different code paths that hopefully
can be reunited.

The we can have only three steps: libc_start_files (which install headers
and a few dummy/minimal .o and .so files), libc and libc_finish.

libc_finish is used by glibc to post-install some utilities. This is legacy
code dating back to long before NPTL, and the only version which still uses
that is 2.3.6 (which is now long obsolete in crosstool-NG, but still used by
the ia64 sample). If I can make that sample to build using a newer version,
then we can get rid of libc_finish altogether.

Which leaves us with two steps: lib_start_files and libc! :-)

Finally, as eglibc is a spin-off from glibc, and regurlarly resynced from
upstream, then we can try to reunite the two code paths that are really,
really similar. In the process, we can play 'drop-the-legacy-code-paths'.

To sum up, on the C library front:
- fold libc_headers and libc_start_files into one single step
- get rid of libc_finish
- re-unite glibc and eglibc

-------------------
This is currently only on my HDD, I'll push those into a test tree later
in the week.

Thank you for your attention!
And congratulations for those who read all down to here! ;-)

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

--
For unsubscribe information see http://sourceware.org/lists.html#faq