Re: Concurrent linking plan for gold

[Daniel Jacobowitz <drow at false dot org>]

On Sun, Dec 28, 2008 at 02:54:09PM -0800, Ian Lance Taylor wrote:
> Interface

I like the interface generally.  One thing I disliked; having to
touch files to inform the incremental linker that they're unchanged.
How about having a tool "gold-notify" instead, which could use the
open-writable/close pair?  I think that won't affect modification
time.  Or if it does, gold-notify could use the pipe instead
(opening the pipe even while using inotify shouldn't be a problem).

I spent a little while thinking about the no-files-changed case.
Because you have a dependency from object files to starting the
incremental linker, make will always consider the destination program
changed.  In a stand-alone concurrent linker, I think you're out of
luck.  If you have both concurrent and incremental linking, though,
gold could delay work until it finds a file that did change
according to incremental linking.

> The more general case, in which the linker can process the files in
> any order, is more complicated.  Here is a sketch of how it might
> work.  Naturally these operations are all conditional on doing a
> concurrent link.  This more general approach will not work when using
> a linker script.

This approach also causes each build to produce a different output
file.  That means concurrent linking with this approach is only
suitable during development, not for deployment.  Our workflow here
does mostly deployable builds, so I wonder if there's any middle
ground to take advantage of concurrency but still produce stable
output.

How do text and data sizes compare for the applications you're
interested in?  If we moved linker-generated sections to the end of
the text segment, we could write out text as it became available and
queue data until the end.  That's still not possible on all platforms,
either because of ABI requirements or limited branch range; I'm
not sure how to handle inserting branch islands without having all
symbols.

Also, do you have any interest in the special case of concurrent
linking when all files are available, i.e. parallel linking?  One
advantage of that mode is that it would not need build system changes.

-- 
Daniel Jacobowitz
CodeSourcery