This is the mail archive of the binutils@sourceware.org mailing list for the binutils project.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Re: Plan for incremental linking with gold


Daniel Jacobowitz <drow@false.org> writes:

> Just some thoughts; I hope they're useful.

Thanks for the comments.


> On Tue, Dec 23, 2008 at 06:30:23PM -0800, Ian Lance Taylor wrote:
>> For additional flexibility, the options --incremental-changed and
>> --incremental-unchanged may be used.  Every input file which appears
>> between --incremental-changed and --incremental-unchanged will be
>> assumed to have not changed since the last incremental link.  This may
>> be used by build systems which have extra knowledge about which files
>> have changed.
>
> Is there an extra negative in this description, or are changed and
> unchanged swapped?  ISTM that you'd want --incremental-changed,
> --incremental-unchanged and --incremental-unknown (or default).

Yes, they were reversed, and --incremental-unknown is a good idea.


> Would you have to wrap each changed or unchanged file without changing
> order?  Since if you do change the link order the binary could be
> semantically different.  But incremental linking can reorder anyway,
> so it would be better to identify order-sensitive sections in the
> linker.  Or order-insensitive sections, I suppose; glibc has custom
> section names which are order sensitive, IIRC.

I was thinking that changing the order of the input files on the
command line would not be permitted, so, yes, the invoker would have
to wrap each file as appropriate.


>> These restrictions could be sometimes avoided with some additional
>> work.  However, they are not the most important case, and the
>> algorithm is considerably simplified if we do not have to consider
>> changes to symbol overriding.
>
> I wrote an incremental linker several years ago.  The goals were
> different than in gold; it was designed to produce binaries with
> minimal deltas, for binary patching of deployed systems.  So it was
> worth it to work a bit harder at link time to reduce delta; we
> measured success in the unchanged proportion of the binary.  Handling
> symbols which moved from one file to another was tricky.  As you've
> written below this is similar to adding or removing a file.  The
> problem I ran into was that this happens for COMDAT groups
> corresponding to potentially inlined functions.  They appear or
> disappear at the compiler's whim, so support for this was pretty much
> mandatory.  For the sort of C++ code that you're probably concerned
> with at Google, I'd expect this to make a big impact.

That is a good point.  COMDAT sections should be handled separately
from the requirements about symbols being defined and undefined.
Since COMDAT sections don't require the linker to choose the first
instance, it's OK for an object to define a COMDAT symbol it did not
define before if the COMDAT symbol was previously defined by some
other object file.  We can continue using the definition from the
other object file, whether that file is changed or not.


> We did similarly for sections; added some padding around sections
> likely to change (not applicable for gold's goals) so that they could
> grow in place.  We also kept track of holes in the binary caused by
> decreased section size to fill things in later without excessive code
> growth; I see that in your plans already.

Padding around sections is also in the plan, though maybe not spelled
out.


> Maybe some day a similar mode (--incremental-thorough?) can be added
> to gold.  I might be able to get permission to release the code if
> you're interested; let me know.

I would certainly be interested if you think it would be helpful.  I
haven't seen the code for any incremental linker.  Which linker were
you modifying?


>>   * If archive:
>>     + Treat each member as a changed file.
>
> Should be trivially easy to do better than this?  ar has timestamps.

Good point, those would probably be good enough.


>>   * If linker script:
>>     + Recompute everything.
>
> It's not appropriate for your goals, but there's ways around this; it
> just complicates layout a bit.  You need to be able to answer the
> question "does this placement violate the new linker script".

Right, but I don't find this to be a very interesting case for gold,
since it normally uses no linker script at all.  And it's not like the
linker script changes very often even for the cases where you need
one.

> Fortunately I was using ARM's scatter-loading files at the time; a
> strict interpretation of GNU linker script format does not give the
> linker a lot of freedom to reorganize.  I think this is a drawback of
> the current format.  If I ever find time to hack on gold for embedded
> use, I've been thinking about adding a simpler format that leaves the
> linker more freedom.

I think that would be great.


>> Handling debugging information which changes size may require
>> modifications to gdb to support non-contiguous debugging information.
>> (Or maybe we will just always put the debugging information at the end
>> of the file).
>
> Have you considered leaving debug info in object files, as Apple does?
> With this --incremental designed for development use only, packaging
> is less of a problem.

Yes, I would like to implement that in gold, and it would certainly be
an option for incremental linking.  However, I do want the incremental
linker to work for the case where debug info stays with the
executable.

Ian


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]