[PATCH RFC] fork: reduce chances for "address space is already occupied" errors

[E. Madison Bray]

On Thu, Mar 28, 2019 at 6:50 PM Achim Gratz wrote:
>
> Michael Haubenwallner writes:
> > It will not help for conflicts between dlls within a single package while this
> > package is built.  I'm thinking of python modules built within the python package
> > itself, where the just built modules are used within the very build process.  Not
> > sure if packages using local modules during build also do use fork then, though.
>
> It does help, that's the whole point.  But you will have to rebase all
> the in-processing DLL together, as the database will only have
> information on the installed DLL.  So if you build in stages, you'll
> need to do something like incremental autorebase does and collect all
> DLL into some file that you can then feed to
>
> rebase -sOT dlls_to_rebase
>
> That is slightly less convenient than using the database in persistent
> mode, but it is much less of a headache when you want to throw things
> away and start over since you don't need to worry about cruft in the
> database file.

That is essentially what I do for incremental builds; I keep
re-running rebase between stages with roughly those same flags and
this works.

However, I can see how this could be inconvenient for some Python
builds where you might have something within the setup.py script
(which, when building Python extension modules, is still usually used)
like (in pseudo-code):

    run_build_ext_command()
    import just_built_module
    # Use just_built_module to generate some files
    run_install_command()

all within the same process.  One could work around this by modifying
the setup.py to call `rebase` as a subprocess and that should work,
but it would suck to have to make such extra considerations just for
Cygwin, much less get some upstream project to accept that.

I don't know if what I described is at all similar to Michael's case,
and I've never run into a problem with this myself (even building
Numpy or SciPy).  But I could see it happening somehow...