Debugging help for fork failure: resource temporarily unavailable

Ryan Johnson ryanjohn@ece.cmu.edu
Wed Mar 9 17:53:00 GMT 2011


On 2:59 PM, Corinna Vinschen wrote:
> On Mar  5 17:15, Ryan Johnson wrote:
>> Might it be possible to do an LD_PRELOAD of some sort which hooks
>> into fork() at the critical moment and prints the differences
>> between /proc/$parent/maps and /proc/$child/maps? The code doesn't
>> even need to be efficient; it just needs to be able to run when
>> whatever internal helper of fork() returns an error but before the
>> nascent child process is terminated.
>>
>> If there exists such a convenient instrumentation point, I might be
>> up to the task of exploiting it, but I wouldn't know where to start.
> It's not that easy.  LD_PRELOAD is only honored after the other
> stuff to duplicate the parent process has already taken place.
>
> This is definitely not something for 1.7.9, but maybe we can utilize
> the functionality we already have on board at one point.  In
> fhandler_process.cc we have the function format_process_maps(), which
> creates a buffer with the content of /proc/$PID/maps.  It might be
> possible to call this function from fork for parent and child if fork
> fails for this reason, and print this information.
>
> Just an idea.  Somebody still would have to do it(*).
I was actually thinking of an LD_PRELOAD in the parent process which 
would cause new/additional code to execute when that parent forks. 
However, after poking around in the code I'm not sure this is possible, 
since IIRC LD_PRELOAD can only override dynamically-linked functions. 
That probably means an actual change to cygwin is required, as you suggest.

BTW, while looking at the code I noticed a potential source of remap 
problems: if B depends on A, and we remap A first, then only A's 
location will be checked carefully; B will be pulled in wherever it 
happens to end up when we do the full load of A. The code seems to 
assume that every DLL we try to remap is currently not loaded.

I'm actually not sure what would happen when time came to remap B, 
because loading it would just return the handle we didn't know we had, 
and closing that handle wouldn't take its reference count to zero.  
Incidentally, this same problem would arise if a BLODA injected a DLL 
into the process -- that DLL would be on the todo list for fork() to 
process (because it was also injected into the parent process), but 
would already be loaded by the time we try to remap it. Also, if we do 
want to force Windows not to put a dll in a certain address, wouldn't it 
make more sense to reserve the (wrong) space it went into on the first 
try? Right now if the offending location is higher than the one we want, 
nothing stops Windows from just putting it right back in its old spot 
because the code only reserves locations lower than the desired one.

Is this accurate or am I missing something here?

I assume there's a way to enumerate the dlls loaded in a given process; 
would it make sense to use a three-step algorithm?
1. Unload all currently-loaded dlls, complaining loudly to stderr or a 
log file (these are due to BLODA and deserve to be called out)
2. Load without deps every DLL and make sure it lands at the right 
address (using memory reservation tricks if needed)
3. Reload with deps every DLL. Presumably once it has landed correctly 
once it will do so thereafter (the current code assumes this, at least)

In theory, the first step might allow cygwin to resist dll injection 
(maybe on an opt-out basis?), though I don't know what the consequences 
of that choice would be.

The third step would be significantly easier if we had a dependency 
graph so that we could ensure dependencies always get processed before 
they're needed, but I don't know if that's feasible. How 
expensive/embeddable is cygcheck?

Ryan


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple



More information about the Cygwin mailing list