Address space clobbers during fork() (was Re: Extending /proc/*/maps)

[Ryan Johnson]

On 21/04/2011 12:40 PM, Chris January wrote:
>> Second, I don't know who maps locale.nls, but it happens *very* early
>> in the life of the process, long before gdb can talk to it. Using
>
> From what I can gather from http://www.alex-ionescu.com/part1.pdf and 
> Reactos source code I believe it's mapped by MmCreatePeb, called from 
> PspCreateProcess, called from NtCreateProcess{Ex}. It doesn't look 
> like mapping it in advance would help (if that was even possible), but 
> it may be possible (but risky) to map it somewhere else, update the 
> pointers in the PEB, and then unmap the original.
No luck. The former two don't seem to have symbols, and the latter 
doesn't get called.

However, having discovered the windbg 'tt' and friends, I finally nailed 
down the culprit: ntdll!ZwInitializeNlsFiles, which is a thin wrapper 
around a syscall. In addition, ntdll!RtlGetLocaleFileMappingAddress 
calls ntdll!ZwInitializeNlsFiles if it wasn't already done. 
Unfortunately, there's no way to the function because it's 64-bit...

Meanwhile, I'm starting to notice a disturbing pattern. When I use 
windbg to run my toy program, it never fails (at least, never yet). When 
I run it within gdb, it invariably fails. If I run from the command line 
it fails 10-15% of the time. Could there be some sort of timing issue 
involved here? The process seems thoroughly single-threaded at this 
point in its life, but yet the pattern is there. Does windbg somehow 
change dll load order, or something?

Further, when the fork fails, it seems due to the *parent* having a 
messed-up address space: locale.nls prefers to load at 002B0000, so a 
misbehaving parent pretty much guarantees a base address mismatch... 
unless the child also happens to misbehave.

Thoughts?
Ryan