Re: malloc crashes for values >3500

[Jeff Johnston <jjohnstn at redhat dot com>]

On 01/02/2013 05:29 PM, Martin Osterloh wrote:
> Hi all,
>
> I have a working newlib port for my x86_64 OS. I can do malloc() without
> any problems if the size of the memory I request is less than ~3500. I
> have enough memory available - far more than 3500.
>
> I am just wondering if anyone of you guys had a hint? My sbrk
> implementation is the one that gets shipped with newib:
>
> char * sbrk(int nbytes);
> {
>    char        *base;
>
>    if (!heap_ptr)
>      heap_ptr = (char *)&_end;
>    base = heap_ptr;
>    heap_ptr += nbytes;
>
>    return base;
> }
>
> My page size is 4096. One thought that comes to my mind is that I reach
> the end of a page and I fail to allocate a new page?
>
> I am keen to hear your opinions.
>
> Thanks,
> --Martin

The memory/sbrk model you are using assumes that the heap / stack is 
allotted a single piece of contiguous space at the start of the program 
with the heap growing up and the stack growing down within this space. 
If the heap + stack size go above the allotted space, you will have a 
collision (e.g. your malloc'd space is overwriting the stack or vice 
versa).  If you need more storage, you are out of luck so you must make 
a good guess at the start.

So, you need to verify where the heap starts (&_end symbol), where the 
stack starts, (i.e. how much contiguous space has been allocated at the 
start of the program), and where the stack and heap pointers are when 
you have a crash.  You can prevent a collision during a malloc by adding 
a check inside the sbrk routine, but if stack growth ultimately causes 
the collision (e.g. recursive routine), there isn't a way to prevent 
this since the stack change will just be a change to the stack pointer 
register and no checks are made in the generated code.  You might be 
able to catch when it occurs using a debugger or something like 
SystemTap, but this will be very intrusive to your program's performance.

One possibility would be to try upping the amount of storage you start 
with.  Another thing you could do is separate the stack and heap and 
replace sbrk with a more dynamic version that gets chunks of storage 
from your OS (if it is more sophisticated) when needed thereby allowing 
you to increase the heap as much as possible and avoid a heap/stack 
collision which you might not see the effects of right away.

-- Jeff J.