possible problem with memory allocation using calloc/mmap/munmap

[Stanislav Kascak]

> > > > > > > > It seems that when mmap() is called with length argument exceeding
> > > > > > > > size of file, only memory to fit that file is allocated. munmap()
> > > > > > > > however frees the full specified length. [...]
> > > > > > > [...]
> > > > > > > I know this situation is unsatisfying, but I have no easy workaround
> > > > > > > to allow this.  Cygwin could add the anonymous mapping on the next
> > > > > > > 64K boundary on 64 bit, but that would result in a hole in the mapping
> > > > > > > which seemed like a rather bad idea when porting mmap to 64 bit.
> > > > > > >
> > > > > > > Ken's also right that munmap is doing the right thing here.  If
> > > > > > > anything's wrong, it's mmap's workaround for mappings beyond the file
> > > > > > > length.  If only 64 bit would allow 4K-aligned mappings :(
> > > > > >
> > > > > > Thanks for the answer. It is appreciated.
> > > > > > I understand the problem and difficulty to resolve it. Maybe returning
> > > > > > an error from mmap (and putting a comment to code for its reason)
> > > > > > would be sufficient. mmap caller could just adjust requested
> > > > > > allocation size to file size. Without error, caller has no way of
> > > > > > knowing memory was not allocated and segfault is then thrown in an
> > > > > > unrelated memory segment which makes the root cause hard to track
> > > > > > down. But, I do not know all the implication that could result from
> > > > > > that, so evaluation of this approach is up to you.
> > > > > [...]
> > > > > Eventually Cygwin adds another mapping to fullfill the entire mapping
> > > > > request:
> > > > >
> > > > >  |-- file 4K --|-- filler 60K --|-- filler 192K --|
> > > > >
> > > > > The problem on WOW64 and real 64 bit is that it's impossible to map
> > > > > the first filler.  However, this area in the VM will *never* be
> > > > > allocated by other application functions due to the allocation
> > > > > granularity of 64K!
> > > > >
> > > > > So my workaround for 64 bit and WOW64 is to just skip allocating the
> > > > > first filler:
> > > > >
> > > > >  |-- file 4K --|-- THE VOID 60K --|-- filler 192K --|
> > > > >
> > > > > The advantage is now that the following munmap of 256K will only
> > > > > unmap the map for the file and the filler, but not the region you
> > > > > calloced before, which formerly was accidentally mapped to the
> > > > > filler region.  This just can't happen anymore now.
> > > > >
> > > > > Would that be feasible?  If so I can push my patch and create a
> > > > > developer snapshot for testing.
> > > >
> > > > Two questions arise when I'm thinking about workaround solution:
> > > > - what happens if caller tries to write to |-- THE VOID 60K --|. Since
> > > > this is unallocated, would there be a segfault?
> > >
> > > Accessing the VOID would raise SIGSEGV, while accessing the filler
> > > raises SIGBUS.  The latter is also used to implement MAP_NORESERVE,
> > > which the VOID can't support.
> >
> > I played around a bit and I can confirm it would be consistent with
> > current behavior:
> > memwrite <0 - filesize) - no error, written to file
> > memwrite <filesize - 4k) - no error, no file change
> > memwrite <4k, 64k) - SIGSEGV
> > memwrite <64k, mmap alloc size) - SIGSEGV or another mem alloc
> > overwrite (depending on whether there is another allocation)
> > With workaround last line would be fixed to SIGBUS (along with proper
> > allocation length). I believe this is completely OK.
> >
> > >
> > > > - is it possible that some subsequent mem alloc request would return
> > > > region from |-- THE VOID 60K --| which could again cause segfault
> > > > after munmap?
> > >
> > > No, as stated above.  Allocations are restricted to Windows' 64K
> > > allocation granularity.
> >
> > I apologize. I missed that sentence. So, your workaround seems fine.
>
> Please try the latest snapshot from https://cygwin.com/snapshots/
> Just replacing the Cygwin DLL is sufficient.

My test programs work fine and also openldap crash that led me to this
issue is gone (openldap tests still fail, but on a different place
after the one above and now with a SIGBUS which already gives a good
hint as to where to look).
I think your patch works quite well. Thanks a lot.

Stanislav Kascak

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple