Re: Optimal kernel semantics for memory allocators and other users?

[Konstantin Serebryany <konstantin dot s dot serebryany at gmail dot com>]

+Dmitry Vyukov

On Wed, Apr 24, 2013 at 11:20 PM, Carlos O'Donell <carlos@redhat.com> wrote:
> Motohiro-san,
>
> At the Linux Foundataion Collaboration Summit 2013 you
> asked Roland and I several questions, but one of them
> had to do with your recent VM related work.
>
> The question itself was "What kind of kernel features
> would allow an optimal implementation of malloc?" I'm
> paraphrasing because I don't remember the exact question.
> More specifically: "Do the semantics of vrange allow more
> optimal memory management in malloc?"
>
> I've included Konstantin, since he jumped into the
> conversation at lunch and had another useful case for
> this kind of memory management. In his case asan needs
> lots of memory for measuring usage information but
> would rather the memory go away under memory pressure
> instead of crashing the application with OOM.
>
> I located the following "vrange" patch:
> https://lkml.org/lkml/2013/3/12/105
>
> The answer to your question is "Yes," and "No."
>
> Yes, because vrange gives us "Want but don't need"
> semantics that asan and malloc use. That is to say
> that they would like the memory but don't need it
> and the kernel can feel free to take it, and later
> give back a zero page or the original page.
>
> No, because the most optimal implementation would
> have a return value for vrange, which would tell
> the application if vrange was returning zero-filled
> pages or the existing pages for the range.
>
> This way if malloc pins memory again and gets back
> a "zero-filled pages" result then it can avoid
> zeroing memory in the case of a calloc() call.
>
> The rest of the implementation looks good to me
> and if it's faster than MADV_DONTNEED then it would
> make sense to prototype a change to support vrange
> and then benchmark the results with ebizzy before
> and after.
>
> Does this make sense?
>
> Cheers,
> Carlos.