[Bug libc/4099] New: Overly agressive caching by stream i/o functions.

["green at linuxhacker dot ru" <sourceware-bugzilla at sourceware dot org>]

It seems some time ago glibc got overly aggressive caching capabilites, that is,
it tries to cache st_blksize bytes for files it deals with on reads, and
moreover, if you do a seek to some position, it also tries to preread data from
block (sized at st_blksize ) start to that seeking position.
Then subsequent reads of smaller size always read st_blksize bytes.
I can see where this (bigger reads) make sense on something like MS DOS, but not
on a system with functioning cache and readahead, linux any modern Linux system.
As such, there is a growing belief that st_blksize should be read not as
recommended i/o size,but recommended write size (which makes total sense now -
you issue big writes to avoid partial block writes that incur whole block read
penalty)
Another problem here is some filesystems want to set this value somewhat high to
force application to issue bigger writes (examples include reiserfs and Lustre,
I am sure many more would like to do this too). Now with current logic there is
a performance penalty on every fseek (why read data before current position, I
wonder? there is no way we can read data backwards), and on every small read,
esp if st_blksize is e.g. 2Mb, then not only we incur this huge reading penalty,
but we also now have wasted 2Mb of memory for essentially another cache copy of
file data which OS already has for us anyway.

I wonder if this behavior could be dropped on platforms with working readahead
and cache? Or at least some way to disable it at runtime?

There was a discussion on the topic on lkml some time ago too, that was not
answered at the time, but the issue is still hot. http://lkml.org/lkml/2006/2/28/187

-- 
           Summary: Overly agressive caching by stream i/o functions.
           Product: glibc
           Version: unspecified
            Status: NEW
          Severity: normal
          Priority: P2
         Component: libc
        AssignedTo: drepper at redhat dot com
        ReportedBy: green at linuxhacker dot ru
                CC: glibc-bugs at sources dot redhat dot com


http://sourceware.org/bugzilla/show_bug.cgi?id=4099

------- You are receiving this mail because: -------
You are on the CC list for the bug, or are watching someone who is.