This is the mail archive of the libc-alpha@sources.redhat.com mailing list for the glibc project.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Re: glibc aio performance


Amos,

    Comments are below:

----- Original Message -----
From: "Amos P Waterland" <waterland@us.ibm.com>
To: "Don Capps" <don.capps2@verizon.net>
Cc: <libc-alpha@sources.redhat.com>; "Thomas Gall" <tom_gall@vnet.ibm.com>
Sent: Monday, June 03, 2002 2:27 PM
Subject: Re: glibc aio performance


> > 1. There is only one disk and multiple threads don't help
>
> Even with a single disk, shouldn't rapidly enqueing many I/O operations
> enable the disk scheduler to greatly improve random access throughput by
> using reordering etc?  That is, if a single thread does this
(pseudo-code):
>
>     b[0] = read(); b[1] = read(); ... ; b[n] = read();
>
> the disk scheduler does not get the i-th read request until the (i - 1)-th
> has completed, so the best that it can do is predictive scheduling (as you
> point out: great for sequential access, but bad for random access).  But
if
> the thread does this:
>
>     b[0] = aio_read(); b[1] = aio_read(); ... ; b[n] = aio_read();
>
> then the disk scheduler will have approximately n I/O operations to work
> with in scheduling the head seek pattern.  So shouldn't there be some
> instances in which using multiple threads on a single disk with AIO
> outperforms SIO?  (I tried to test this hypothesis with IOzone, and the -i
> 2 test with AIO does seem to outperform -i 2 with SIO.)

    I tried to craft my previous email so that sequential I/O would
not likely benefit from multiple outstanding I/O operations due
to the read-ahead in the O.S. In any case, you are correct that
if the access behavior is non-sequential (random) and one has
multiple outstanding requests, and the operating system has
a disk sort routine, (or the controller or disks have sorting
at the command tag queue level) then indeed one can benefit
from multiple outstanding requests even if there is only one
disk.

>
> > 5. Normal read/write have a nice fast I/O completion notification model
> that is implemented in the
> >    operating system. POSIX async I/O was a "group think" design and has
a
> very poor I/O completion
> >    model that slows things down. Polling for I/O completion or signals
> was and is a very poor design.
> >    Most vendors have custom async I/O routines that have a fast I/O
> completion
> >    mechanism. (call back notification)

Hmmm... The rub here is that the ratio of operating systems that has
support for POSIX real-time signals to POSIX aio is a very
small number. This impacts portability in a very significant
way.  Also, the latency of signals (realtime or not) is probably
not as low as other strategies. :-( For example: The latency
of a signal delivery in a monolithic kernel may be pretty good,
but the latency of signal delivery in OSF1-AD is an
order of magnitude higher :-(
Since Iozone runs on a large number of operating systems it
tries to use mechanisms that are portable and widely
in use. POSIX real-time signals fall a bit short of the mark.


>
> While I am not completely enamoured with the POSIX design, I do believe
> they did include a facility for callbacks.  I wrote a small program that
> shows how to use POSIX real-time signals to pass a signal handler an
> arbitrary pointer upon completion of an AIO write:

This is very nice. I appreciate any and all code examples.  They
stimulate my thought processes. Thank you.

Note: If you want some fun, try compiling this on say:
    AIX, Windows, and HP-UX and see if it runs, or
    even compiles :-) (I will)

Thanks,
Don Capps




Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]