Tracking file I/O

[Matt Walsh <walshes at gmail dot com>]

I'm trying to write a script that will kick out data I can later use
to plot disk/network I/O by process.

I am far from a kernel expert so this has been a bit of a kernel
education as well.

Like some of the examples, I have tried to tap into sys_write() / vfs_write().

When I test this with writing a big file, I get results but not
exactly what I want.

The initial fwrite results in a sys_write() that returns immediately
saying it did 99% of the write.

Then after a fflush() or fclose() the real work happens.  The
sys_write() comes back saying it's writing the remaining 1% of the
data and takes the extended duration required to do the write.

So I guess the following questions remain for me...

1) Is there a way to detect that the sys_write() is an unflushed
write, so that I can save up the amount and 'join' it to the
sys_write() that happens during the flush/close()?

(I need this, because otherwise I'll falsely report that a lot of data
was written in a VERY short period of time)

2) Is there a deeper function I should hook into that could give me
finer-grained data?  That is, if I write out a file that takes 10
seconds, I want to know how much was written every second - and with
the sys_write() hook all I get is 2 'taps' that in total take 10
seconds.  I tried tapping into 'aio_read' but systap claims it can't
find the symbol - and I'm assuming it's some kind of issue with that
aio_read is a function pointer or something.

Thanks in advance; I think this script will be interesting and will
post it when I'm done.  Basically it should give me the same data I
get from SAR but with association to processes.

-Matt