Trying stapdyn (pure-userspace systemtap with dyninst)

[Josh Stone <jistone at redhat dot com>]

Hi all,

Yesterday I finally merged my jistone/stapdyn branch into git master,
and I want to document how people can try this out.  It's still very
rough, with lots left in the stapdyn/TODO file, but brave souls are
welcome to start testing it, and perhaps start helping too. :)

A little background: the goal of this new runtime is to let stap run (in
some limited subset) without any special system or kernel privileges --
a pure userspace solution.  The library we've chosen to implement this
on is Dyninst (www.dyninst.org), which doesn't even require any new
kernel features, just normal ptrace.

There are two ways right now to build systemtap against dyninst.  If
you're on Fedora rawhide, Will Cohen has added rpms to the distro.
Install dyninst-devel, then build stap with "./configure
--with-dyninst", and it should pick up the system libraries.  (Side note
to Will - it might be nice to have a pkgconfig script to locate dyninst,
so we don't have to hardcode distro choices.)  The stap build should now
include two new artifacts in the root, dynsdt and stapdyn.

The other way is with your own dyninst build.  Clone the master branch
from http://git.dyninst.org/dyninst.git and run "./configure
--prefix=/your/root", "make", "make install".  Then build stap with
"./configure --with-dyninst=/your/root".  Running this way will probably
need you to also set "LD_LIBRARY_PATH=/your/root/lib".

Dyninst also requires an environment variable set for its runtime, e.g.
"DYNINSTAPI_RT_LIB=/usr/lib64/dyninst/libdyninstAPI_RT.so.7.99" for the
rawhide package. (I hope to make this use a sane default in the future.)

Also, since dyninst uses an executable .dyninst_heap section for its
generated code, you'll need to enable selinux's "allow_execstack".

The dynsdt program is a test one can use first to make sure things are
basically working - it instruments and prints basic traces of SDT
markers in a program and its libraries.  Try it like "dynsdt ./stap -l
begin" to run it on stap itself.

Now for using stap+dyninst itself.  First add the "--runtime=dyninst"
option to select this new mode.  Since this runs in a ptrace-attach
style, you'll need a -c command too.  (-x PID should also be possible,
just not yet.)  Note that the runtime port is still in progress, and
most tapset functions are still kernel-oriented, so it's fairly limited
right now what you can do.  But for a basic sanity check, try:
  stap -e 'probe process.function("main") { println(pn()) }' \
    --runtime=dyninst -c './stap -V'

Feedback and especially contributions are welcome. :)  I know I've
thrown up caution signs everywhere -- I'm just trying to manage
expectations while this develops.  But it's coming together!

Thanks,
Josh