Re: RFC: ideas for release notes 0.9.X

[Mark Wielaard <mjw at redhat dot com>]

Hi,

On Wed, 2009-03-25 at 14:20 +0100, Mark Wielaard wrote:
> We wanted to push out a new (0.9.5?) release today/this week. I updated
> the NEWS file with some things. And below are some suggestions for the
> release notes. Please read through them and suggest additions/fixes.
> Note that there are a number of XXX in there that need to be
> replaced/filled in. I am sure I missed some cool new stuff, there were
> more than 225 commits since 0.9.

I added some suggestions from people, a list of PRs resolved since the
0.9 release date, another man page misunderstanding fixed up and some of
the kernels/architectures people have been testing on today.

Frank felt the address relocation issue (PR10000) at least was a
blocker. Are there any other blocker bugs?


The SystemTap team proudly announces release 0.9.5.

 Kernel tracepoint support, standard tapset functions and probe now
 have manual pages, typecasting supports module search paths, on-file
 flight recorder support, probe each instruction (block) step of a
 process, syscall wrappers support, security fixes, tested against
 latest 2.6.29 kernels.

 = Where to get it

   http://sourceware.org/systemtap/ - our project page   
   http://sourceware.org/systemtap/ftp/releases/systemtap-0.9.5.tar.gz
   git commit [XXX - should refer to release-0.9.5 tag?]

 = How to build it

   See the README and NEWS files at 
   http://sourceware.org/git/?p=systemtap.git;a=tree
   Further information at http://sourceware.org/systemtap/wiki/

 = Systemtap frontend (stap) changes

 - On-file flight recorder is supported. It allows stap to record huge
   trace log on the disk and to run in background.  Passing -F option
   with -o option runs stap in background mode. In this mode, staprun
   is detached from console, and stap itself shows staprun's pid and
   exits.  Specifying the max size and the max number of log files are
   also available by passing -S option. This option has one or two
   arguments seperated by a comma. The first argument is the max size
   of a log file in MB. If the size of a log file exceeds it, stap
   switches to the next log file automatically. The second is how many
   files are kept on the disk. If the number of log files exceeds it,
   the oldest log file is removed automatically. The second argument
   can be omitted.

   For example, this will record output on log files each of them is
   smaller than 1024MB and keep last 3 logs, in background.
     % stap -F -o /tmp/staplog -S 1024,3 script.stp

 - In guru mode (-g), the kernel probing blacklist is disabled, leaving
   only a subset - the kernel's own internal kprobe blacklist - to
   attempt to filter out areas unsafe to probe.  The differences may
   be enough to probe more interrupt handlers.

 - Variables unavailable in current context may be skipped by setting a
   session level flag with command line option --skip-badvars now
   available. This replaces any dwarf $variable expressions that could
   not be resolved with literal numeric zeros, along with a warning
   message.

 - Both kernel markers and kernel tracepoint support argument listing
   through stap -L 'kernel.mark("*")' or stap -L 'kernel.trace("*")'

 - Users can use -DINTERRUPTIBLE=0 to prevent interrupt reentrancy in
   their script, at the cost of a bit more overhead to toggle the
   interrupt mask.

 - Added reentrancy debugging. If stap is run with the arguments
   "-t -DDEBUG_REENTRANCY", additional warnings will be printed for
   every reentrancy event, including the probe points of the
   resident and interloper probes.

   For a full overview of the stap frontend see man stap(1).

 = Systemtap script language changes

 - New probes process().insn and process().insn.block that allows
   inspection of the process after each instruction or block of
   instructions executed. So to count the total number of instructions
   a process executes during a run do something like:
   $ stap -e 'global steps; probe process("/bin/ls").insn {steps++}
              probe end {printf("Total instructions: %d\n", steps);}' \
          -c /bin/ls
   This feature can slow down execution of a process somewhat.

 - Typecasting with @cast now supports modules search paths, which is
   useful in case there are multiple places where the type definition
   may be found.  For example:
     @cast(sdev, "scsi_device", "kernel:scsi_mod")->sdev_state

 - Kernel tracepoints are now supported for probing predefined kernel
   events without any debuginfo.  Tracepoints incur less overhead than
   kprobes, and context parameters are available with full type
   information.  Any kernel 2.6.28 and later should have defined
   tracepoints.  Try the following to see what's available:
    $ stap -L 'kernel.trace("*")'

   For an overview of all probes, functions and variables see
   man stapprobes(3stap), man stapfuncs(3stap), man stapvars(3stap)

 = Documentation

 - Systemtap probes and function man pages extracted from the tapsets
   are now available under 3stap. To show the page for the probe
   vm.pagefault or the stap function pexecname do:
     $ man [3stap] vm.pagefault
     $ man [3stap] pexecname

 = Miscellaneous changes

 - Default to --disable-pie for configure. Use --enable-pie to turn it
   back on.

 - Improved sdt.h compatibility and test suite for static dtrace
   compatible user space markers.

 - Some architectures now use syscall wrappers (HAVE_SYSCALL_WRAPPERS).
   The syscall tapset has been enhanced to take care of the syscall
   wrappers in this release.

 - Security fix for CVE-2009-0784: stapusr module-path checking race.


 = Code contributors for this release

   Ananth N Mavinakayanahalli, Breno Leitao, Dan Horak, Dave Brolley,
   David Smith, Don Domingo, Eugeniy Meshcheryakov, Frank Ch. Eigler,
   Josh Stone, Lubomir Rintel, Mahesh J Salgaonkar, Mark Wielaard,
   Masami Hiramatsu, Maynard Johnson, Rajan Arora, Roland McGrath,
   Stan Cox, Tim Moore, Wenji Huang and William Cohen


 = Examples of tested kernel versions

 - 2.6.18 (ia64, i686, x86_64-xen)
 - 2.6.27.19(i686)
 - 2.6.29-rc8 (i586)


 = Know issues with this release

   [XXX]


 = Problems resolved for this release

 PR4311 support probing without debuginfo via symbol tables
 PR4862 -rt kernel compatibility for locks
 PR4901 tcp_sendmsg arguments changed in 2.6.23-rc2
 PR5634 support pointer type punning
 PR5951 process lifetime/memorymap monitor
 PR6899 offer option to avoid disabling interrupts during more probe
        handlers
 PR6921 tracepoint support
 PR6950 buildok/twentyfive.stp fails with elfutils 0.137
 PR7071 optional $context variables
 PR7092 uprobes probes causing OOM
 PR9692 stap-{client,server}-connect fail to build on Debian
 PR9718 utrace - ftrace demo consumer
 PR9719 improve type mismatch messages
 PR9770 stap -l 'process(bla).mark(*)' should print marker names
 PR9788 permissions error in staprun
 PR9821 Support strftime like format for output file
 PR9866 save_stack_trace code in runtime stack* needs more than
        CONFIG_STACKTRACE
 PR9875 building rpm fails in checking for unpackaged file
 PR9876 Syscall tapset broken on architectures that use
        SYSCALL_WRAPPERS
 PR9882 drop Core from Fedora name
 PR9883 static markers in userspace libraries segfaults
 PR9896 Systemtap source compilation failure on latest Ubuntu
 PR9919 array overflow error points to wrong statement
 PR9922 need to configure with --disable-pie on ubuntu
 PR9927 2.6.29-rc7 - kernel crash with sharedbuf.exp stap test script
 PR9930 produce more information about probe reentrancy
 PR9932 @cast module search path
 PR9935 New sdt.exp test fails for mark probes with more than 4
        arguments
 PR9936 server.exp  hangs
 PR9947 lockdep triggered on rawhide with tracepoints
 PR9951 runtime error: error: â_vâ may be used uninitialized in this
        function
 PR9959 tracepoints should gloss over unrecognized parameter types
 PR9960 tracepoints should support complex parameters by-value
 PR9964 Can't find seperate debuginfo file  for user space probing
 PR9967 raise MAXSKIPPED for -DINTERRUPTIBLE!=0 case
 PR9974 adapt to utrace renaming of utrace_attached_engine ->
        utrace_engine,
 PR9980 sdt.h doesn't compile against gcc 4.3.2 on i386
 PR9989 utrace or task-finder crash in upstream utrace git tree
 PR9993 2.6.29-rc8-next-20090323 tracepoint build-breaking changes
 PR10006 CVE-2009-0784: staprun race condition, privilege escalation


 = Test results on various systems

   After running "sudo make installcheck" from the test suite, on a
   suitably equipped machine (kernel debugging data and other stuff
   installed), you should see 500-600 passes and a small handful of
   failures.