Re: Release announcement suggestions for 0.7

[Jim Keniston <jkenisto at us dot ibm dot com>]

On Wed, 2008-06-18 at 22:32 +0200, Mark Wielaard wrote:
> Hi,
> 
> I came up with the following as ideas for when we want to make a new
> release. It should list all major changes since the last numbered
> release (0.6.2) in March. Although I am sure I missed some things. And I
> might have misunderstood some changes or made some mistakes trying to
> summarize. So comments more than welcome.

Impressive.  See comments below.  (I haven't read everybody else's
comments, so there may be duplications or conflicts with those.)

Jim

> 
> I'll also add the changes to NEWS (if they aren't already in there, lots
> of info actually came from the NEWS file) unless people point out any
> mistakes.
> 
> I didn't really know what to put under Known Issues. Unless people can
> think of some urgent things to list there I will remove that section.

Maybe just include the URL for the bug list.  I think that since the bug
list is public, we don't need to go into detail about each outstanding
bug.

> 
> I listed all the bugs fixes since 0.6.2, plus their resolution. But I
> think I will remove the ones that are marked DUPLICATE, INVALID,
> WORKSFORME and WONTFIX since they are probably not really interesting.

My main concern is that a resolution of DUPLICATE doesn't tell the
reader whether the bug is fixed or not.  I'd probably add a second line
with that info -- e.g.,

5954  DUPLICATE translator support a structure-dumping command 
	duplicate of #3672, not yet implemented
4843  DUPLICATE translator systemtap doesn't support probing assembler
                           functions anymore 
	implemented as part of enhancement #4311

Otherwise, I don't feel strongly, but would lean toward leaving them in.

> 
> There are lists of test results (make installcheck) on various systems I
> had access to. The 2.6.25.6-55.fc9.i686 results don't look that good
> though and are from a qemu-kvm instance. On real hardware I have not
> been able to get a full make installcheck run because the machine will
> randomly hang. Till now I have been unable to get to the bottom of this
> issue (and strangely in a virtual environment everything seems fine).

Seems like something that we'd need to fix before doing the release.
It's not the kind of problem you can bandaid with a release note.

> 
> Maybe we should include some results on other architectures so people
> know what to expect?

For a while, at least some people reporting test results tagged each
failure with the associated bug number.  (If there was no PR, they
opened one.)  I'm a little leery of publishing test results without that
info.  But that's a big job -- certainly not in your bailiwick.

> 
> Cheers,
> 
> Mark
...
> = Systemtap frontend (stap) changes
> 
> - A probe listing mode is available.
>   % stap -l vm.*
>   vm.brk
>   vm.mmap
>   vm.munmap
>   vm.oom_kill
>   vm.pagefault
>   vm.write_shared
> 
> - Partial support for probing without debuginfo via symbol tables
>   (See --kelf, --kmap, --ignore-vmlinux, and --ignore-dwarf documentation)
> 

I'm not sure what you mean by "partial."  It's tested only on i386,
x86_64, and powerpc, which you should probably mention.  Certain
constructs (such as $var) aren't supported, but never will be.  Phase
III (dwarfless support for user-mode probing) is not yet implemented,
but other uprobes-related support isn't either.

How about this?
- Support for limited kernel probing without debuginfo via symbol
  tables -- on i386, x86_64, and powerpc so far.  (See --kelf, --kmap,
  --ignore-vmlinux, and --ignore-dwarf documentation)

There's also a wiki page about dwarfless probing.

BTW, some folks use x86 to mean just the 32-bit version (i.e., what used
to be arch/i386), while others use it to mean both i386 and x86_64.  I
think the latter is more common.  It'd be good to be consistent about
that.

> - The environment variable SYSTEMTAP_DEBUGINFO_PATH now refers to a
>   base directory where kernel and module debuginfo when not installed
>   in the default location.
> 
> For a full overview see the stap(1) manual page.
> 
> 
> = Systemtap script language changes
> 
> - Globals now accept ; terminators
>   global odds, evens;
>   global little[10], big[5];
> 
> - The vector of script command line arguments is available in a
>   tapset-provided global array argv[].  It is indexed 1 ... argc,
>   another global.  This can substitute for of preprocessor
>   directives @NNN that fail at parse time if there are not
>   enough arguments.
>       printf("argv: %s %s %s", argv[1], argv[2], argv[3])
> 
> For a full overview see the SystemTap Language Reference manual.
> 

I don't see the argv[] stuff in the Lang Ref doc.

> 
> = Systemtap probe points changes
> 
> - More user-space probe types are added:
>   probe process(PID).begin { }
>   probe process("PATH").begin { }
>   probe process(PID).thread.begin { }
>   probe process("PATH").thread.begin { }
>   probe process(PID).end { }
>   probe process("PATH").end { }
>   probe process(PID).thread.end { }
>   probe process("PATH").thread.end { }
>   probe process(PID).syscall { }
>   probe process("PATH").syscall { }
>   probe process(PID).syscall.return { }
>   probe process("PATH").syscall.return { }
> 
> - .statement("func@file+line") probes are now supported to allow a
>   match relative to the entry of the function incremented by line
>   number.  This allows using the same systemtap script if the rest
>   of the file.c source only changes slightly.


How about this?
.statement("func@file+line") probes are now supported to allow
probing by line-number offset relative to the "{" that begins the
function.  This allows using the same systemtap script without change
if source changes are confined to other areas of the .c file.

> 
> For a full overview see the stapprobes(5) manual page.
> 
> 
> = Standard tapsets functions changes
> 
> - Added functions to access different user space data types:
>   user_string_n, user_string_n2, user_string_n_warn, user_string_n_quoted,
>   user_short, user_short_warn, user_int, user_int_warn, user_long,
>   user_long_warn, user_char, user_char_warn.
> 
> - Added "cpu register" and "numbered function argument" support
>   function which help writing "dwarfless" probes.

s/function/functions/

>   register:long (name:string), u_register:long (name:string),
>   int_arg:long (n:long), uint_arg:long (n:long), long_arg:long
>   (n:long), ulong_arg:long (n:long), longlong_arg:long (n:long),
>   ulonglong_arg:long (n:long), pointer_arg:long (n:long), s32_arg:long
>   (n:long), u32_arg:long (n:long), s64_arg:long (n:long), u64_arg:long
>   (n:long), asmlinkage:unknown (), fastcall:unknown () and
>   regparm:unknown (n:long).
> 
> - New function returnval:long () for usage with syscall tapset probes
>   as counterpart to the returnstr:string (returnp:long) function.
> 
> - New function user_mode:long () that returns 1 if the probe point
>   occurred in user-mode.
> 
> For a full overview see the stapfuncs(5) manual page.
> 
> 
> = Standard tapset global variable changes
> 
> - NULL is now a new global variable (defined as global NULL = 0).
> 
> For a full overview see the stapvars(5) manual page.
> 
> 
> = Systemtap examples changes
> 
> - examples are now stored in testsuite/systemtap.examples.  Examples

s/- e/- E/

>   are now explicitly run during make check and make installcheck to
>   make sure they are always up to date.
> 
> - New examples:
> 
>   * io_submit.stp - When a reschedule occurs during an AIO io_submit
>     call, accumulate the traceback in a histogram. When the script
>     exits prints out a sorted list from most common to least common
>     backtrace.
> 
>   * traceio.stp - Every second print out the top ten executables
>     sorted in decending order based on cumulative I/O traffic

s/decending/descending/

>     observed.
> 
>   * iotop.stp - Every five seconds print out the top ten executables
>     generating I/O traffic during that interval sorted in decending

s/decending/descending/

>     order.
> 
>  * disktop.stp - Get the status of reading/writing disk every 5
>    seconds, output top ten entries during that period.
> 
>  * sigkill.stp - The script traces any SIGKILL signals. When that
>    SIGKILL signal is sent to a process, the script prints out the
>    signal name, the desination executable and process ID, the
>    executable name user ID that sent the signal.
> 
>  * thread-times.stp - Sets up time-based sampling. Every five seconds
>    it prints out a sorted list with the top twenty processes with
>    samples broken down into percentage total time spent in user-space
>    and kernel-space.
> 
> For a full overview see the stapex(5) manual page.
> 
> 
> = Miscellaneous changes
> 
> - Much faster batch unregistering of (thousands of) kprobes implemented
>   (needs linux 2.6.25).
> 
> - Stack backtraces for x86 and x86-64 are generated by a dwarf
>   debuginfo-based unwinder based on the code from <jbeulich@novell.com>.
>   This should give more accurate backtraces.
> 
> - Scripts relying on probe point wildcards like "syscall.*" that expand
>   to distinct kprobes are processed significantly faster than before.
> 
> - Shiny new Vim highlighting (see vim/syntax/stap.vim).
> 
> - stap now warns about read-only variables.

How about this?
- stap now warns about variables read but not set ("read-only
  variables"), which are usually due to script typos.

...