Hotspot JVM GDBJIT plugin

[Kaushik Srenevasan <kaushik at twitter dot com>]

I've been working on a Hotspot JVM JIT plugin for GDB and noticed that
there was some interest about this earlier [1]. I currently have mixed
mode stack traces working [2] and plan to at least add file and
line number support in the future.

I'm writing to see if there is any interest in pulling the
changes [3] I've had to make to GDB, to get this to work. I also
intend to ping the OpenJDK folks to try to get the GDBJIT hooks into
the official distribution.

The first of these changes [3] is a simple fix to a crash while trying to
display the return type of a JIT frame (say on 'finish'.)

The second (frame based symbol handler) is a feature added to help GDB
understand frames created by Hotspot's "template" interpreter. Hotspot
generates expansions for bytecodes from templates (once) into a buffer
and uses jump tables to handle dispatch. The same code range thus
corresponds to different functions, with the only differentiating
factor being metadata in stack frames. Neither ELF symbol tables nor
callbacks work in this case and hence the need for frame based symbol
resolution.

This adds an additional callback that the reader may choose to
implement to return the name (file path or line number) of the function
executing in the frame in question. GDB's stack walk code consults this
handler if it can't find a symbol in its global symbol table. The JIT
host inserts itself as a handler and simply forwards the call to the
reader. Symbols are only ever created lazily; when the unwinder finds
the frame on the stack. I also want to add support to display
inlined frames which I can't imagine being realized without frame
based handlers. [3] is only a rough implementation that I intend to
complete based on feedback I get from this list.

Thanks,
   -Kaushik

[1] http://sources.redhat.com/ml/gdb/2012-02/msg00025.html
[2] http://sysenter.org/~ks/gdb/gdb-jvm.png
[3] https://github.com/kaushiks/GDB/commit/98d8b224c3f48639368e91f0a531394124596a2e