This is the mail archive of the systemtap@sourceware.org mailing list for the systemtap project.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Re: Differences of stack traces between linux kernel and systap


On Sat, May 03, 2008 at 12:35:41AM +0800, Peter Teoh wrote:
> Notice the diference in the stack trace between systap:
> 
>  0xc047ecf1 : slab_pad_check+0x1/0xb9
>  0xc064dd69 : kretprobe_trampoline_holder+0x18/0x38
>  0xc064dd69 : kretprobe_trampoline_holder+0x18/0x38
>  0xc064dd69 : kretprobe_trampoline_holder+0x18/0x38
>  0xc0434e33 : send_signal+0x4a/0x10e
>  0xc0435476 : __group_send_sig_info+0x6e/0x8c
>  0xc0435a8e : group_send_sig_info+0x43/0x58
>  0xc048d42a : send_sigio+0xe3/0x133
>  0xc048d4c9 : __kill_fasync+0x4f/0x52
>  0xc048dc78 : vfs_ioctl+0x0/0x6c
>  0xf8943ef5 : snd_timer_user_tinterrupt+0x11f/0x13c [snd_timer]
>  0xf8942c74 : snd_timer_interrupt+0x1de/0x23f [snd_timer]
>  0xf896667d : snd_pcm_period_elapsed+0xf0/0x134 [snd_pcm]
>  0xf8bfef49 : azx_interrupt+0x60/0xa4 [snd_hda_intel]
>  0xc0459eab : handle_IRQ_event+0x26/0x4d
> 
> 
> And that of the linux kernel (latest patch):
> 
>  [<c042bad6>] warn_on_slowpath+0x46/0x56
>  [<c0415a33>] ? apic_wait_icr_idle+0x16/0x1d
>  [<c0415243>] ? __send_IPI_dest_field+0x50/0x54
>  [<c04020e5>] ? send_IPI_mask+0xd/0xf
>  [<c046773c>] ? get_pageblock_flags_group+0x50/0x6e
>  [<c046777e>] ? get_pageblock_migratetype+0x24/0x27
>  [<c0468472>] ? free_hot_page+0xf/0x11
>  [<c0468494>] ? __free_pages+0x20/0x2b
>  [<c047f471>] ? __free_slab+0xac/0xb4
>  [<c0480754>] kmem_cache_destroy+0xfe/0x108
>  [<f8d337c0>] nf_conntrack_cleanup+0x53/0x7a [nf_conntrack]
>  [<f8d3766d>] nf_conntrack_standalone_fini+0x1c/0x1e [nf_conntrack]
>  [<c044b56f>] sys_delete_module+0x177/0x1af
>  [<c0472c00>] ? remove_vma+0x31/0x53
>  [<c0473468>] ? do_munmap+0x182/0x19c
>  [<c0404bae>] sysenter_past_esp+0x6a/0x90
>  [<c0640000>] ? pci_scan_bridge+0x1dc/0x2eb
> 
> The difference lies in the "?" in front of the function, which is to
> indicate that this function has just been executed and returned, so it
> is no longer on the stack.
> 

I'm pretty sure that on x86 arch's the kernel's stack traces are really
more like intelligent guesses rather than guarantees.  The kernel
searches back though it's stack and checks for possibly valid function
addresses and prints those out.

Other arches (like some PPC flavors) do this much better since they have
a dedicate register for storing stack frames and pre-defined stack frame
sizes and parameters.  Those are much more trustworthy, but I find the
backtraces on x86 to be quite usable.

> Is it possible to modify the systap stacktrace dump to be inlined with
> that of the kernel?   It this not more informative?

Can you quantify the performance impact of something like this running
all the time?


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]