This is the mail archive of the
systemtap@sources.redhat.com
mailing list for the systemtap project.
Re: cost of kprobe and jprobe operations
On Tue, Feb 22, 2005 at 02:16:38PM -0500, William Cohen wrote:
<snip>
> 156 11.2069 memcpy
> 142 10.2011 restore_all
> 127 9.1236 _spin_lock
> 119 8.5489 kprobe_handler
> 110 7.9023 do_debug
> 105 7.5431 kprobe_exceptions_notify
> 93 6.6810 setjmp_pre_handler
> 90 6.4655 longjmp_break_handler
> 77 5.5316 __module_text_address
> 58 4.1667 resume_execution
> 51 3.6638 _spin_unlock
> 43 3.0891 notifier_call_chain
> 39 2.8017 int3
> 37 2.6580 get_kprobe
> 26 1.8678 do_int3
> 22 1.5805 debug_stack_correct
> 17 1.2213 debug
> 12 0.8621 _spin_lock_irqsave
> 9 0.6466 lock_kprobes
> 4 0.2874 _spin_unlock_irqrestore
> 4 0.2874 poison_obj
> 4 0.2874 unlock_kprobes
>
> -Will
On Tue, Feb 22, 2005 at 05:20:17PM -0500, William Cohen wrote:
> Frank Ch. Eigler wrote:
> >Hi -
> >
> >
> >
> >>[...]
> >>These performance discussions might be more meaningful if
> >>we had explicit performance goals for systemtap. [...]
> >>Any thoughts on defining performance goals for systemtap?
> >
> >
> >I don't think picking numbers does harm, but I don't know how
> >much good that does either. Would we stop-ship a systemtap
> >implementation that didn't meet the goals? In the abstract,
> >should one opt to optimize vs. extend functionality? Maybe
> >these questions should be put aside until something runnable
> >exists as a point of reference, when actual decisions might
> >need to be made.
> >
> >
> >- FChE
>
> I generated these number to give some concrete numbers on the cost of
> the operations. We do want to be aware of the costs of the operations.
> There are certainly going to be some points where instrumentation is
> going to cause excessive overhead.
>
> There are other performance issues with kprobes. There is a single
> spinlock used by all kprobes and this is going to serialize the probes.
> This will have a impact on smp scalability.
>
> -Will
Very interesting numbers.. As earlier mentioned by Prasanna, I think
we can use RCU to reduce the lock overhead. The golbal spinlock is for
kprobes hash table and it is read-mostly data structure. As of now I think
we can use RCU in this case. I will try to come up with a patch using RCU
for kprobes hashtable. Hopefully that will improve the numbers.
Thanks
Maneesh
--
Maneesh Soni
Linux Technology Center,
IBM India Software Labs,
Bangalore, India
email: maneesh@in.ibm.com
Phone: 91-80-25044990