This is the mail archive of the
systemtap@sourceware.org
mailing list for the systemtap project.
Re: Questions about kprobes implementation on SMP and preemptible kernels
>It doesn't seem like preemption can be triggered in scenario quoted
>from preempt-locking.txt ...can somebody explain this?
Sure looks like the documentation doesn't match the code. I went
back to the oldest 2.6 kernel I had around (2.6.6) and it had the
same code as above. As long as interrupts are disabled, a CPU
can't be rescheduled no matter what.
It would be nice to have some concrete clarification on this point.
Is the docs to be trusted regardless of the code, or are the docs
out of date?
If the documentation is out of date, does it suggest that at some
point there was a possibility of being preempted while interrupts are
disabled?
>The abort
>mode handling code-path however seems to have been opened up for
>preemption (upon entry into SVC mode). In case of prefetch abort,
>the processor sensitive information has already been gathered
>in registers r0 and r1 before which interrupts are enabled.
No, it hasn't. Not all of the processor sensitive information has
been gathered. That's my point. There is other information in the
case of an exception that exists outside of the CPU register file
(r0-r14). That information may reside in coprocessor registers
like the MMU or other system state, or in our case, in kprobes data
structures.
What is going to use that processor specific info?
>So preempting them seems safe.
I don't think so, not until their individual handlers have a chance
to say so.
If such handlers needed such processor specific stuff, then they'd
collect it before enabling interrupts (like by calling .LCprocfns in
case of __dabt_svc). The call to notify_page_fault() in
do_translation_fault() is actually misleading. I had put it there just
to hide kprobe calls from within fault.c. I wanted a generic way to
enter kprobe_exceptions_notify() and chose the notifier chain as was
done for some other archs. It really doesn't mean that we can register
new handlers and let them read processor specific info. If they need
such info then it must be collected before re-enabling interrupts in
__pabt_svc and have it passed to do_translation_fault(). It can then
push that info to notify_page_fault() and the concerned handler can
get the right processor info, even if it was scheduled out to another
processor in between.
Data and prefetch aborts are special in their behavior as they open
themselves up for preepmtion. But you're right, there are handlers
that do need to pick up info like floating point or debug register
values, which don't make it into pt_regs. Such handlers can be
registered for undefined instruction exception on ARM, for example.
But for them preemption is disabled (by keeping interrupts disabled)
leading up to calling those handlers.
>Similarly, in __dabt_svc, the processor specific data gathering
>routines are called before interrupts are enabled. So preemption
>seems safe there as well. I've just skimmed through that part
>of entry-armv.S, so kindly correct me if I've missed something
>apparent.
I still don't think this works right on SMP ARM systems. There is
other CPU-sensitive context information that can be lost on a task
switch. Whether or not it break the general case, it does break
our use in the kprobes case. If a reschedule to another processor
happens between the enable in __pabt_svc and the call to
notify_page_fault() call in do_translation_fault(), it's game over.
Does my explanation and scenario make sense?
Yes it does. Luckily, both prefetch and data aborts restore the
original interrupt state before calling their abort handlers. When
they're invoked by a page fault from a kprobe user handler, interrupts
are originally off, so the code-path leading up to notify_page_fault()
remains non-preemptible.
Regards
Abhishek Sagar