This is the mail archive of the
systemtap@sourceware.org
mailing list for the systemtap project.
[Bug runtime/13639] New: Bad interaction between itrace and stap_stop_task_finder
- From: "mjw at redhat dot com" <sourceware-bugzilla at sourceware dot org>
- To: systemtap at sourceware dot org
- Date: Wed, 01 Feb 2012 10:27:37 +0000
- Subject: [Bug runtime/13639] New: Bad interaction between itrace and stap_stop_task_finder
- Auto-submitted: auto-generated
http://sourceware.org/bugzilla/show_bug.cgi?id=13639
Bug #: 13639
Summary: Bad interaction between itrace and
stap_stop_task_finder
Product: systemtap
Version: unspecified
Status: NEW
Severity: normal
Priority: P2
Component: runtime
AssignedTo: systemtap@sourceware.org
ReportedBy: mjw@redhat.com
Classification: Unclassified
On the mjw/atomic_create branch (which I will soon merge with master) I am
sometimes seeing the following issue on 3.3.0-0.rc1.git6.1.fc17.x86_64 while
running make installcheck RUNTESTFLAGS=itrace.exp. It looks like
stap_stop_task_finder -> stap_utrace_detach_ops is trying to tear down an task
that itrace had already an utrace engine on, and the task is in the process of
disappearing. stap_utrace_detach_ops is holding the rcu_read_lock so when
utrace_barrier () sees the -ESRCH and does a schedule_timeout_interruptible
things go BOOM!
[ 150.085673] BUG: scheduling while atomic: stapio/1853/0x00000002
[ 150.086599] 1 lock held by stapio/1853:
[ 150.087250] #0: (rcu_read_lock){.+.+..}, at: [<ffffffffa02429f6>]
stap_utrace_detach_ops+0x36/0x340 [stap_f730c2426e8ad6ccb3254967339ba04c_1853]
[ 150.089359] Modules linked in: stap_f730c2426e8ad6ccb3254967339ba04c_1853
nfs lockd fscache auth_rpcgss nfs_acl sunrpc ip6t_REJECT nf_conntrack_ipv6
nf_defrag_ipv6 nf_conntrack_ipv4 nf_defrag_ipv4 xt_state nf_conntrack
ip6table_filter ip6_tables snd_intel8x0 snd_ac97_codec 8139too microcode
ac97_bus snd_pcm snd_timer snd soundcore snd_page_alloc 8139cp i2c_piix4
virtio_balloon i2c_core mii uinput [last unloaded:
stap_0d467885e93ebc4e5e3aefd434974bf3_1679]
[ 150.097308] Pid: 1853, comm: stapio Not tainted
3.3.0-0.rc1.git6.1.fc17.x86_64 #1
[ 150.098947] Call Trace:
[ 150.099574] [<ffffffff816957f4>] __schedule_bug+0x80/0x85
[ 150.101017] [<ffffffff8169f7eb>] __schedule+0x8db/0x9a0
[ 150.102440] [<ffffffff8169fbff>] schedule+0x3f/0x60
[ 150.103606] [<ffffffff8169cf92>] schedule_timeout+0x1b2/0x4d0
[ 150.105019] [<ffffffff810a271f>] ? local_clock+0x6f/0x80
[ 150.106211] [<ffffffff810710c0>] ? internal_add_timer+0x140/0x140
[ 150.107489] [<ffffffff8169d30e>] schedule_timeout_interruptible+0x1e/0x20
[ 150.108877] [<ffffffff810ef802>] utrace_barrier+0x72/0x90
[ 150.110035] [<ffffffffa0242b7b>] stap_utrace_detach_ops+0x1bb/0x340
[stap_f730c2426e8ad6ccb3254967339ba04c_1853]
[ 150.112293] [<ffffffffa02429f6>] ? stap_utrace_detach_ops+0x36/0x340
[stap_f730c2426e8ad6ccb3254967339ba04c_1853]
[ 150.115993] [<ffffffffa0242d22>] stap_stop_task_finder.part.26+0x22/0xb0
[stap_f730c2426e8ad6ccb3254967339ba04c_1853]
[ 150.118546] [<ffffffffa0243dc8>] _stp_cleanup_and_exit+0x148/0x3b0
[stap_f730c2426e8ad6ccb3254967339ba04c_1853]
[ 150.120796] [<ffffffffa024483a>] _stp_ctl_write_cmd+0x26a/0xfe0
[stap_f730c2426e8ad6ccb3254967339ba04c_1853]
[ 150.122983] [<ffffffff810c6c8d>] ? trace_hardirqs_off+0xd/0x10
[ 150.124332] [<ffffffff812c36c7>] ? selinux_file_permission+0x107/0x150
[ 150.125816] [<ffffffff812bcbec>] ? security_file_permission+0x2c/0xb0
[ 150.127316] [<ffffffff811baacf>] vfs_write+0xaf/0x190
[ 150.128623] [<ffffffff811bae0d>] sys_write+0x4d/0x90
[ 150.129770] [<ffffffff816aa369>] system_call_fastpath+0x16/0x1b
[ 155.576934] stap_6270c5cb94928b67c826ed9ed7e5104b_2027: systemtap:
1.7/0.152, base: ffffffffa0224000, memory: 60data/36text/2ctx/2058net/34alloc
kb, probes: 5
[ 155.580779]
=============================================================================
[ 155.581295] BUG utrace_engine (Not tainted): Poison overwritten
[ 155.581295]
-----------------------------------------------------------------------------
[ 155.581295]
[ 155.581295] INFO: 0xffff88004504d6a0-0xffff88004504d6b6. First byte 0xe0
instead of 0x6b
[ 155.581295] INFO: Allocated in utrace_attach_task+0xca/0x2c0 age=5512 cpu=0
pid=1856
[ 155.581295] __slab_alloc+0x450/0x4fd
[ 155.581295] kmem_cache_alloc+0x20b/0x240
[ 155.581295] utrace_attach_task+0xca/0x2c0
[ 155.581295] 0xffffffffa023fe36
[ 155.581295] 0xffffffffa024375a
[ 155.581295] start_callback+0xa2/0x110
[ 155.581295] utrace_resume+0xf4/0x140
[ 155.581295] do_notify_resume+0xb0/0xc0
[ 155.581295] int_signal+0x12/0x17
[ 155.581295] INFO: Freed in __utrace_engine_release+0x39/0x50 age=5505 cpu=0
pid=1856
[ 155.581295] __slab_free+0x3d/0x265
[ 155.581295] kmem_cache_free+0x234/0x250
[ 155.581295] __utrace_engine_release+0x39/0x50
[ 155.581295] utrace_reset+0x1e6/0x310
[ 155.581295] utrace_maybe_reap+0x14f/0x180
[ 155.581295] utrace_report_death+0x100/0x120
[ 155.581295] do_exit+0x5d3/0xa80
[ 155.581295] do_group_exit+0x4c/0xc0
[ 155.581295] sys_exit_group+0x17/0x20
[ 155.581295] system_call_fastpath+0x16/0x1b
[ 155.581295] INFO: Slab 0xffffea0001141300 objects=21 used=21 fp=0x
(null) flags=0x10000000004080
[ 155.581295] INFO: Object 0xffff88004504d680 @offset=5760
fp=0xffff88004504d800
[ 155.581295]
[ 155.581295] Bytes b4 ffff88004504d670: 00 00 00 00 00 00 00 00 5a 5a 5a 5a
5a 5a 5a 5a ........ZZZZZZZZ
[ 155.581295] Object ffff88004504d680: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b
6b 6b 6b kkkkkkkkkkkkkkkk
[ 155.581295] Object ffff88004504d690: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b
6b 6b 6b kkkkkkkkkkkkkkkk
[ 155.581295] Object ffff88004504d6a0: e0 29 81 81 ff ff ff ff 6b 6b 6b 6b 6b
6b 6b 6b .)......kkkkkkkk
[ 155.581295] Object ffff88004504d6b0: 01 00 00 00 00 00 00 00
........
[ 155.581295] Redzone ffff88004504d6b8: bb bb bb bb bb bb bb bb
........
[ 155.581295] Padding ffff88004504d7f8: 5a 5a 5a 5a 5a 5a 5a 5a
ZZZZZZZZ
[ 155.581295] Pid: 2027, comm: stapio Not tainted
3.3.0-0.rc1.git6.1.fc17.x86_64 #1
[ 155.581295] Call Trace:
[ 155.581295] [<ffffffff8119d4fd>] ? print_section+0x3d/0x40
[ 155.581295] [<ffffffff8119e15e>] print_trailer+0xfe/0x160
[ 155.581295] [<ffffffff8119e2f2>] check_bytes_and_report+0xe2/0x120
[ 155.581295] [<ffffffff8119e82f>] check_object+0x1cf/0x250
[ 155.581295] [<ffffffff810efa5a>] ? utrace_attach_task+0xca/0x2c0
[ 155.581295] [<ffffffff8169801c>] alloc_debug_processing+0x67/0x109
[ 155.581295] [<ffffffff8169850e>] __slab_alloc+0x450/0x4fd
[ 155.581295] [<ffffffff810efa5a>] ? utrace_attach_task+0xca/0x2c0
[ 155.581295] [<ffffffff8132c0be>] ? random32+0x2e/0x40
[ 155.581295] [<ffffffff810efa5a>] ? utrace_attach_task+0xca/0x2c0
[ 155.581295] [<ffffffff811a145b>] kmem_cache_alloc+0x20b/0x240
[ 155.581295] [<ffffffff810efa5a>] utrace_attach_task+0xca/0x2c0
[ 155.581295] [<ffffffffa0228670>] __stp_utrace_attach_flags+0x70/0x220
[stap_6270c5cb94928b67c826ed9ed7e5104b_2027]
[ 155.581295] [<ffffffffa022b8c4>] _stp_ctl_write_cmd+0xcb4/0x1140
[stap_6270c5cb94928b67c826ed9ed7e5104b_2027]
[ 155.581295] [<ffffffffa022b939>] ? _stp_ctl_write_cmd+0xd29/0x1140
[stap_6270c5cb94928b67c826ed9ed7e5104b_2027]
[ 155.581295] [<ffffffff811baacf>] vfs_write+0xaf/0x190
[ 155.581295] [<ffffffff811bae0d>] sys_write+0x4d/0x90
[ 155.581295] [<ffffffff816aa369>] system_call_fastpath+0x16/0x1b
[ 155.581295] FIX utrace_engine: Restoring
0xffff88004504d6a0-0xffff88004504d6b6=0x6b
[ 155.581295]
[ 155.581295] FIX utrace_engine: Marking all objects used
--
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.