Contents of console log from crash of ipp020 on 2013-March-15
{{{
<Mar/15 08:35 am>[3690682.353980] general protection fault: 0000 [#1] SMP 
<Mar/15 08:35 am>[3690682.354269] last sysfs file: /sys/class/i2c-adapter/i2c-0/name
<Mar/15 08:35 am>[3690682.354269] CPU 0 
<Mar/15 08:35 am>[3690682.354269] Modules linked in: autofs4 smsc37b787_wdt i2c_i801 i2c_core e1000e iTCO_wdt tg3 libphy e1000 xfs dm_snapshot dm_mirror dm_region_hash dm_log aacraid 3w_9xxx 3w_xxxx atp870u arcmsr aic7xxx scsi_wait_scan
<Mar/15 08:35 am>[3690682.354269] Pid: 1524, comm: rpciod/0 Tainted: G        W  2.6.28-rc7-00105-gfeaf384 #4
<Mar/15 08:35 am>[3690682.354269] RIP: 0010:[<ffffffff802488f9>]  [<ffffffff802488f9>] run_workqueue+0x73/0x1ed
<Mar/15 08:35 am>[3690682.354269] RSP: 0018:ffff88063c969e80  EFLAGS: 00010087
<Mar/15 08:35 am>[3690682.354269] RAX: ffff88033cc43a30 RBX: ffff88010035d580 RCX: 0000000000000000
<Mar/15 08:35 am>[3690682.354269] RDX: 6c6c654320202020 RSI: ffff88010035d5b0 RDI: ffff88033cc43a00
<Mar/15 08:35 am>[3690682.354269] RBP: ffff88033cc43a00 R08: 0000000000000001 R09: 0000000000000001
<Mar/15 08:35 am>[3690682.354269] R10: 0000000000000000 R11: ffffffff8027dff4 R12: ffff88010035d578
<Mar/15 08:35 am>[3690682.354269] R13: 202020202020200a R14: ffffffff808b85a0 R15: 0000000000000000
<Mar/15 08:35 am>[3690682.354269] FS:  0000000000000000(0000) GS:ffffffff80872d80(0000) knlGS:0000000000000000
<Mar/15 08:35 am>[3690682.354269] CS:  0010 DS: 0018 ES: 0018 CR0: 000000008005003b
<Mar/15 08:35 am>[3690682.354269] CR2: 00007fa11c204000 CR3: 0000000128a31000 CR4: 00000000000006e0
<Mar/15 08:35 am>[3690682.354269] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000

<Mar/15 08:35 am>[3690682.354269] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
<Mar/15 08:35 am>[3690682.354269] Process rpciod/0 (pid: 1524, threadinfo ffff88063c968000, task ffff88063c954bf0)
<Mar/15 08:35 am>[3690682.354269] Stack:
<Mar/15 08:35 am>[3690682.354269]  ffffffff8024891d 0000000000000046 203a4f4544495620 20200a3120302030
<Mar/15 08:35 am>[3690682.354269]  6e49202020202020 ffff88033cc43a00 ffff88033e457bf0 ffff88033cc43a00
<Mar/15 08:35 am>[3690682.354269]  ffff88033cc43a40 ffff88033e457bf0 ffffffff808b5d60 ffffffff80249511
<Mar/15 08:35 am>[3690682.354269] Call Trace:
<Mar/15 08:35 am>[3690682.354269]  [<ffffffff8024891d>] ? run_workqueue+0x97/0x1ed
<Mar/15 08:35 am>[3690682.354269]  [<ffffffff80249511>] ? worker_thread+0xd8/0xe3
<Mar/15 08:35 am>[3690682.354269]  [<ffffffff8024bfda>] ? autoremove_wake_function+0x0/0x2e
<Mar/15 08:35 am>[3690682.354269]  [<ffffffff80249439>] ? worker_thread+0x0/0xe3
<Mar/15 08:35 am>[3690682.354269]  [<ffffffff8024becc>] ? kthread+0x47/0x73
<Mar/15 08:35 am>[3690682.354269]  [<ffffffff80234852>] ? schedule_tail+0x27/0x5f
<Mar/15 08:35 am>[3690682.354269]  [<ffffffff8020cd19>] ? child_rip+0xa/0x11
<Mar/15 08:35 am>[3690682.354269]  [<ffffffff804fa23c>] ? menu_reflect+0x0/0x6d
<Mar/15 08:35 am>[3690682.354269]  [<ffffffff8024be85>] ? kthread+0x0/0x73
<Mar/15 08:35 am>[3690682.354269]  [<ffffffff8020cd0f>] ? child_rip+0x0/0x11
<Mar/15 08:35 am>[3690682.354269] Code: 4c 8d 63 f8 48 8d 7c 24 10 48 8d 73 18 fc 4d 8b 6c 24 18 b9 06 00 00 00 f3 a5 4c 89 a5 80 00 00 00 48 89 ef 48 8b 43 08 48 8b 13 <48> 89 42 08 48 89 10 48 89 5b 08 48 89 1b e8 d1 ed 36 00 48 8b 
<Mar/15 08:35 am>[3690682.354269] RIP  [<ffffffff802488f9>] run_workqueue+0x73/0x1ed
<Mar/15 08:35 am>[3690682.354269]  RSP <ffff88063c969e80>
<Mar/15 08:35 am>[3690682.354269] ---[ end trace 24952920e41d62b6 ]---
<Mar/15 08:36 am>[3690683.731331] BUG: spinlock lockup on CPU#0, rpciod/0/1524, <Mar/15 08:36 am>

<Mar/15 08:36 am>[3690683.731331] Pid: 1524, comm: rpciod/0 Tainted: G      D W  2.6.28-rc7-00105-gfeaf384 #4
<Mar/15 08:36 am>[3690683.731331] Call Trace:
<Mar/15 08:36 am>[3690683.731331]  <IRQ>  [<ffffffff803a103f>] _raw_spin_lock+0xfb/0x122
<Mar/15 08:36 am>[3690683.731331]  [<ffffffff805b7758>] _spin_lock_irqsave+0x37/0x3f
<Mar/15 08:36 am>[3690683.731331]  [<ffffffff802492ab>] __queue_work+0x10/0x32
<Mar/15 08:36 am>[3690683.731331]  [<ffffffff802492ab>] __queue_work+0x10/0x32
<Mar/15 08:36 am>[3690683.731331]  [<ffffffff8024932c>] queue_work_on+0x37/0x40
<Mar/15 08:36 am>[3690683.731331]  [<ffffffff8058ce2d>] rpc_wake_up_task_queue_locked+0x261/0x2a6
<Mar/15 08:36 am>[3690683.731331]  [<ffffffff8058d888>] __rpc_queue_timer_fn+0x74/0xcb
<Mar/15 08:36 am>[3690683.731331]  [<ffffffff8058d814>] __rpc_queue_timer_fn+0x0/0xcb
<Mar/15 08:36 am>[3690683.731331]  [<ffffffff80241e07>] run_timer_softirq+0x169/0x1dd
<Mar/15 08:36 am>[3690683.731331]  [<ffffffff8023e144>] __do_softirq+0x7a/0x13d
<Mar/15 08:36 am>[3690683.731331]  [<ffffffff8020d07c>] call_softirq+0x1c/0x28
<Mar/15 08:36 am>[3690683.731331]  [<ffffffff8020e028>] do_softirq+0x2c/0x68
<Mar/15 08:36 am>[3690683.731331]  [<ffffffff8023e084>] irq_exit+0x3f/0x85
<Mar/15 08:36 am>[3690683.731331]  [<ffffffff8021e66d>] smp_apic_timer_interrupt+0x93/0xac
<Mar/15 08:36 am>[3690683.731331]  [<ffffffff8020cacb>] apic_timer_interrupt+0x6b/0x70
<Mar/15 08:36 am>[3690683.731331]  <EOI>  [<ffffffff805b76fe>] _spin_unlock_irq+0x21/0x22
<Mar/15 08:36 am>[3690683.731331]  [<ffffffff803f3936>] tty_audit_exit+0x60/0x92
<Mar/15 08:36 am>[3690683.731331]  [<ffffffff8023c516>] do_exit+0x1b0/0x74d
<Mar/15 08:36 am>[3690683.731331]  [<ffffffff805b83fa>] oops_begin+0x0/0x8d
<Mar/15 08:36 am>[3690683.731331]  [<ffffffff805b795d>] error_exit+0x0/0x9a

<Mar/15 08:36 am>[3690683.731331]  [<ffffffff8027dff4>] mempool_free_slab+0x0/0xe
<Mar/15 08:36 am>[3690683.731331]  [<ffffffff802488f9>] run_workqueue+0x73/0x1ed
<Mar/15 08:36 am>[3690683.731331]  [<ffffffff8024891d>] run_workqueue+0x97/0x1ed
<Mar/15 08:36 am>[3690683.731331]  [<ffffffff80249511>] worker_thread+0xd8/0xe3
<Mar/15 08:36 am>[3690683.731331]  [<ffffffff8024bfda>] autoremove_wake_function+0x0/0x2e
<Mar/15 08:36 am>[3690683.731331]  [<ffffffff80249439>] worker_thread+0x0/0xe3
<Mar/15 08:36 am>[3690683.731331]  [<ffffffff8024becc>] kthread+0x47/0x73
<Mar/15 08:36 am>[3690683.731331]  [<ffffffff80234852>] schedule_tail+0x27/0x5f
<Mar/15 08:36 am>[3690683.731331]  [<ffffffff8020cd19>] child_rip+0xa/0x11
<Mar/15 08:36 am>[3690683.731331]  [<ffffffff804fa23c>] menu_reflect+0x0/0x6d
<Mar/15 08:36 am>[3690683.731331]  [<ffffffff8024be85>] kthread+0x0/0x73
<Mar/15 08:36 am>[3690683.731331]  [<ffffffff8020cd0f>] child_rip+0x0/0x11
}}}