IPP Software Navigation Tools IPP Links Communication Pan-STARRS Links
wiki:ipp061-crash-20131017T153400

from console

File content:
 ? nfsd+0x0/0x139
<Oct/17 03:54 pm>[6853629.759017]  [<ffffffff811f0d78>] nfsd+0xa9/0x139
<Oct/17 03:54 pm>[6853629.759019]  [<ffffffff810546e2>] kthread+0x7d/0x85
<Oct/17 03:54 pm>[6853629.759021]  [<ffffffff81003794>] kernel_thread_helper+0x4/0x10
<Oct/17 03:54 pm>[6853629.759023]  [<ffffffff81054665>] ? kthread+0x0/0x85
<Oct/17 03:54 pm>[6853629.759025]  [<ffffffff81003790>] ? kernel_thread_helper+0x0/0x10
<Oct/17 03:54 pm>[6853629.759027] NMI backtrace for cpu 11
<Oct/17 03:54 pm>[6853629.759027] CPU 11 
<Oct/17 03:54 pm>[6853629.759028] Modules linked in: w83793 hwmon_vid autofs4 i2c_i801 i2c_core iTCO_wdt e1000e ioatdma dca rtc_cmos tg3 libphy e1000 fuse dm_snapshot dm_mirror dm_region_hash dm_log scsi_wait_scan qla2xxx megaraid_sas megaraid_mbox megaraid_mm aacraid 3w_9xxx 3w_xxxx arcmsr aic7xxx aic79xx sr_mod sg ata_piix ahci sata_sis sata_nv sata_via pata_sl82c105 pata_sis
<Oct/17 03:54 pm>[6853629.759042] 
<Oct/17 03:54 pm>[6853629.759043] Pid: 2921, comm: pclient Tainted: G      D    2.6.34.4 #3 S7010/empty
<Oct/17 03:54 pm>[6853629.759045] RIP: 0010:[<ffffffff812c08b7>]  [<ffffffff812c08b7>] delay_tsc+0x2d/0x59
<Oct/17 03:54 pm>[6853629.759048] RSP: 0000:ffff8806552a3dd0  EFLAGS: 00000012
<Oct/17 03:54 pm>[6853629.759049] RAX: 000000000000001c RBX: 00000000042e42cb RCX: 000000008158fc08
<Oct/17 03:54 pm>[6853629.759051] RDX: 000000000040ee78 RSI: 000000008158fc24 RDI: 0000000000000001
<Oct/17 03:54 pm>[6853629.759052] RBP: ffff8806552a3dd8 R08: 000000000000000b R09: 000000000000000b
<Oct/17 03:54 pm>[6853629.759054] R10: 0000000000000000 R11: ffff8806552a3c08 R12: ffff8806552b3c00
<Oct/17 03:54 pm>[6853629.759055] R13: 0000000000000000 R14: ffff8803ce096120 R15: ffff8803ce0965a8
<Oct/17 03:54 pm>[6853629.759057] FS:  00007fe0e24916f0(0000) GS:ffff8806552a0000(0000) knlGS:0000000000000000
<Oct/17 03:54 pm>[6853629.759059] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<Oct/17 03:54 pm>[6853629.759060] CR2: 000000000186bb18 CR3: 000000062112d000 CR4: 00000000000006e0

<Oct/17 03:54 pm>[6853629.759062] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
<Oct/17 03:54 pm>[6853629.759063] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
<Oct/17 03:54 pm>[6853629.759065] Process pclient (pid: 2921, threadinfo ffff880<Oct/17 03:54 pm>fff8803ce096120)
                 <Oct/17 03:54 pm>[6853629.759066] Stack:
<Oct/17 03:54 pm>[6853629.759067]  ffffffff81019da9 ffff8806552a3de8 ffffffff812c08fe ffff8806552a3e28
<Oct/17 03:54 pm>[6853629.759069] <0> ffffffff812c4e65 000000009ef2e6b4 ffff8806552b3c00 ffff880023799030
<Oct/17 03:54 pm>[6853629.759071] <0> 0000000000013c00 ffff8806552a3e98 000000000000000b ffff8806552a3e48
<Oct/17 03:54 pm>[6853629.759074] Call Trace:
<Oct/17 03:54 pm>[6853629.759075]  <IRQ> 
<Oct/17 03:54 pm>[6853629.759077]  [<ffffffff81019da9>] ? arch_trigger_all_cpu_backtrace+0x51/0x5d
<Oct/17 03:54 pm>[6853629.759079]  [<ffffffff812c08fe>] __delay+0xa/0xc
<Oct/17 03:54 pm>[6853629.759081]  [<ffffffff812c4e65>] do_raw_spin_lock+0xde/0x147
<Oct/17 03:54 pm>[6853629.759083]  [<ffffffff8151f193>] _raw_spin_lock+0x34/0x39
<Oct/17 03:54 pm>[6853629.759085]  [<ffffffff8102fb6b>] ? task_rq_lock+0x7e/0x9a
<Oct/17 03:54 pm>[6853629.759087]  [<ffffffff8102fb6b>] task_rq_lock+0x7e/0x9a
<Oct/17 03:54 pm>[6853629.759089]  [<ffffffff81057842>] ? hrtimer_wakeup+0x0/0x21
<Oct/17 03:54 pm>[6853629.759091]  [<ffffffff81036578>] try_to_wake_up+0x2e/0x342
<Oct/17 03:54 pm>[6853629.759093]  [<ffffffff81057842>] ? hrtimer_wakeup+0x0/0x21
<Oct/17 03:54 pm>[6853629.759095]  [<ffffffff810368b8>] wake_up_process+0x10/0x12
<Oct/17 03:54 pm>[6853629.759097]  [<ffffffff8105785f>] hrtimer_wakeup+0x1d/0x21
<Oct/17 03:54 pm>[6853629.759099]  [<ffffffff81057933>] __run_hrtimer+0xd0/0x12d
<Oct/17 03:54 pm>[6853629.759101]  [<ffffffff81057b81>] hrtimer_interrupt+0xb5/0x19f

<Oct/17 03:54 pm>[6853629.759103]  [<ffffffff810197a5>] smp_apic_timer_interrupt+0x82/0x95
<Oct/17 03:54 pm>[6853629.759106]  [<ffffffff81003353>] apic_timer_interrupt+0x13/0x20
<Oct/17 03:54 pm>[6853629.759107]  <EOI> 
<Oct/17 03:54 pm>[6853629.759108] Code: 89 e5 48 83 ec 08 65 8b 34 25 f0 d1 00 00 66 66 90 0f ae e8 0f 31 89 c1 41 89 f1 66 66 90 0f ae e8 0f 31 89 c6 48 89 f0 48 29 c8 <48> 39 f8 73 25 f3 90 65 44 8b 04 25 f0 d1 00 00 45 39 c1 74 db 
<Oct/17 03:54 pm>[6853629.759124] Call Trace:
<Oct/17 03:54 pm>[6853629.759125]  <IRQ>  [<ffffffff81019da9>] ? arch_trigger_all_cpu_backtrace+0x51/0x5d
<Oct/17 03:54 pm>[6853629.759128]  [<ffffffff812c08fe>] __delay+0xa/0xc
<Oct/17 03:54 pm>[6853629.759130]  [<ffffffff812c4e65>] do_raw_spin_lock+0xde/0x147
<Oct/17 03:54 pm>[6853629.759133]  [<ffffffff8151f193>] _raw_spin_lock+0x34/0x39
<Oct/17 03:54 pm>[6853629.759135]  [<ffffffff8102fb6b>] ? task_rq_lock+0x7e/0x9a
<Oct/17 03:54 pm>[6853629.759136]  [<ffffffff8102fb6b>] task_rq_lock+0x7e/0x9a
<Oct/17 03:54 pm>[6853629.759138]  [<ffffffff81057842>] ? hrtimer_wakeup+0x0/0x21
<Oct/17 03:54 pm>[6853629.759140]  [<ffffffff81036578>] try_to_wake_up+0x2e/0x342
<Oct/17 03:54 pm>[6853629.759142]  [<ffffffff81057842>] ? hrtimer_wakeup+0x0/0x21
<Oct/17 03:54 pm>[6853629.759144]  [<ffffffff810368b8>] wake_up_process+0x10/0x12
<Oct/17 03:54 pm>[6853629.759146]  [<ffffffff8105785f>] hrtimer_wakeup+0x1d/0x21
<Oct/17 03:54 pm>[6853629.759148]  [<ffffffff81057933>] __run_hrtimer+0xd0/0x12d
<Oct/17 03:54 pm>[6853629.759150]  [<ffffffff81057b81>] hrtimer_interrupt+0xb5/0x19f
<Oct/17 03:54 pm>[6853629.759153]  [<ffffffff810197a5>] smp_apic_timer_interrupt+0x82/0x95
<Oct/17 03:54 pm>[6853629.759155]  [<ffffffff81003353>] apic_timer_interrupt+0x13/0x20
<Oct/17 03:54 pm>[6853629.759156]  <EOI> Pid: 2921, comm: pclient Tainted: G      D    2.6.34.4 #3
<Oct/17 03:54 pm>[6853629.759159] Call Trace:

<Oct/17 03:54 pm>[6853629.759160]  <NMI>  [<ffffffff81009b2a>] ? show_regs+0x26/0x2a
<Oct/17 03:54 pm>[6853629.759163]  [<ffffffff81520db8>] nmi_watchdog_tick+0xc3/0x1a5
<Oct/17 03:54 pm>[6853629.759165]  [<ffffffff81520360>] do_nmi+0xb2/0x259
<Oct/17 03:54 pm>[6853629.759167]  [<ffffffff8151fdfa>] nmi+0x1a/0x20
<Oct/17 03:54 pm>[6853629.759169]  [<ffffffff812c08b7>] ? delay_tsc+0x2d/0x59
<Oct/17 03:54 pm>[6853629.759170]  <<EOE>>  <IRQ>  [<ffffffff81019da9>] ? arch_trigger_all_cpu_backtrace+0x51/0x5d
<Oct/17 03:54 pm>[6853629.759174]  [<ffffffff812c08fe>] __delay+0xa/0xc
<Oct/17 03:54 pm>[6853629.759176]  [<ffffffff812c4e65>] do_raw_spin_lock+0xde/0x147
<Oct/17 03:54 pm>[6853629.759179]  [<ffffffff8151f193>] _raw_spin_lock+0x34/0x39
<Oct/17 03:54 pm>[6853629.759181]  [<ffffffff8102fb6b>] ? task_rq_lock+0x7e/0x9a
<Oct/17 03:54 pm>[6853629.759182]  [<ffffffff8102fb6b>] task_rq_lock+0x7e/0x9a
<Oct/17 03:54 pm>[6853629.759184]  [<ffffffff81057842>] ? hrtimer_wakeup+0x0/0x21
<Oct/17 03:54 pm>[6853629.759186]  [<ffffffff81036578>] try_to_wake_up+0x2e/0x34<Oct/17 03:54 pm>
                 <Oct/17 03:54 pm>[6853629.759188]  [<ffffffff81057842>] ? hrtimer_wakeup+0x0/0x21
<Oct/17 03:54 pm>[6853629.759191]  [<ffffffff810368b8>] wake_up_process+0x10/0x12
<Oct/17 03:54 pm>[6853629.759192]  [<ffffffff8105785f>] hrtimer_wakeup+0x1d/0x21
<Oct/17 03:54 pm>[6853629.759194]  [<ffffffff81057933>] __run_hrtimer+0xd0/0x12d
<Oct/17 03:54 pm>[6853629.759196]  [<ffffffff81057b81>] hrtimer_interrupt+0xb5/0x19f
<Oct/17 03:54 pm>[6853629.759199]  [<ffffffff810197a5>] smp_apic_timer_interrupt+0x82/0x95
<Oct/17 03:54 pm>[6853629.759201]  [<ffffffff81003353>] apic_timer_interrupt+0x13/0x20
<Oct/17 03:54 pm>[6853629.759202]  <EOI> 
<Oct/17 03:54 pm>[6853629.759203] NMI backtrace for cpu 10

<Oct/17 03:54 pm>[6853629.759204] CPU 10 
<Oct/17 03:54 pm>[6853629.759205] Modules linked in: w83793 hwmon_vid autofs4 i2c_i801 i2c_core iTCO_wdt e1000e ioatdma dca rtc_cmos tg3 libphy e1000 fuse dm_snapshot dm_mirror dm_region_hash dm_log scsi_wait_scan qla2xxx megaraid_sas megaraid_mbox megaraid_mm aacraid 3w_9xxx 3w_xxxx arcmsr aic7xxx aic79xx sr_mod sg ata_piix ahci sata_sis sata_nv sata_via pata_sl82c105 pata_sis
<Oct/17 03:54 pm>[6853629.759218] 
<Oct/17 03:54 pm>[6853629.759220] Pid: 19726, comm: ppImage Tainted: G      D    2.6.34.4 #3 S7010/empty
<Oct/17 03:54 pm>[6853629.759221] RIP: 0010:[<ffffffff812c08a2>]  [<ffffffff812c08a2>] delay_tsc+0x18/0x59
<Oct/17 03:54 pm>[6853629.759224] RSP: 0000:ffff88002c88b7a0  EFLAGS: 00000086
<Oct/17 03:54 pm>[6853629.759225] RAX: 000000008154e97c RBX: 00000000042ec12b RCX: 000000008154e938
<Oct/17 03:54 pm>[6853629.759227] RDX: 000000000040ee78 RSI: 000000000000000a RDI: 0000000000000001
<Oct/17 03:54 pm>[6853629.759228] RBP: ffff88002c88b7a8 R08: 000000000000000a R09: 000000000000000a
<Oct/17 03:54 pm>[6853629.759230] R10: 0000000000000000 R11: ffff88002c88b5d8 R12: ffff8806552b3c00
<Oct/17 03:54 pm>[6853629.759231] R13: 0000000000000000 R14: ffff880611553090 R15: ffff880611553518
<Oct/17 03:54 pm>[6853629.759233] FS:  000000004243e950(0063) GS:ffff880655280000(0000) knlGS:0000000000000000
<Oct/17 03:54 pm>[6853629.759235] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<Oct/17 03:54 pm>[6853629.759236] CR2: 00007fa3e03daab0 CR3: 00000003e2c88000 CR4: 00000000000006e0
<Oct/17 03:54 pm>[6853629.759238] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
<Oct/17 03:54 pm>[6853629.759239] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
<Oct/17 03:54 pm>[6853629.759241] Process ppImage (pid: 19726, threadinfo ffff88002c88a000, task ffff880611553090)
<Oct/17 03:54 pm>[6853629.759242] Stack:
<Oct/17 03:54 pm>[6853629.759243]  ffffffff81019da9 ffff88002c88b7b8 ffffffff812c08fe ffff88002c88b7f8
<Oct/17 03:54 pm>[6853629.759245] <0> ffffffff812c4e65 000000009ef2e6b4 ffff8806552b3c00 ffff8806552b3c00
<Oct/17 03:54 pm>[6853629.759248] <0> ffff88065528e3f0 0000000000000400 0000000000000800 ffff88002c88b818
<Oct/17 03:54 pm>[6853629.759250] Call Trace:

<Oct/17 03:54 pm>[6853629.759252]  [<ffffffff81019da9>] ? arch_trigger_all_cpu_backtrace+0x51/0x5d
<Oct/17 03:54 pm>[6853629.759254]  [<ffffffff812c08fe>] __delay+0xa/0xc
<Oct/17 03:54 pm>[6853629.759256]  [<ffffffff812c4e65>] do_raw_spin_lock+0xde/0x147
<Oct/17 03:54 pm>[6853629.759259]  [<ffffffff8151f0dc>] _raw_spin_lock_nested+0x32/0x36
<Oct/17 03:54 pm>[6853629.759261]  [<ffffffff81035934>] ? double_rq_lock+0x4a/0x76
<Oct/17 03:54 pm>[6853629.759263]  [<ffffffff81035934>] double_rq_lock+0x4a/0x76
<Oct/17 03:54 pm>[6853629.759265]  [<ffffffff81037672>] load_balance+0xbb6/0xf63
<Oct/17 03:54 pm>[6853629.759267]  [<ffffffff810b3cfb>] ? get_dirty_limits+0x22/0x258
<Oct/17 03:54 pm>[6853629.759270]  [<ffffffff8151cc0e>] schedule+0x25a/0x86a
<Oct/17 03:54 pm>[6853629.759272]  [<ffffffff810480a2>] ? lock_timer_base+0x26/0x4a
<Oct/17 03:54 pm>[6853629.759275]  [<ffffffff8151d559>] schedule_timeout+0x222/0x24c
<Oct/17 03:54 pm>[6853629.759277]  [<ffffffff810482a5>] ? process_timeout+0x0/0xb
<Oct/17 03:54 pm>[6853629.759279]  [<ffffffff8151c6c5>] io_schedule_timeout+0x45/0x5e
<Oct/17 03:54 pm>[6853629.759281]  [<ffffffff810c0abe>] congestion_wait+0x6b/0x87
<Oct/17 03:54 pm>[6853629.759284]  [<ffffffff81054b6c>] ? autoremove_wake_function+0x0/0x38
<Oct/17 03:54 pm>[6853629.759286]  [<ffffffff810b3279>] __alloc_pages_nodemask+0x65a/0x671
<Oct/17 03:54 pm>[6853629.759289]  [<ffffffff810da90f>] alloc_page_vma+0x148/0x14d
<Oct/17 03:54 pm>[6853629.759291]  [<ffffffff810c4d2a>] handle_mm_fault+0x294/0x73e
<Oct/17 03:54 pm>[6853629.759294]  [<ffffffff815227e9>] do_page_fault+0x351/0x375
<Oct/17 03:54 pm>[6853629.759296]  [<ffffffff8151e35f>] ? do_nanosleep+0x7b/0xb3
<Oct/17 03:54 pm>[6853629.759298]  [<ffffffff810348d8>] ? finish_task_switch+0x69/0xdd
<Oct/17 03:54 pm>[6853629.759300]  [<ffffffff8103486f>] ? finish_task_switch+0x0/0xdd

<Oct/17 03:54 pm>[6853629.759302]  [<ffffffff81058477>] ? hrtimer_nanosleep+0x97/0x105
<Oct/17 03:54 pm>[6853629.759304]  [<ffffffff8106472c>] ? lockdep_sys_exit+0x20/0x75
<Oct/17 03:54 pm>[6853629.759307]  [<ffffffff8151ef6f>] ? lockdep_sys_exit_thunk+0x35/0x67
<Oct/17 03:54 pm>[6853629.759309]  [<ffffffff810c4dda>] ? handle_mm_fault+0x344/0x73e
<Oct/17 03:54 pm>[6853629.759311]  [<ffffffff8151fb5f>] page_fault+0x1f/0x30
                 <Oct/17 03:54 pm>[6853629.759312] Code: 2e 0f 1f 84 00 00 00 00 00 48 ff c8 75 fb 48 ff c8 c9 c3 55 48 89 e5 48 83 ec 08 65 8b 34 25 f0 d1 00 00 66 66 90 0f ae e8 0f 31 <89> c1 41 89 f1 66 66 90 0f ae e8 0f 31 89 c6 48 89 f0 48 29 c8 
<Oct/17 03:54 pm>[6853629.759327] Call Trace:
<Oct/17 03:54 pm>[6853629.759329]  [<ffffffff81019da9>] ? arch_trigger_all_cpu_backtrace+0x51/0x5d
<Oct/17 03:54 pm>[6853629.759332]  [<ffffffff812c08fe>] __delay+0xa/0xc
<Oct/17 03:54 pm>[6853629.759333]  [<ffffffff812c4e65>] do_raw_spin_lock+0xde/0x147
<Oct/17 03:54 pm>[6853629.759336]  [<ffffffff8151f0dc>] _raw_spin_lock_nested+0x32/0x36
<Oct/17 03:54 pm>[6853629.759338]  [<ffffffff81035934>] ? double_rq_lock+0x4a/0x76
<Oct/17 03:54 pm>[6853629.759340]  [<ffffffff81035934>] double_rq_lock+0x4a/0x76
<Oct/17 03:54 pm>[6853629.759342]  [<ffffffff81037672>] load_balance+0xbb6/0xf63
<Oct/17 03:54 pm>[6853629.759344]  [<ffffffff810b3cfb>] ? get_dirty_limits+0x22/0x258
<Oct/17 03:54 pm>[6853629.759347]  [<ffffffff8151cc0e>] schedule+0x25a/0x86a
<Oct/17 03:54 pm>[6853629.759349]  [<ffffffff810480a2>] ? lock_timer_base+0x26/0x4a
<Oct/17 03:54 pm>[6853629.759352]  [<ffffffff8151d559>] schedule_timeout+0x222/0x24c
<Oct/17 03:54 pm>[6853629.759354]  [<ffffffff810482a5>] ? process_timeout+0x0/0xb
<Oct/17 03:54 pm>[6853629.759356]  [<ffffffff8151c6c5>] io_schedule_timeout+0x45/0x5e
<Oct/17 03:54 pm>[6853629.759358]  [<ffffffff810c0abe>] congestion_wait+0x6b/0x87
<Oct/17 03:54 pm>[6853629.759361]  [<ffffffff81054b6c>] ? autoremove_wake_function+0x0/0x38

<Oct/17 03:54 pm>[6853629.759363]  [<ffffffff810b3279>] __alloc_pages_nodemask+0x65a/0x671
<Oct/17 03:54 pm>[6853629.759366]  [<ffffffff810da90f>] alloc_page_vma+0x148/0x14d
<Oct/17 03:54 pm>[6853629.759368]  [<ffffffff810c4d2a>] handle_mm_fault+0x294/0x73e
<Oct/17 03:54 pm>[6853629.759370]  [<ffffffff815227e9>] do_page_fault+0x351/0x375
<Oct/17 03:54 pm>[6853629.759372]  [<ffffffff8151e35f>] ? do_nanosleep+0x7b/0xb3
<Oct/17 03:54 pm>[6853629.759375]  [<ffffffff810348d8>] ? finish_task_switch+0x69/0xdd
<Oct/17 03:54 pm>[6853629.759377]  [<ffffffff8103486f>] ? finish_task_switch+0x0/0xdd
<Oct/17 03:54 pm>[6853629.759379]  [<ffffffff81058477>] ? hrtimer_nanosleep+0x97<Oct/17 03:54 pm>
                 <Oct/17 03:54 pm>[6853629.759381]  [<ffffffff8106472c>] ? lockdep_sys_exit+0x20/0x75
<Oct/17 03:54 pm>[6853629.759383]  [<ffffffff8151ef6f>] ? lockdep_sys_exit_thunk+0x35/0x67
<Oct/17 03:54 pm>[6853629.759385]  [<ffffffff810c4dda>] ? handle_mm_fault+0x344/0x73e
<Oct/17 03:54 pm>[6853629.759387]  [<ffffffff8151fb5f>] page_fault+0x1f/0x30
<Oct/17 03:54 pm>[6853629.759389] Pid: 19726, comm: ppImage Tainted: G      D    2.6.34.4 #3
<Oct/17 03:54 pm>[6853629.759390] Call Trace:
<Oct/17 03:54 pm>[6853629.759391]  <NMI>  [<ffffffff81009b2a>] ? show_regs+0x26/0x2a
<Oct/17 03:54 pm>[6853629.759395]  [<ffffffff81520db8>] nmi_watchdog_tick+0xc3/0x1a5
<Oct/17 03:54 pm>[6853629.759397]  [<ffffffff81520360>] do_nmi+0xb2/0x259
<Oct/17 03:54 pm>[6853629.759398]  [<ffffffff8151fdfa>] nmi+0x1a/0x20
<Oct/17 03:54 pm>[6853629.759400]  [<ffffffff812c08a2>] ? delay_tsc+0x18/0x59
<Oct/17 03:54 pm>[6853629.759402]  <<EOE>>  [<ffffffff81019da9>] ? arch_trigger_all_cpu_backtrace+0x51/0x5d
<Oct/17 03:54 pm>[6853629.759405]  [<ffffffff812c08fe>] __delay+0xa/0xc
<Oct/17 03:54 pm>[6853629.759407]  [<ffffffff812c4e65>] do_raw_spin_lock+0xde/0x147

<Oct/17 03:54 pm>[6853629.759410]  [<ffffffff8151f0dc>] _raw_spin_lock_nested+0x32/0x36
<Oct/17 03:54 pm>[6853629.759412]  [<ffffffff81035934>] ? double_rq_lock+0x4a/0x76
<Oct/17 03:54 pm>[6853629.759414]  [<ffffffff81035934>] double_rq_lock+0x4a/0x76
<Oct/17 03:54 pm>[6853629.759416]  [<ffffffff81037672>] load_balance+0xbb6/0xf63
<Oct/17 03:54 pm>[6853629.759418]  [<ffffffff810b3cfb>] ? get_dirty_limits+0x22/0x258
<Oct/17 03:54 pm>[6853629.759421]  [<ffffffff8151cc0e>] schedule+0x25a/0x86a
<Oct/17 03:54 pm>[6853629.759423]  [<ffffffff810480a2>] ? lock_timer_base+0x26/0x4a
<Oct/17 03:54 pm>[6853629.759425]  [<ffffffff8151d559>] schedule_timeout+0x222/0x24c
<Oct/17 03:54 pm>[6853629.759428]  [<ffffffff810482a5>] ? process_timeout+0x0/0xb
<Oct/17 03:54 pm>[6853629.759430]  [<ffffffff8151c6c5>] io_schedule_timeout+0x45/0x5e
<Oct/17 03:54 pm>[6853629.759432]  [<ffffffff810c0abe>] congestion_wait+0x6b/0x87
<Oct/17 03:54 pm>[6853629.759435]  [<ffffffff81054b6c>] ? autoremove_wake_function+0x0/0x38
<Oct/17 03:54 pm>[6853629.759437]  [<ffffffff810b3279>] __alloc_pages_nodemask+0x65a/0x671
<Oct/17 03:54 pm>[6853629.759440]  [<ffffffff810da90f>] alloc_page_vma+0x148/0x14d
<Oct/17 03:54 pm>[6853629.759442]  [<ffffffff810c4d2a>] handle_mm_fault+0x294/0x73e
<Oct/17 03:54 pm>[6853629.759444]  [<ffffffff815227e9>] do_page_fault+0x351/0x375
<Oct/17 03:54 pm>[6853629.759446]  [<ffffffff8151e35f>] ? do_nanosleep+0x7b/0xb3
<Oct/17 03:54 pm>[6853629.759448]  [<ffffffff810348d8>] ? finish_task_switch+0x69/0xdd
<Oct/17 03:54 pm>[6853629.759451]  [<ffffffff8103486f>] ? finish_task_switch+0x0/0xdd
<Oct/17 03:54 pm>[6853629.759453]  [<ffffffff81058477>] ? hrtimer_nanosleep+0x97/0x105
<Oct/17 03:54 pm>[6853629.759455]  [<ffffffff8106472c>] ? lockdep_sys_exit+0x20/0x75
<Oct/17 03:54 pm>[6853629.759457]  [<ffffffff8151ef6f>] ? lockdep_sys_exit_thunk+0x35/0x67

<Oct/17 03:54 pm>[6853629.759459]  [<ffffffff810c4dda>] ? handle_mm_fault+0x344/0x73e
<Oct/17 03:54 pm>[6853629.759461]  [<ffffffff8151fb5f>] page_fault+0x1f/0x30
<Oct/17 03:54 pm>[6853659.183500] Pid: 5270, comm: sshd Tainted: G      D    2.6.34.4 #3
<Oct/17 03:54 pm>[6853659.190210] Call Trace:
<Oct/17 03:54 pm>[6853659.193112]  [<ffffffff812c4e9f>] do_raw_spin_lock+0x118/0x147
<Oct/17 03:54 pm>[6853659.199477]  [<ffffffff8151f193>] _raw_spin_lock+0x34/0x39
<Oct/17 03:54 pm>[6853659.205508]  [<ffffffff8102fb6b>] ? task_rq_lock+0x7e/0x9a
<Oct/17 03:54 pm>[6853659.211511]  [<ffffffff8102fb6b>] task_rq_lock+0x7e/0x9a
<Oct/17 03:54 pm>[6853659.217311]  [<ffffffff81036578>] try_to_wake_up+0x2e/0x342
<Oct/17 03:54 pm>[6853659.223423]  [<ffffffff81036899>] default_wake_function+0xd/0xf
<Oct/17 03:54 pm>[6853659.229879]  [<ffffffff810f9aba>] pollwake+0x57/0x59
<Oct/17 03:54 pm>[6853659.235372]  [<ffffffff8103688c>] ? default_wake_function+0x0/0xf
<Oct/17 03:54 pm>[6853659.241972]  [<ffffffff8102c08d>] __wake_up_common+0x46/0x<Oct/17 03:54 pm>
                 <Oct/17 03:54 pm>[6853659.248166]  [<ffffffff8102fa4a>] __wake_up_sync_key+0x43/0x5a
<Oct/17 03:54 pm>[6853659.254519]  [<ffffffff8102fa6c>] __wake_up_sync+0xb/0xd
<Oct/17 03:54 pm>[6853659.260342]  [<ffffffff810f1b84>] pipe_write+0x4de/0x514
<Oct/17 03:54 pm>[6853659.266175]  [<ffffffff810f94de>] ? core_sys_select+0x3d/0x295
<Oct/17 03:54 pm>[6853659.272505]  [<ffffffff810ea282>] do_sync_write+0xc7/0x10d
<Oct/17 03:54 pm>[6853659.278483]  [<ffffffff8104a535>] ? sigprocmask+0x30/0xc8
<Oct/17 03:54 pm>[6853659.284420]  [<ffffffff812937db>] ? security_file_permission+0x11/0x13
<Oct/17 03:54 pm>[6853659.291495]  [<ffffffff810eaba1>] vfs_write+0xae/0x14a
<Oct/17 03:54 pm>[6853659.297151]  [<ffffffff810ead01>] sys_write+0x47/0x6e

<Oct/17 03:54 pm>[6853659.302706]  [<ffffffff810029ab>] system_call_fastpath+0x16/0x1b
<Oct/17 03:54 pm>[6853659.309261] sending NMI to all CPUs:
<Oct/17 03:54 pm>[6853659.313300] NMI backtrace for cpu 1
<Oct/17 03:54 pm>[6853659.317269] CPU 1 
<Oct/17 03:54 pm>[6853659.319352] Modules linked in: w83793 hwmon_vid autofs4 i2c_i801 i2c_core iTCO_wdt e1000e ioatdma dca rtc_cmos tg3 libphy e1000 fuse dm_snapshot dm_mirror dm_region_hash dm_log scsi_wait_scan qla2xxx megaraid_sas megaraid_mbox megaraid_mm aacraid 3w_9xxx 3w_xxxx arcmsr aic7xxx aic79xx sr_mod sg ata_piix ahci sata_sis sata_nv sata_via pata_sl82c105 pata_sis
<Oct/17 03:54 pm>[6853659.355218] 
<Oct/17 03:54 pm>[6853659.357132] Pid: 8438, comm: nfsd Tainted: G      D    2.6.34.4 #3 S7010/empty
<Oct/17 03:54 pm>[6853659.365099] RIP: 0010:[<ffffffff812c08a2>]  [<ffffffff812c08a2>] delay_tsc+0x18/0x59
<Oct/17 03:54 pm>[6853659.373623] RSP: 0018:ffff880621efb9a8  EFLAGS: 00000092
<Oct/17 03:54 pm>[6853659.379434] RAX: 00000000daed9cc8 RBX: 00000000474a1b11 RCX: 00000000daed9c84
<Oct/17 03:54 pm>[6853659.387320] RDX: 000000000040ee8a RSI: 0000000000000001 RDI: 0000000000000001
<Oct/17 03:54 pm>[6853659.395179] RBP: ffff880621efb9b0 R08: 0000000000000001 R09: 0000000000000001
<Oct/17 03:54 pm>[6853659.403074] R10: 0000000000000000 R11: ffff880621efb7e0 R12: ffff880655253c00
<Oct/17 03:54 pm>[6853659.410919] R13: 0000000000000000 R14: ffff880621ef2060 R15: ffff880621ef24e8
<Oct/17 03:54 pm>[6853659.418789] FS:  0000000000000000(0000) GS:ffff880002620000(0000) knlGS:0000000000000000
<Oct/17 03:54 pm>[6853659.427625] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
<Oct/17 03:54 pm>[6853659.433907] CR2: 00007fa3dcdf1330 CR3: 0000000ad0e7b000 CR4: 00000000000006e0
<Oct/17 03:54 pm>[6853659.441783] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
<Oct/17 03:54 pm>[6853659.449669] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
<Oct/17 03:54 pm>[6853659.457522] Process nfsd (pid: 8438, threadinfo ffff880621efa000, task ffff880621ef2060)
<Oct/17 03:54 pm>[6853659.466374] Stack:
<Oct/17 03:54 pm>[6853659.468814]  ffffffff81019da9 ffff880621efb9c0 ffffffff812c08fe ffff880621efba00

<Oct/17 03:54 pm>[6853659.476760] <0> ffffffff812c4e65 000000009ef2e6b4 ffff880655253c00 ffff880621ed8000
<Oct/17 03:54 pm>[6853659.485435] <0> 0000000000013c00 ffff880621efba70 0000000000000001 ffff880621efba20
<Oct/17 03:54 pm>[6853659.494514] Call Trace:
<Oct/17 03:54 pm>[6853659.497391]  [<ffffffff81019da9>] ? arch_trigger_all_cpu_backtrace+0x51/0x5d
<Oct/17 03:54 pm>[6853659.505138]  [<ffffffff812c08fe>] __delay+0xa/0xc
<Oct/17 03:54 pm>[6853659.510364]  [<ffffffff812c4e65>] do_raw_spin_lock+0xde/0x147
<Oct/17 03:54 pm>[6853659.516619]  [<ffffffff8151f193>] _raw_spin_lock+0x34/0x39
<Oct/17 03:54 pm>[6853659.522633]  [<ffffffff8102fb6b>] ? task_rq_lock+0x7e/0x9a
<Oct/17 03:54 pm>[6853659.528629]  [<ffffffff8102fb6b>] task_rq_lock+0x7e/0x9a
<Oct/17 03:54 pm>[6853659.534462]  [<ffffffff81036578>] try_to_wake_up+0x2e/0x342
<Oct/17 03:54 pm>[6853659.540530]  [<ffffffff81036899>] default_wake_function+0xd/0xf
<Oct/17 03:54 pm>[6853659.546997]  [<ffffffff81054b7d>] autoremove_wake_function+0x11/0x38
<Oct/17 03:54 pm>[6853659.553912]  [<ffffffff8102c08d>] __wake_up_common+0x46/0x76
<Oct/17 03:54 pm>[6853659.560074]  [<ffffffff8102faa6>] __wake_up+0x38/0x4f
<Oct/17 03:54 pm>[6853659.565643]  [<ffffffff8144b2f8>] release_sock+0xd3/0xe4
<Oct/17 03:54 pm>[6853659.571470]  [<ffffffff8149c3f8>] udp_sendpage+0x124/0x13c
<Oct/17 03:54 pm>[6853659.577511]  [<ffffffff81261435>] ? xfs_iunlock+0x5d/0xb5
<Oct/17 03:54 pm>[6853659.583430]  [<ffffffff814a1b41>] inet_sendpage+0x61/0x8e
<Oct/17 03:54 pm>[6853659.589375]  [<ffffffff81446e42>] kernel_sendpage+0x16/0x1f
<Oct/17 03:54 pm>[6853659.595487]  [<ffffffff814f33b3>] svc_send_common+0xb9/0x11d
<Oct/17 03:54 pm>[6853659.601669]  [<ffffffff814f353c>] svc_sendto+0x125/0x17e
                 <Oct/17 03:54 pm>[6853659.607507]  [<ffffffff814fc1aa>] ? svc_send+0x56/0xa1

<Oct/17 03:54 pm>[6853659.613140]  [<ffffffff8151e1c3>] ? mutex_lock_nested+0x24b/0x25a
<Oct/17 03:54 pm>[6853659.619749]  [<ffffffff814f35ae>] svc_udp_sendto+0x19/0x2e
<Oct/17 03:54 pm>[6853659.625734]  [<ffffffff814fc1bb>] svc_send+0x67/0xa1
<Oct/17 03:54 pm>[6853659.631200]  [<ffffffff814f07bb>] svc_process+0x71b/0x72a
<Oct/17 03:54 pm>[6853659.637114]  [<ffffffff811f0ccf>] ? nfsd+0x0/0x139
<Oct/17 03:54 pm>[6853659.642406]  [<ffffffff811f0dc2>] nfsd+0xf3/0x139
<Oct/17 03:54 pm>[6853659.652141]  [<ffffffff810546e2>] kthread+0x7d/0x85
<Oct/17 03:54 pm>[6853659.657530]  [<ffffffff81003794>] kernel_thread_helper+0x4/0x10
<Oct/17 03:54 pm>[6853659.663958]  [<ffffffff81054665>] ? kthread+0x0/0x85
<Oct/17 03:54 pm>[6853659.669414]  [<ffffffff81003790>] ? kernel_thread_helper+0x0/0x10
<Oct/17 03:54 pm>[6853659.676064] Code: 2e 0f 1f 84 00 00 00 00 00 48 ff c8 75 fb 48 ff c8 c9 c3 55 48 89 e5 48 83 ec 08 65 8b 34 25 f0 d1 00 00 66 66 90 0f ae e8 0f 31 <89> c1 41 89 f1 66 66 90 0f ae e8 0f 31 89 c6 48 89 f0 48 29 c8 
<Oct/17 03:54 pm>[6853659.699375] Call Trace:
<Oct/17 03:54 pm>[6853659.702269]  [<ffffffff81019da9>] ? arch_trigger_all_cpu_backtrace+0x51/0x5d
<Oct/17 03:54 pm>[6853659.710044]  [<ffffffff812c08fe>] __delay+0xa/0xc
<Oct/17 03:54 pm>[6853659.715211]  [<ffffffff812c4e65>] do_raw_spin_lock+0xde/0x147
<Oct/17 03:54 pm>[6853659.721480]  [<ffffffff8151f193>] _raw_spin_lock+0x34/0x39
<Oct/17 03:54 pm>[6853659.727492]  [<ffffffff8102fb6b>] ? task_rq_lock+0x7e/0x9a
<Oct/17 03:54 pm>[6853659.733525]  [<ffffffff8102fb6b>] task_rq_lock+0x7e/0x9a
<Oct/17 03:54 pm>[6853659.739357]  [<ffffffff81036578>] try_to_wake_up+0x2e/0x342
<Oct/17 03:54 pm>[6853659.745478]  [<ffffffff81036899>] default_wake_function+0xd/0xf
<Oct/17 03:54 pm>[6853659.751951]  [<ffffffff81054b7d>] autoremove_wake_function+0x11/0x38
<Oct/17 03:54 pm>[6853659.758862]  [<ffffffff8102c08d>] __wake_up_common+0x46/0x76

<Oct/17 03:54 pm>[6853659.765029]  [<ffffffff8102faa6>] __wake_up+0x38/0x4f
<Oct/17 03:54 pm>[6853659.770601]  [<ffffffff8144b2f8>] release_sock+0xd3/0xe4
<Oct/17 03:54 pm>[6853659.776400]  [<ffffffff8149c3f8>] udp_sendpage+0x124/0x13c
<Oct/17 03:54 pm>[6853659.782432]  [<ffffffff81261435>] ? xfs_iunlock+0x5d/0xb5
<Oct/17 03:54 pm>[6853659.788325]  [<ffffffff814a1b41>] inet_sendpage+0x61/0x8e
<Oct/17 03:54 pm>[6853659.794219]  [<ffffffff81446e42>] kernel_sendpage+0x16/0x1f
<Oct/17 03:54 pm>[6853659.800304]  [<ffffffff814f33b3>] svc_send_common+0xb9/0x11d
<Oct/17 03:54 pm>[6853659.806494]  [<ffffffff814f353c>] svc_sendto+0x125/0x17e
<Oct/17 03:54 pm>[6853659.812316]  [<ffffffff814fc1aa>] ? svc_send+0x56/0xa1
<Oct/17 03:54 pm>[6853659.817993]  [<ffffffff8151e1c3>] ? mutex_lock_nested+0x24b/0x25a
<Oct/17 03:54 pm>[6853659.824625]  [<ffffffff814f35ae>] svc_udp_sendto+0x19/0x2e
<Oct/17 03:54 pm>[6853659.830622]  [<ffffffff814fc1bb>] svc_send+0x67/0xa1
<Oct/17 03:54 pm>[6853659.836078]  [<ffffffff814f07bb>] svc_process+0x71b/0x72a
<Oct/17 03:54 pm>[6853659.841974]  [<ffffffff811f0ccf>] ? nfsd+0x0/0x139<Oct/17 03:54 pm>[6853659.847249]  [<ffffffff811f0dc2>] nfsd+0xf3/0x139
<Oct/17 03:54 pm>[6853659.852475]  [<ffffffff810546e2>] kthread+0x7d/0x85
<Oct/17 03:54 pm>[6853659.857858]  [<ffffffff81003794>] kernel_thread_helper+0x4/0x10
<Oct/17 03:54 pm>[6853659.864318]  [<ffffffff81054665>] ? kthread+0x0/0x85
<Oct/17 03:54 pm>[6853659.869820]  [<ffffffff81003790>] ? kernel_thread_helper+0x0/0x10
<Oct/17 03:54 pm>[6853659.876454] Pid: 8438, comm: nfsd Tainted: G      D    2.6.34.4 #3
<Oct/17 03:54 pm>[6853659.883151] Call Trace:
<Oct/17 03:54 pm>[6853659.886062]  <NMI>  [<ffffffff81009b2a>] ? show_regs+0x26/0x2a

<Oct/17 03:54 pm>[6853659.892455]  [<ffffffff81520db8>] nmi_watchdog_tick+0xc3/0x1a5
<Oct/17 03:54 pm>[6853659.898814]  [<ffffffff81520360>] do_nmi+0xb2/0x259
<Oct/17 03:54 pm>[6853659.904192]  [<ffffffff8151fdfa>] nmi+0x1a/0x20
<Oct/17 03:54 pm>[6853659.909219]  [<ffffffff812c08a2>] ? delay_tsc+0x18/0x59
<Oct/17 03:54 pm>[6853659.914967]  <<EOE>>  [<ffffffff81019da9>] ? arch_trigger_all_cpu_backtrace+0x51/0x5d
<Oct/17 03:54 pm>[6853659.923603]  [<ffffffff812c08fe>] __delay+0xa/0xc
<Oct/17 03:54 pm>[6853659.928841]  [<ffffffff812c4e65>] do_raw_spin_lock+0xde/0x147
<Oct/17 03:54 pm>[6853659.935120]  [<ffffffff8151f193>] _raw_spin_lock+0x34/0x39
<Oct/17 03:54 pm>[6853659.941161]  [<ffffffff8102fb6b>] ? task_rq_lock+0x7e/0x9a
<Oct/17 03:54 pm>[6853659.947175]  [<ffffffff8102fb6b>] task_rq_lock+0x7e/0x9a
<Oct/17 03:54 pm>[6853659.953024]  [<ffffffff81036578>] try_to_wake_up+0x2e/0x342
<Oct/17 03:54 pm>[6853659.959126]  [<ffffffff81036899>] default_wake_function+0xd/0xf
<Oct/17 03:54 pm>[6853659.965584]  [<ffffffff81054b7d>] autoremove_wake_function+0x11/0x38
<Oct/17 03:54 pm>[6853659.972493]  [<ffffffff8102c08d>] __wake_up_common+0x46/0x76
<Oct/17 03:54 pm>[6853659.978698]  [<ffffffff8102faa6>] __wake_up+0x38/0x4f
<Oct/17 03:54 pm>[6853659.984268]  [<ffffffff8144b2f8>] release_sock+0xd3/0xe4
<Oct/17 03:54 pm>[6853659.990102]  [<ffffffff8149c3f8>] udp_sendpage+0x124/0x13c
<Oct/17 03:54 pm>[6853659.996092]  [<ffffffff81261435>] ? xfs_iunlock+0x5d/0xb5
<Oct/17 03:54 pm>[6853660.002002]  [<ffffffff814a1b41>] inet_sendpage+0x61/0x8e
<Oct/17 03:54 pm>[6853660.007930]  [<ffffffff81446e42>] kernel_sendpage+0x16/0x1f
<Oct/17 03:54 pm>[6853660.014058]  [<ffffffff814f33b3>] svc_send_common+0xb9/0x11d
<Oct/17 03:54 pm>[6853660.020239]  [<ffffffff814f353c>] svc_sendto+0x125/0x17e

<Oct/17 03:54 pm>[6853660.026069]  [<ffffffff814fc1aa>] ? svc_send+0x56/0xa1
<Oct/17 03:54 pm>[6853660.031713]  [<ffffffff8151e1c3>] ? mutex_lock_nested+0x24b/0x25a
<Oct/17 03:54 pm>[6853660.038362]  [<ffffffff814f35ae>] svc_udp_sendto+0x19/0x2e
<Oct/17 03:54 pm>[6853660.044376]  [<ffffffff814fc1bb>] svc_send+0x67/0xa1
<Oct/17 03:54 pm>[6853660.049877]  [<ffffffff814f07bb>] svc_process+0x71b/0x72a
<Oct/17 03:54 pm>[6853660.055798]  [<ffffffff811f0ccf>] ? nfsd+0x0/0x139
<Oct/17 03:54 pm>[6853660.061091]  [<ffffffff811f0dc2>] nfsd+0xf3/0x139
<Oct/17 03:54 pm>[6853660.066275]  [<ffffffff810546e2>] kthread+0x7d/0x85
<Oct/17 03:54 pm>[6853660.071673]  [<ffffffff81003794>] kernel_thread_helper+0x4/0x10
<Oct/17 03:54 pm>[6853660.078144]  [<ffffffff81054665>] ? kthread+0x0/0x85
<Oct/17 03:54 pm>[6853660.083619]  [<ffffffff81003790>] ? kernel_thread_helper+0x0/0x10
<Oct/17 03:54 pm>[6853660.090260] NMI backtrace for cpu 5
<Oct/17 03:54 pm>[6853660.094220] CPU 5 
<Oct/17 03:54 pm>[6853660.096319] Modules linked in: w83793 hwmon_vid autofs4 i2c_i801 i2c_core iTCO_wdt e1000e ioatdma dca rtc_cmos tg3 libphy e1000 fuse dm_snapshot dm_mirror dm_region_hash dm_log scsi_wait_scan qla2xxx megaraid_sas megaraid_mbox megaraid_mm aacraid 3w_9xxx 3w_xxxx arcmsr aic7xxx aic79xx sr_mod sg ata_piix ahci sata_sis sata_nv sata_via pata_sl82c105 pata_sis
<Oct/17 03:54 pm>[6853660.132159] 
<Oct/17 03:54 pm>[6853660.134062] Pid: 5270, comm: sshd Tainted: G      D    2.6.34.4 #3 S7010/empty
<Oct/17 03:54 pm>[6853660.142015] RIP: 0010:[<ffffffff812c08af>]  [<ffffffff812c08af>] delay_tsc+0x25/0x59
<Oct/17 03:54 pm>[6853660.150563] RSP: 0018:ffff88037e3e1ad0  EFLAGS: 00000046
<Oct/17 03:54 pm>[6853660.156376] RAX: 00000000daf1afa4 RBX: 0000000000000000 RCX: 00000000daf1af58
<Oct/17 03:54 pm>[6853660.164261] RDX: 000000000040ee8a RSI: 00000000daf1af78 RDI: 00000000000411b2
<Oct/17 03:54 pm>[6853660.172123] RBP: ffff88037e3e1ad8 R08: 0000000000000005 R09: 0000000000000005
<Oct/17 03:54 pm>[6853660.180001] R10: ffff8800000bc600 R11: 0000000000000082 R12: 0000000000001000

<Oct/17 03:54 pm>[6853660.187854] R13: 0000000000000002 R14: 0000000000000096 R15: ffff880372ec0020
<Oct/17 03:54 pm>[6853660.195723] FS:  00007fb16effc6f0(0000) GS:ffff8800026a0000(0000) knlGS:0000000000000000
<Oct/17 03:54 pm>[6853660.204582] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
<Oct/17 03:54 pm>[6853660.210832] CR2: 00007f5ad9eeae87 CR3: 000000050e583000 CR4: 00000000000006e0
<Oct/17 03:54 pm>[6853660.218717] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
<Oct/17 03:54 pm>[6853660.226593] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
<Oct/17 03:54 pm>[6853660.234473] Process sshd (pid: 5270, threadinfo ffff88037e3e0000, task ffff880372ec3090)
<Oct/17 03:54 pm>[6853660.243333] Stack:
<Oct/17 03:54 pm>[6853660.245781]  ffff88037e3e1879 ffff88037e3e1ae8 ffffffff812c093d ffff88037e3e1b08
<Oct/17 03:54 pm>[6853660.253710] <0> ffffffff810197e9 0000000000000006 000000000000d200 ffff88037e3e1b48
<Oct/17 03:54 pm>[6853660.262394] <0> ffffffff81019c76 ffffffff81b24210 0000000000000000 ffff8806552b3c00
<Oct/17 03:54 pm>[6853660.271483] Call Trace:
<Oct/17 03:54 pm>[6853660.274370]  [<ffffffff812c093d>] __const_udelay+0x3d/0x3f
<Oct/17 03:54 pm>[6853660.280377]  [<ffffffff810197e9>] native_safe_apic_wait_icr_idle+0x31/0x48
<Oct/17 03:54 pm>[6853660.287793]  [<ffffffff81019c76>] default_send_IPI_mask_sequence_phys+0x49/0xab
<Oct/17 03:54 pm>[6853660.295879]  [<ffffffff8101c5d8>] physflat_send_IPI_all+0x14/0x16
<Oct/17 03:54 pm>[6853660.302488]  [<ffffffff81019d94>] arch_trigger_all_cpu_backtrace+0x3c/0x5d
<Oct/17 03:54 pm>[6853660.309934]  [<ffffffff812c4ea4>] do_raw_spin_lock+0x11d/0x147
<Oct/17 03:54 pm>[6853660.316286]  [<ffffffff8151f193>] _raw_spin_lock+0x34/0x39
<Oct/17 03:54 pm>[6853660.322299]  [<ffffffff8102fb6b>] ? task_rq_lock+0x7e/0x9a
<Oct/17 03:54 pm>[6853660.328286]  [<ffffffff8102fb6b>] task_rq_lock+0x7e/0x9a
<Oct/17 03:54 pm>[6853660.334103]  [<ffffffff81036578>] try_to_wake_up+0x2e/0x342

<Oct/17 03:54 pm>[6853660.340205]  [<ffffffff81036899>] default_wake_function+0xd/0xf
<Oct/17 03:54 pm>[6853660.346671]  [<ffffffff810f9aba>] pollwake+0x57/0x59
<Oct/17 03:54 pm>[6853660.352163]  [<ffffffff8103688c>] ? default_wake_function+0x0/0xf
<Oct/17 03:54 pm>[6853660.358788]  [<ffffffff8102c08d>] __wake_up_common+0x46/0x76
<Oct/17 03:54 pm>[6853660.364982]  [<ffffffff8102fa4a>] __wake_up_sync_key+0x43/0x5a
<Oct/17 03:54 pm>[6853660.371335]  [<ffffffff8102fa6c>] __wake_up_sync+0xb/0xd
<Oct/17 03:54 pm>[6853660.377160]  [<ffffffff810f1b84>] pipe_write+0x4de/0x514
<Oct/17 03:54 pm>[6853660.382982]  [<ffffffff810f94de>] ? core_sys_select+0x3d/0x295
<Oct/17 03:54 pm>[6853660.389322]  [<ffffffff810ea282>] do_sync_write+0xc7/0x10d
<Oct/17 03:54 pm>[6853660.395336]  [<ffffffff8104a535>] ? sigprocmask+0x30/0xc8
<Oct/17 03:54 pm>[6853660.401265]  [<ffffffff812937db>] ? security_file_permission+0x11/0x13
<Oct/17 03:54 pm>[6853660.408357]  [<ffffffff810eaba1>] vfs_write+0xae/0x14a
<Oct/17 03:54 pm>[6853660.414028]  [<ffffffff810ead01>] sys_write+0x47/0x6e
<Oct/17 03:54 pm>[6853660.419593]  [<ffffffff810029ab>] system_call_fastpath+0x16/0x1b
<Oct/17 03:54 pm>[6853660.426121] Code: fb 48 ff c8 c9 c3 55 48 89 e5 48 83 ec 08 65 8b 34 25 f0 d1 00 00 66 66 90 0f ae e8 0f 31 89 c1 41 89 f1 66 66 90 0f ae e8 0f 31 <89> c6 48 89 f0 48 29 c8 48 39 f8 73 25 f3 90 65 44 8b 04 25 f0 
<Oct/17 03:54 pm>[6853660.449425] Call Trace:
<Oct/17 03:54 pm>[6853660.452319]  [<ffffffff812c093d>] __const_udelay+0x3d/0x3f
<Oct/17 03:54 pm>[6853660.458336]  [<ffffffff810197e9>] native_safe_apic_wait_icr_idle+0x31/0x48
<Oct/17 03:54 pm>[6853660.465768]  [<ffffffff81019c76>] default_send_IPI_mask_sequence_phys+0x49/0xab
<Oct/17 03:54 pm>[6853660.473831]  [<ffffffff8101c5d8>] physflat_send_IPI_all+0x14/0x16
<Oct/17 03:54 pm>[6853660.480491]  [<ffffffff81019d94>] arch_trigger_all_cpu_bac<Oct/17 03:54 pm>
                 <Oct/17 03:54 pm>[6853660.487947]  [<ffffffff812c4ea4>] do_raw_spin_lock+0x11d/0x147

<Oct/17 03:54 pm>[6853660.494287]  [<ffffffff8151f193>] _raw_spin_lock+0x34/0x39
<Oct/17 03:54 pm>[6853660.500284]  [<ffffffff8102fb6b>] ? task_rq_lock+0x7e/0x9a
<Oct/17 03:54 pm>[6853660.506274]  [<ffffffff8102fb6b>] task_rq_lock+0x7e/0x9a
<Oct/17 03:54 pm>[6853660.512097]  [<ffffffff81036578>] try_to_wake_up+0x2e/0x342
<Oct/17 03:54 pm>[6853660.518172]  [<ffffffff81036899>] default_wake_function+0xd/0xf
<Oct/17 03:54 pm>[6853660.524604]  [<ffffffff810f9aba>] pollwake+0x57/0x59
<Oct/17 03:54 pm>[6853660.530113]  [<ffffffff8103688c>] ? default_wake_function+0x0/0xf
<Oct/17 03:54 pm>[6853660.536738]  [<ffffffff8102c08d>] __wake_up_common+0x46/0x76
<Oct/17 03:54 pm>[6853660.542934]  [<ffffffff8102fa4a>] __wake_up_sync_key+0x43/0x5a
<Oct/17 03:54 pm>[6853660.549296]  [<ffffffff8102fa6c>] __wake_up_sync+0xb/0xd
<Oct/17 03:54 pm>[6853660.555119]  [<ffffffff810f1b84>] pipe_write+0x4de/0x514
<Oct/17 03:54 pm>[6853660.560961]  [<ffffffff810f94de>] ? core_sys_select+0x3d/0x295
<Oct/17 03:54 pm>[6853660.567297]  [<ffffffff810ea282>] do_sync_write+0xc7/0x10d
<Oct/17 03:54 pm>[6853660.573285]  [<ffffffff8104a535>] ? sigprocmask+0x30/0xc8
<Oct/17 03:54 pm>[6853660.579214]  [<ffffffff812937db>] ? security_file_permission+0x11/0x13
<Oct/17 03:54 pm>[6853660.586289]  [<ffffffff810eaba1>] vfs_write+0xae/0x14a
<Oct/17 03:54 pm>[6853660.591953]  [<ffffffff810ead01>] sys_write+0x47/0x6e
<Oct/17 03:54 pm>[6853660.597526]  [<ffffffff810029ab>] system_call_fastpath+0x16/0x1b
<Oct/17 03:54 pm>[6853660.604072] Pid: 5270, comm: sshd Tainted: G      D    2.6.34.4 #3
<Oct/17 03:54 pm>[6853660.610814] Call Trace:
<Oct/17 03:54 pm>[6853660.613715]  <NMI>  [<ffffffff81009b2a>] ? show_regs+0x26/0x2a
<Oct/17 03:54 pm>[6853660.620107]  [<ffffffff81520db8>] nmi_watchdog_tick+0xc3/0x1a5

<Oct/17 03:54 pm>[6853660.626485]  [<ffffffff81520360>] do_nmi+0xb2/0x259
<Oct/17 03:54 pm>[6853660.631862]  [<ffffffff8151fdfa>] nmi+0x1a/0x20
<Oct/17 03:54 pm>[6853660.636881]  [<ffffffff812c08af>] ? delay_tsc+0x25/0x59
<Oct/17 03:54 pm>[6853660.642601]  <<EOE>>  [<ffffffff812c093d>] __const_udelay+0x3d/0x3f
<Oct/17 03:54 pm>[6853660.649446]  [<ffffffff810197e9>] native_safe_apic_wait_icr_idle+0x31/0x48
<Oct/17 03:54 pm>[6853660.661397]  [<ffffffff81019c76>] default_send_IPI_mask_sequence_phys+0x49/0xab
<Oct/17 03:54 pm>[6853660.669441]  [<ffffffff8101c5d8>] physflat_send_IPI_all+0x14/0x16
<Oct/17 03:54 pm>[6853660.676061]  [<ffffffff81019d94>] arch_trigger_all_cpu_backtrace+0x3c/0x5d
<Oct/17 03:54 pm>[6853660.683490]  [<ffffffff812c4ea4>] do_raw_spin_lock+0x11d/0x147
<Oct/17 03:54 pm>[6853660.689832]  [<ffffffff8151f193>] _raw_spin_lock+0x34/0x39
<Oct/17 03:54 pm>[6853660.695845]  [<ffffffff8102fb6b>] ? task_rq_lock+0x7e/0x9a
<Oct/17 03:54 pm>[6853660.701868]  [<ffffffff8102fb6b>] task_rq_lock+0x7e/0x9a
<Oct/17 03:54 pm>[6853660.707667]  [<ffffffff81036578>] try_to_wake_up+0x2e/0x342
<Oct/17 03:54 pm>[6853660.713750]  [<ffffffff81036899>] default_wake_function+0xd/0xf
<Oct/17 03:54 pm>[6853660.720166]  [<ffffffff810f9aba>] pollwake+0x57/0x59
<Oct/17 03:54 pm>[6853660.725648]  [<ffffffff8103688c>] ? default_wake_function+<Oct/17 03:54 pm>
                 <Oct/17 03:54 pm>[6853660.732274]  [<ffffffff8102c08d>] __wake_up_common+0x46/0x76
<Oct/17 03:54 pm>[6853660.738495]  [<ffffffff8102fa4a>] __wake_up_sync_key+0x43/0x5a
<Oct/17 03:54 pm>[6853660.744847]  [<ffffffff8102fa6c>] __wake_up_sync+0xb/0xd
<Oct/17 03:54 pm>[6853660.750646]  [<ffffffff810f1b84>] pipe_write+0x4de/0x514
<Oct/17 03:54 pm>[6853660.756469]  [<ffffffff810f94de>] ? core_sys_select+0x3d/0x295
<Oct/17 03:54 pm>[6853660.762825]  [<ffffffff810ea282>] do_sync_write+0xc7/0x10d

<Oct/17 03:54 pm>[6853660.768846]  [<ffffffff8104a535>] ? sigprocmask+0x30/0xc8
<Oct/17 03:54 pm>[6853660.774749]  [<ffffffff812937db>] ? security_file_permission+0x11/0x13
<Oct/17 03:54 pm>[6853660.781850]  [<ffffffff810eaba1>] vfs_write+0xae/0x14a
<Oct/17 03:54 pm>[6853660.787522]  [<ffffffff810ead01>] sys_write+0x47/0x6e
<Oct/17 03:54 pm>[6853660.793095]  [<ffffffff810029ab>] system_call_fastpath+0x16/0x1b
<Oct/17 03:54 pm>[6853660.799616] NMI backtrace for cpu 6
                 <Oct/17 03:54 pm>[6853660.803575] CPU 6 
<Oct/17 03:54 pm>[6853660.805669] Modules linked in: w83793 hwmon_vid autofs4 i2c_i801 i2c_core iTCO_wdt e1000e ioatdma dca rtc_cmos tg3 libphy e1000 fuse dm_snapshot dm_mirror dm_region_hash dm_log scsi_wait_scan qla2xxx megaraid_sas megaraid_mbox megaraid_mm aacraid 3w_9xxx 3w_xxxx arcmsr aic7xxx aic79xx sr_mod sg ata_piix ahci sata_sis sata_nv sata_via pata_sl82c105 pata_sis
<Oct/17 03:54 pm>[6853660.841447] 
<Oct/17 03:54 pm>[6853660.843360] Pid: 8402, comm: nfsd Tainted: G      D    2.6.34.4 #3 S7010/empty
<Oct/17 03:54 pm>[6853660.851292] RIP: 0010:[<ffffffff812c08a2>]  [<ffffffff812c08a2>] delay_tsc+0x18/0x59
<Oct/17 03:54 pm>[6853660.859807] RSP: 0018:ffff880655203ce0  EFLAGS: 00000082
<Oct/17 03:54 pm>[6853660.865646] RAX: 00000000c72bf6a8 RBX: 00000000467b1e4c RCX: 00000000c72bf660
<Oct/17 03:54 pm>[6853660.873521] RDX: 000000000040ee8b RSI: 0000000000000006 RDI: 0000000000000001
<Oct/17 03:54 pm>[6853660.881386] RBP: ffff880655203ce8 R08: 0000000000000006 R09: 0000000000000006
<Oct/17 03:54 pm>[6853660.889237] R10: 0000000000000000 R11: ffff880655203b18 R12: ffff880655213c00
<Oct/17 03:54 pm>[6853660.897115] R13: 0000000000000000 R14: ffff880621c69030 R15: ffff880621c694b8
<Oct/17 03:54 pm>[6853660.904977] FS:  0000000000000000(0000) GS:ffff880655200000(0000) knlGS:0000000000000000
<Oct/17 03:54 pm>[6853660.913821] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
<Oct/17 03:54 pm>[6853660.920113] CR2: fffffffb9092c200 CR3: 0000000001a8a000 CR4: 00000000000006e0
<Oct/17 03:54 pm>[6853660.927969] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
<Oct/17 03:54 pm>[6853660.935856] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400

<Oct/17 03:54 pm>[6853660.943734] Process nfsd (pid: 8402, threadinfo ffff880621c72000, task ffff880621c69030)
<Oct/17 03:54 pm>[6853660.952536] Stack:
<Oct/17 03:54 pm>[6853660.954983]  ffffffff81019da9 ffff880655203cf8 ffffffff812c08fe ffff880655203d38
<Oct/17 03:54 pm>[6853660.962940] <0> ffffffff812c4e65 000000009ef2e6b4 ffff880655213c00 ffff880c2377c0c0
<Oct/17 03:54 pm>[6853660.971613] <0> ffff880655213c00 0000000000000000 0000000000000006 ffff880655203d58
<Oct/17 03:54 pm>[6853660.980639] Call Trace:
                 <Oct/17 03:54 pm>[6853660.983535]  <IRQ> 
<Oct/17 03:54 pm>[6853660.986137]  [<ffffffff81019da9>] ? arch_trigger_all_cpu_backtrace+0x51/0x5d
<Oct/17 03:54 pm>[6853660.993907]  [<ffffffff812c08fe>] __delay+0xa/0xc
<Oct/17 03:54 pm>[6853660.999127]  [<ffffffff812c4e65>] do_raw_spin_lock+0xde/0x147
<Oct/17 03:54 pm>[6853661.005405]  [<ffffffff8151f193>] _raw_spin_lock+0x34/0x39
<Oct/17 03:54 pm>[6853661.011410]  [<ffffffff8103666d>] ? try_to_wake_up+0x123/0x342
<Oct/17 03:54 pm>[6853661.017773]  [<ffffffff8103666d>] try_to_wake_up+0x123/0x342
<Oct/17 03:54 pm>[6853661.023968]  [<ffffffff81036899>] default_wake_function+0xd/0xf
<Oct/17 03:54 pm>[6853661.030444]  [<ffffffff810f9aba>] pollwake+0x57/0x59
<Oct/17 03:54 pm>[6853661.035928]  [<ffffffff8103688c>] ? default_wake_function+0x0/0xf
<Oct/17 03:54 pm>[6853661.042570]  [<ffffffff8102c08d>] __wake_up_common+0x46/0x76
<Oct/17 03:54 pm>[6853661.048765]  [<ffffffff8102faa6>] __wake_up+0x38/0x4f
<Oct/17 03:54 pm>[6853661.054326]  [<ffffffff8103b959>] printk_tick+0x39/0x3b
<Oct/17 03:54 pm>[6853661.060065]  [<ffffffff810483c7>] update_process_times+0x3f/0x59
<Oct/17 03:54 pm>[6853661.066629]  [<ffffffff810610b4>] tick_sched_timer+0x73/0x9d
<Oct/17 03:54 pm>[6853661.072816]  [<ffffffff81061041>] ? tick_sched_timer+0x0/0x9d

<Oct/17 03:54 pm>[6853661.079091]  [<ffffffff81057933>] __run_hrtimer+0xd0/0x12d
<Oct/17 03:54 pm>[6853661.085121]  [<ffffffff81057b81>] hrtimer_interrupt+0xb5/0x19f
<Oct/17 03:54 pm>[6853661.091485]  [<ffffffff810197a5>] smp_apic_timer_interrupt+0x82/0x95
<Oct/17 03:54 pm>[6853661.098394]  [<ffffffff81003353>] apic_timer_interrupt+0x13/0x20
<Oct/17 03:54 pm>[6853661.104934]  <EOI> 
<Oct/17 03:54 pm>[6853661.107124] Code: 2e 0f 1f 84 00 00 00 00 00 48 ff c8 75 fb 48 ff c8 c9 c3 55 48 89 e5 48 83 ec 08 65 8b 34 25 f0 d1 00 00 66 66 90 0f ae e8 0f 31 <89> c1 41 89 f1 66 66 90 0f ae e8 0f 31 89 c6 48 89 f0 48 29 c8 
<Oct/17 03:54 pm>[6853661.130831] Call Trace:
<Oct/17 03:54 pm>[6853661.133707]  <IRQ>  [<ffffffff81019da9>] ? arch_trigger_all_cpu_backtrace+0x51/0x5d
<Oct/17 03:54 pm>[6853661.142178]  [<ffffffff812c08fe>] __delay+0xa/0xc
<Oct/17 03:54 pm>[6853661.147403]  [<ffffffff812c4e65>] do_raw_spin_lock+0xde/0x147
<Oct/17 03:54 pm>[6853661.153690]  [<ffffffff8151f193>] _raw_spin_lock+0x34/0x39
<Oct/17 03:54 pm>[6853661.159704]  [<ffffffff8103666d>] ? try_to_wake_up+0x123/0x342
<Oct/17 03:54 pm>[6853661.166057]  [<ffffffff8103666d>] try_to_wake_up+0x123/0x342
<Oct/17 03:54 pm>[6853661.172235]  [<ffffffff81036899>] default_wake_function+0xd/0xf
<Oct/17 03:54 pm>[6853661.178711]  [<ffffffff810f9aba>] pollwake+0x57/0x59
<Oct/17 03:54 pm>[6853661.184210]  [<ffffffff8103688c>] ? default_wake_function+0x0/0xf
<Oct/17 03:54 pm>[6853661.190862]  [<ffffffff8102c08d>] __wake_up_common+0x46/0x76
<Oct/17 03:54 pm>[6853661.197039]  [<ffffffff8102faa6>] __wake_up+0x38/0x4f
<Oct/17 03:54 pm>[6853661.202601]  [<ffffffff8103b959>] printk_tick+0x39/0x3b
                 <Oct/17 03:54 pm>[6853661.208349]  [<ffffffff810483c7>] update_process_times+0x3f/0x59
<Oct/17 03:54 pm>[6853661.214879]  [<ffffffff810610b4>] tick_sched_timer+0x73/0x9d
<Oct/17 03:54 pm>[6853661.221073]  [<ffffffff81061041>] ? tick_sched_timer+0x0/0x9d

<Oct/17 03:54 pm>[6853661.227348]  [<ffffffff81057933>] __run_hrtimer+0xd0/0x12d
<Oct/17 03:54 pm>[6853661.233344]  [<ffffffff81057b81>] hrtimer_interrupt+0xb5/0x19f
<Oct/17 03:54 pm>[6853661.239715]  [<ffffffff810197a5>] smp_apic_timer_interrupt+0x82/0x95
<Oct/17 03:54 pm>[6853661.246601]  [<ffffffff81003353>] apic_timer_interrupt+0x13/0x20
<Oct/17 03:54 pm>[6853661.253125]  <EOI> Pid: 8402, comm: nfsd Tainted: G      D    2.6.34.4 #3
<Oct/17 03:54 pm>[6853661.260521] Call Trace:
<Oct/17 03:54 pm>[6853661.263410]  <NMI>  [<ffffffff81009b2a>] ? show_regs+0x26/0x2a
<Oct/17 03:54 pm>[6853661.269801]  [<ffffffff81520db8>] nmi_watchdog_tick+0xc3/0x1a5
<Oct/17 03:54 pm>[6853661.276153]  [<ffffffff81520360>] do_nmi+0xb2/0x259
<Oct/17 03:54 pm>[6853661.281575]  [<ffffffff8151fdfa>] nmi+0x1a/0x20
<Oct/17 03:54 pm>[6853661.286592]  [<ffffffff812c08a2>] ? delay_tsc+0x18/0x59
<Oct/17 03:54 pm>[6853661.292305]  <<EOE>>  <IRQ>  [<ffffffff81019da9>] ? arch_trigger_all_cpu_backtrace+0x51/0x5d
<Oct/17 03:54 pm>[6853661.301589]  [<ffffffff812c08fe>] __delay+0xa/0xc
<Oct/17 03:54 pm>[6853661.306787]  [<ffffffff812c4e65>] do_raw_spin_lock+0xde/0x147
<Oct/17 03:54 pm>[6853661.313066]  [<ffffffff8151f193>] _raw_spin_lock+0x34/0x39
<Oct/17 03:54 pm>[6853661.319097]  [<ffffffff8103666d>] ? try_to_wake_up+0x123/0x342
<Oct/17 03:54 pm>[6853661.325458]  [<ffffffff8103666d>] try_to_wake_up+0x123/0x342
<Oct/17 03:54 pm>[6853661.331647]  [<ffffffff81036899>] default_wake_function+0xd/0xf
<Oct/17 03:54 pm>[6853661.338112]  [<ffffffff810f9aba>] pollwake+0x57/0x59
<Oct/17 03:54 pm>[6853661.343596]  [<ffffffff8103688c>] ? default_wake_function+0x0/0xf
<Oct/17 03:54 pm>[6853661.350205]  [<ffffffff8102c08d>] __wake_up_common+0x46/0x76
<Oct/17 03:54 pm>[6853661.356390]  [<ffffffff8102faa6>] __wake_up+0x38/0x4f

<Oct/17 03:54 pm>[6853661.361970]  [<ffffffff8103b959>] printk_tick+0x39/0x3b
<Oct/17 03:54 pm>[6853661.367734]  [<ffffffff810483c7>] update_process_times+0x3f/0x59
<Oct/17 03:54 pm>[6853661.374254]  [<ffffffff810610b4>] tick_sched_timer+0x73/0x9d
<Oct/17 03:54 pm>[6853661.380468]  [<ffffffff81061041>] ? tick_sched_timer+0x0/0x9d
<Oct/17 03:54 pm>[6853661.386725]  [<ffffffff81057933>] __run_hrtimer+0xd0/0x12d
<Oct/17 03:54 pm>[6853661.392714]  [<ffffffff81057b81>] hrtimer_interrupt+0xb5/0x19f
<Oct/17 03:54 pm>[6853661.399068]  [<ffffffff810197a5>] smp_apic_timer_interrupt+0x82/0x95
<Oct/17 03:54 pm>[6853661.405934]  [<ffffffff81003353>] apic_timer_interrupt+0x13/0x20
<Oct/17 03:54 pm>[6853661.412493]  <EOI> 
<Oct/17 03:54 pm>[6853661.414689] NMI backtrace for cpu 7
<Oct/17 03:54 pm>[6853661.419053] CPU 7 
<Oct/17 03:54 pm>[6853661.421162] Modules linked in: w83793 hwmon_vid autofs4 i2c_i801 i2c_core iTCO_wdt e1000e ioatdma dca rtc_cmos tg3 libphy e1000 fuse dm_snapshot dm_mirror dm_region_hash dm_log scsi_wait_scan qla2xxx megaraid_sas megaraid_mbox megaraid_mm aacraid 3w_9xxx 3w_xxxx arcmsr aic7xxx aic79xx sr_mod sg ata_piix ahci sata_sis sata_nv sata_via pata_sl82c105 pata_sis
<Oct/17 03:54 pm>[6853661.457037] 
<Oct/17 03:54 pm>[6853661.458949] Pid: 19727, comm: ppImage Tainted: G      D    2.6.34.4 #3 S7010/empty
<Oct/17 03:54 pm>[6853661.467292] RIP: 0010:[<ffffffff812c088a>]  [<ffffffff812c088a>] delay_tsc+0x0/0x59
<Oct/17 03:54 pm>[6853661.475728] RSP: 0000:ffff880088bb97b0  EFLAGS: 00000002
<Oct/17 03:54 pm>[6853661.481575] RAX: 0000000000000000 RBX: 000000004aaa6716 RCX: 00000000c7300a38
<Oct/17 03:54 pm>[6853661.489468] RDX: 0000000000001900 RSI: 00000000c7300a54 RDI: 0000000000000001
<Oct/17 03:54 pm>[6853661.497363] RBP: ffff880088bb97b8 R08: 0000000000000007 R09: 0000000000000007
<Oct/17 03:54 pm>[6853661.505224] R10: 0000000000000000 R11: ffff880088bb95d8 R12: ffff8806552b3c00
<Oct/17 03:54 pm>[6853661.513128] R13: 0000000000000000 R14: ffff880611552060 R15: ffff8806115524e8
<Oct/17 03:54 pm>[6853661.520991] FS:  0000000041128950(0063) GS:ffff880655220000(0000) knlGS:0000000000000000

<Oct/17 03:54 pm>[6853661.529826] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<Oct/17 03:54 pm>[6853661.536091] CR2: 00007fa3df2d2be0 CR3: 00000003e2c88000 CR4: 00000000000006e0
<Oct/17 03:54 pm>[6853661.544001] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
<Oct/17 03:54 pm>[6853661.551880] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
<Oct/17 03:54 pm>[6853661.559712] Process ppImage (pid: 19727, threadinfo ffff880088bb8000, task ffff880611552060)
<Oct/17 03:54 pm>[6853661.568930] Stack:
<Oct/17 03:54 pm>[6853661.571378]  ffffffff812c08fe ffff880088bb97f8 ffffffff812c4e65 000000009ef2e6b4
<Oct/17 03:54 pm>[6853661.579324] <0> ffff8806552b3c00 ffff8806552b3c00 ffff88065522e3f0 0000000000000400
<Oct/17 03:54 pm>[6853661.587948] <0> 0000000000000800 ffff880088bb9818 ffffffff8151f0dc ffffffff81035934
<Oct/17 03:54 pm>[6853661.601586] Call Trace:
<Oct/17 03:54 pm>[6853661.604479]  [<ffffffff812c08fe>] ? __delay+0xa/0xc
<Oct/17 03:54 pm>[6853661.609889]  [<ffffffff812c4e65>] do_raw_spin_lock+0xde/0x147
<Oct/17 03:54 pm>[6853661.616168]  [<ffffffff8151f0dc>] _raw_spin_lock_nested+0x32/0x36
<Oct/17 03:54 pm>[6853661.622807]  [<ffffffff81035934>] ? double_rq_lock+0x4a/0x76
<Oct/17 03:54 pm>[6853661.629011]  [<ffffffff81035934>] double_rq_lock+0x4a/0x76
<Oct/17 03:54 pm>[6853661.635024]  [<ffffffff81037672>] load_balance+0xbb6/0xf63
<Oct/17 03:54 pm>[6853661.641014]  [<ffffffff810b3cfb>] ? get_dirty_limits+0x22/0x258
<Oct/17 03:54 pm>[6853661.647508]  [<ffffffff8151cc0e>] schedule+0x25a/0x86a
<Oct/17 03:54 pm>[6853661.653173]  [<ffffffff810480a2>] ? lock_timer_base+0x26/0x4a
<Oct/17 03:54 pm>[6853661.659432]  [<ffffffff8151d559>] schedule_timeout+0x222/0x24c
<Oct/17 03:54 pm>[6853661.665803]  [<ffffffff810482a5>] ? process_timeout+0x0/0xb
<Oct/17 03:54 pm>[6853661.671896]  [<ffffffff8151c6c5>] io_schedule_timeout+0x45/0x5e

<Oct/17 03:54 pm>[6853661.678353]  [<ffffffff810c0abe>] congestion_wait+0x6b/0x87
<Oct/17 03:54 pm>[6853661.684452]  [<ffffffff81054b6c>] ? autoremove_wake_function+0x0/0x38
<Oct/17 03:54 pm>[6853661.691442]  [<ffffffff810b3279>] __alloc_pages_nodemask+0x65a/0x671
<Oct/17 03:54 pm>[6853661.698358]  [<ffffffff810da90f>] alloc_page_vma+0x148/0x14d
<Oct/17 03:54 pm>[6853661.704552]  [<ffffffff810c4d2a>] handle_mm_fault+0x294/0x73e
<Oct/17 03:54 pm>[6853661.710861]  [<ffffffff815227e9>] do_page_fault+0x351/0x375
<Oct/17 03:54 pm>[6853661.716980]  [<ffffffff8151e35f>] ? do_nanosleep+0x7b/0xb3
<Oct/17 03:54 pm>[6853661.723003]  [<ffffffff810348d8>] ? finish_task_switch+0x69/0xdd
<Oct/17 03:54 pm>[6853661.729523]  [<ffffffff8103486f>] ? finish_task_switch+0x0<Oct/17 03:54 pm>
                 <Oct/17 03:54 pm>[6853661.735988]  [<ffffffff81058477>] ? hrtimer_nanosleep+0x97/0x105
<Oct/17 03:54 pm>[6853661.742539]  [<ffffffff81057842>] ? hrtimer_wakeup+0x0/0x21
<Oct/17 03:54 pm>[6853661.748641]  [<ffffffff8151ef6f>] ? lockdep_sys_exit_thunk+0x35/0x67
<Oct/17 03:54 pm>[6853661.755568]  [<ffffffff810c4dda>] ? handle_mm_fault+0x344/0x73e
<Oct/17 03:54 pm>[6853661.762014]  [<ffffffff8151fb5f>] page_fault+0x1f/0x30
<Oct/17 03:54 pm>[6853661.767653] Code: 90 55 48 89 f8 48 89 e5 48 85 c0 74 19 eb 02 66 90 eb 0e 66 66 66 66 66 2e 0f 1f 84 00 00 00 00 00 48 ff c8 75 fb 48 ff c8 c9 c3 <55> 48 89 e5 48 83 ec 08 65 8b 34 25 f0 d1 00 00 66 66 90 0f ae 
<Oct/17 03:54 pm>[6853661.790940] Call Trace:
<Oct/17 03:54 pm>[6853661.793836]  [<ffffffff812c08fe>] ? __delay+0xa/0xc
<Oct/17 03:54 pm>[6853661.799191]  [<ffffffff812c4e65>] do_raw_spin_lock+0xde/0x147
<Oct/17 03:54 pm>[6853661.805452]  [<ffffffff8151f0dc>] _raw_spin_lock_nested+0x32/0x36
<Oct/17 03:54 pm>[6853661.812084]  [<ffffffff81035934>] ? double_rq_lock+0x4a/0x76
<Oct/17 03:54 pm>[6853661.818297]  [<ffffffff81035934>] double_rq_lock+0x4a/0x76
                 <Oct/17 03:54 pm>[6853661.824320]  [<ffffffff81037672>] load_balance+0xbb6/0xf63

<Oct/17 03:54 pm>[6853661.830317]  [<ffffffff810b3cfb>] ? get_dirty_limits+0x22/0x258
<Oct/17 03:54 pm>[6853661.836803]  [<ffffffff8151cc0e>] schedule+0x25a/0x86a
<Oct/17 03:54 pm>[6853661.842451]  [<ffffffff810480a2>] ? lock_timer_base+0x26/0x4a
<Oct/17 03:54 pm>[6853661.848710]  [<ffffffff8151d559>] schedule_timeout+0x222/0x24c
<Oct/17 03:54 pm>[6853661.855071]  [<ffffffff810482a5>] ? process_timeout+0x0/0xb
<Oct/17 03:54 pm>[6853661.861172]  [<ffffffff8151c6c5>] io_schedule_timeout+0x45/0x5e
<Oct/17 03:54 pm>[6853661.867631]  [<ffffffff810c0abe>] congestion_wait+0x6b/0x87
<Oct/17 03:54 pm>[6853661.873758]  [<ffffffff81054b6c>] ? autoremove_wake_function+0x0/0x38
<Oct/17 03:54 pm>[6853661.880782]  [<ffffffff810b3279>] __alloc_pages_nodemask+0x65a/0x671
<Oct/17 03:54 pm>[6853661.887689]  [<ffffffff810da90f>] alloc_page_vma+0x148/0x14d
<Oct/17 03:55 pm>[6853661.893857]  [<ffffffff810c4d2a>] handle_mm_fault+0x294/0x73e
<Oct/17 03:55 pm>[6853661.900121]  [<ffffffff815227e9>] do_page_fault+0x351/0x375
<Oct/17 03:55 pm>[6853661.906240]  [<ffffffff8151e35f>] ? do_nanosleep+0x7b/0xb3
<Oct/17 03:55 pm>[6853661.912247]  [<ffffffff810348d8>] ? finish_task_switch+0x69/0xdd
<Oct/17 03:55 pm>[6853661.918792]  [<ffffffff8103486f>] ? finish_task_switch+0x0/0xdd
<Oct/17 03:55 pm>[6853661.925249]  [<ffffffff81058477>] ? hrtimer_nanosleep+0x97/0x105
<Oct/17 03:55 pm>[6853661.931793]  [<ffffffff81057842>] ? hrtimer_wakeup+0x0/0x21
<Oct/17 03:55 pm>[6853661.937902]  [<ffffffff8151ef6f>] ? lockdep_sys_exit_thunk+0x35/0x67
<Oct/17 03:55 pm>[6853661.944819]  [<ffffffff810c4dda>] ? handle_mm_fault+0x344/0x73e
<Oct/17 03:55 pm>[6853661.951282]  [<ffffffff8151fb5f>] page_fault+0x1f/0x30
<Oct/17 03:55 pm>[6853661.956941] Pid: 19727, comm: ppImage Tainted: G      D    2.6.34.4 #3
<Oct/17 03:55 pm>[6853661.964042] Call Trace:

<Oct/17 03:55 pm>[6853661.966916]  <NMI>  [<ffffffff81009b2a>] ? show_regs+0x26/0x2a
<Oct/17 03:55 pm>[6853661.973273]  [<ffffffff81520db8>] nmi_watchdog_tick+0xc3/0x1a5
<Oct/17 03:55 pm>[6853661.979668]  [<ffffffff81520360>] do_nmi+0xb2/0x259
<Oct/17 03:55 pm>[6853661.985037]  [<ffffffff8151fdfa>] nmi+0x1a/0x20
<Oct/17 03:55 pm>[6853661.990047]  [<ffffffff812c088a>] ? delay_tsc+0x0/0x59
                 <Oct/17 03:55 pm>[6853661.995715]  <<EOE>>  [<ffffffff812c08fe>] ? __delay+0xa/0xc
<Oct/17 03:55 pm>[6853662.001926]  [<ffffffff812c4e65>] do_raw_spin_lock+0xde/0x147
<Oct/17 03:55 pm>[6853662.008192]  [<ffffffff8151f0dc>] _raw_spin_lock_nested+0x32/0x36
<Oct/17 03:55 pm>[6853662.014806]  [<ffffffff81035934>] ? double_rq_lock+0x4a/0x76
<Oct/17 03:55 pm>[6853662.020984]  [<ffffffff81035934>] double_rq_lock+0x4a/0x76
<Oct/17 03:55 pm>[6853662.026962]  [<ffffffff81037672>] load_balance+0xbb6/0xf63
<Oct/17 03:55 pm>[6853662.032962]  [<ffffffff810b3cfb>] ? get_dirty_limits+0x22/0x258
<Oct/17 03:55 pm>[6853662.039409]  [<ffffffff8151cc0e>] schedule+0x25a/0x86a
<Oct/17 03:55 pm>[6853662.045059]  [<ffffffff810480a2>] ? lock_timer_base+0x26/0x4a
<Oct/17 03:55 pm>[6853662.051336]  [<ffffffff8151d559>] schedule_timeout+0x222/0x24c
<Oct/17 03:55 pm>[6853662.057671]  [<ffffffff810482a5>] ? process_timeout+0x0/0xb
<Oct/17 03:55 pm>[6853662.063781]  [<ffffffff8151c6c5>] io_schedule_timeout+0x45/0x5e
<Oct/17 03:55 pm>[6853662.070238]  [<ffffffff810c0abe>] congestion_wait+0x6b/0x8<Oct/17 03:55 pm>
                 <Oct/17 03:55 pm>[6853662.076340]  [<ffffffff81054b6c>] ? autoremove_wake_function+0x0/0x38
<Oct/17 03:55 pm>[6853662.083310]  [<ffffffff810b3279>] __alloc_pages_nodemask+0x65a/0x671
<Oct/17 03:55 pm>[6853662.090209]  [<ffffffff810da90f>] alloc_page_vma+0x148/0x14d
<Oct/17 03:55 pm>[6853662.096403]  [<ffffffff810c4d2a>] handle_mm_fault+0x294/0x73e

<Oct/17 03:55 pm>[6853662.102651]  [<ffffffff815227e9>] do_page_fault+0x351/0x375
<Oct/17 03:55 pm>[6853662.108763]  [<ffffffff8151e35f>] ? do_nanosleep+0x7b/0xb3
<Oct/17 03:55 pm>[6853662.114784]  [<ffffffff810348d8>] ? finish_task_switch+0x69/0xdd
<Oct/17 03:55 pm>[6853662.121331]  [<ffffffff8103486f>] ? finish_task_switch+0x0/0xdd
<Oct/17 03:55 pm>[6853662.127832]  [<ffffffff81058477>] ? hrtimer_nanosleep+0x97/0x105
<Oct/17 03:55 pm>[6853662.134383]  [<ffffffff81057842>] ? hrtimer_wakeup+0x0/0x21
<Oct/17 03:55 pm>[6853662.140499]  [<ffffffff8151ef6f>] ? lockdep_sys_exit_thunk+0x35/0x67
<Oct/17 03:55 pm>[6853662.147420]  [<ffffffff810c4dda>] ? handle_mm_fault+0x344/0x73e
<Oct/17 03:55 pm>[6853662.153857]  [<ffffffff8151fb5f>] page_fault+0x1f/0x30
<Oct/17 03:55 pm>[6853662.159504] NMI backtrace for cpu 8
<Oct/17 03:55 pm>[6853662.163453] CPU 8 
<Oct/17 03:55 pm>[6853662.165551] Modules linked in: w83793 hwmon_vid autofs4 i2c_i801 i2c_core iTCO_wdt e1000e ioatdma dca rtc_cmos tg3 libphy e1000 fuse dm_snapshot dm_mirror dm_region_hash dm_log scsi_wait_scan qla2xxx megaraid_sas megaraid_mbox megaraid_mm aacraid 3w_9xxx 3w_xxxx arcmsr aic7xxx aic79xx sr_mod sg ata_piix ahci sata_sis sata_nv sata_via pata_sl82c105 pata_sis
<Oct/17 03:55 pm>[6853662.201375] 
<Oct/17 03:55 pm>[6853662.203296] Pid: 8429, comm: nfsd Tainted: G      D    2.6.34.4 #3 S7010/empty
<Oct/17 03:55 pm>[6853662.211253] RIP: 0010:[<ffffffff812c08a0>]  [<ffffffff812c08a0>] delay_tsc+0x16/0x59
<Oct/17 03:55 pm>[6853662.219814] RSP: 0018:ffff880621ee16b8  EFLAGS: 00000096
<Oct/17 03:55 pm>[6853662.225660] RAX: 0000000000000000 RBX: 000000004aa9f2fe RCX: 00000000c7341cd8
<Oct/17 03:55 pm>[6853662.233511] RDX: 0000000000001900 RSI: 0000000000000008 RDI: 0000000000000001
<Oct/17 03:55 pm>[6853662.241412] RBP: ffff880621ee16c0 R08: 0000000000000008 R09: 0000000000000008
<Oct/17 03:55 pm>[6853662.249292] R10: 0000000000000000 R11: ffff880621ee14f0 R12: ffff8806552b3c00
<Oct/17 03:55 pm>[6853662.257197] R13: 0000000000000000 R14: ffff880621ed8000 R15: ffff880621ed8488
<Oct/17 03:55 pm>[6853662.265076] FS:  0000000000000000(0000) GS:ffff880655240000(0000) knlGS:0000000000000000

<Oct/17 03:55 pm>[6853662.273954] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
<Oct/17 03:55 pm>[6853662.280194] CR2: 0000000003178f50 CR3: 0000000001a8a000 CR4: 00000000000006e0
<Oct/17 03:55 pm>[6853662.288087] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
<Oct/17 03:55 pm>[6853662.295983] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
<Oct/17 03:55 pm>[6853662.303860] Process nfsd (pid: 8429, threadinfo ffff880621ee0000, task ffff880621ed8000)
<Oct/17 03:55 pm>[6853662.312703] Stack:
<Oct/17 03:55 pm>[6853662.315161]  ffffffff81019da9 ffff880621ee16d0 ffffffff812c08fe ffff880621ee1710
<Oct/17 03:55 pm>[6853662.323073] <0> ffffffff812c4e65 000000009ef2e6b4 ffff8806552b3c00 ffff8806552b3c00
<Oct/17 03:55 pm>[6853662.331758] <0> ffff88065524e3f0 0000000000000400 0000000000000800 ffff880621ee1730
<Oct/17 03:55 pm>[6853662.340852] Call Trace:
<Oct/17 03:55 pm>[6853662.343738]  [<ffffffff81019da9>] ? arch_trigger_all_cpu_backtrace+0x51/0x5d
<Oct/17 03:55 pm>[6853662.351510]  [<ffffffff812c08fe>] __delay+0xa/0xc
<Oct/17 03:55 pm>[6853662.356728]  [<ffffffff812c4e65>] do_raw_spin_lock+0xde/0x147
<Oct/17 03:55 pm>[6853662.363026]  [<ffffffff8151f0dc>] _raw_spin_lock_nested+0x32/0x36
<Oct/17 03:55 pm>[6853662.369649]  [<ffffffff81035934>] ? double_rq_lock+0x4a/0x76
<Oct/17 03:55 pm>[6853662.375834]  [<ffffffff81035934>] double_rq_lock+0x4a/0x76
<Oct/17 03:55 pm>[6853662.381874]  [<ffffffff81037672>] load_balance+0xbb6/0xf63
<Oct/17 03:55 pm>[6853662.387908]  [<ffffffff810b2173>] ? __pagevec_free+0x70/0x85
<Oct/17 03:55 pm>[6853662.394116]  [<ffffffff8151cc0e>] schedule+0x25a/0x86a
<Oct/17 03:55 pm>[6853662.399737]  [<ffffffff8144b38b>] lock_sock_nested+0x82/0xe2
<Oct/17 03:55 pm>[6853662.405937]  [<ffffffff81054b6c>] ? autoremove_wake_function+0x0/0x38
<Oct/17 03:55 pm>[6853662.412941]  [<ffffffff81452976>] skb_free_datagram_locked+0x14/0x2c

<Oct/17 03:55 pm>[6853662.419831]  [<ffffffff8149bbfb>] udp_recvmsg+0x1a5/0x209
<Oct/17 03:55 pm>[6853662.425730]  [<ffffffff8144a75d>] sock_common_recvmsg+0x32/0x47
<Oct/17 03:55 pm>[6853662.432196]  [<ffffffff81448584>] sock_recvmsg+0xc1/0xda
<Oct/17 03:55 pm>[6853662.438030]  [<ffffffff810480a2>] ? lock_timer_base+0x26/0x4a
<Oct/17 03:55 pm>[6853662.444304]  [<ffffffff810480a2>] ? lock_timer_base+0x26/0x4a
<Oct/17 03:55 pm>[6853662.450570]  [<ffffffff81048174>] ? try_to_del_timer_sync+0xae/0xbb
<Oct/17 03:55 pm>[6853662.457360]  [<ffffffff814485d2>] kernel_recvmsg+0x35/0x41
<Oct/17 03:55 pm>[6853662.463363]  [<ffffffff814f1954>] svc_udp_recvfrom+0xa1/0x2f9
<Oct/17 03:55 pm>[6853662.469639]  [<ffffffff814fc980>] svc_recv+0x758/0x7f5
<Oct/17 03:55 pm>[6853662.475313]  [<ffffffff814fc1d9>] ? svc_send+0x85/0xa1
<Oct/17 03:55 pm>[6853662.480973]  [<ffffffff8103688c>] ? default_wake_function+0x0/0xf
<Oct/17 03:55 pm>[6853662.487613]  [<ffffffff811f0ccf>] ? nfsd+0x0/0x139
<Oct/17 03:55 pm>[6853662.492922]  [<ffffffff811f0d78>] nfsd+0xa9/0x139
<Oct/17 03:55 pm>[6853662.498131]  [<ffffffff810546e2>] kthread+0x7d/0x85
<Oct/17 03:55 pm>[6853662.503514]  [<ffffffff81003794>] kernel_thread_helper+0x4/0x10
<Oct/17 03:55 pm>[6853662.509966]  [<ffffffff81054665>] ? kthread+0x0/0x85
<Oct/17 03:55 pm>[6853662.515457]  [<ffffffff81003790>] ? kernel_thread_helper+0x0/0x10
<Oct/17 03:55 pm>[6853662.522075] Code: 66 66 2e 0f 1f 84 00 00 00 00 00 48 ff c8 75 fb 48 ff c8 c9 c3 55 48 89 e5 48 83 ec 08 65 8b 34 25 f0 d1 00 00 66 66 90 0f ae e8 <0f> 31 89 c1 41 89 f1 66 66 90 0f ae e8 0f 31 89 c6 48 89 f0 48 
<Oct/17 03:55 pm>[6853662.545331] Call Trace:
                 <Oct/17 03:55 pm>[6853662.548202]  [<ffffffff81019da9>] ? arch_trigger_all_cpu_backtrace+0x51/0x5d
<Oct/17 03:55 pm>[6853662.555974]  [<ffffffff812c08fe>] __delay+0xa/0xc
<Oct/17 03:55 pm>[6853662.561167]  [<ffffffff812c4e65>] do_raw_spin_lock+0xde/0x147

<Oct/17 03:55 pm>[6853662.567410]  [<ffffffff8151f0dc>] _raw_spin_lock_nested+0x32/0x36
<Oct/17 03:55 pm>[6853662.574016]  [<ffffffff81035934>] ? double_rq_lock+0x4a/0x76
<Oct/17 03:55 pm>[6853662.580201]  [<ffffffff81035934>] double_rq_lock+0x4a/0x76
<Oct/17 03:55 pm>[6853662.586225]  [<ffffffff81037672>] load_balance+0xbb6/0xf63
<Oct/17 03:55 pm>[6853662.592259]  [<ffffffff810b2173>] ? __pagevec_free+0x70/0x85
<Oct/17 03:55 pm>[6853662.598465]  [<ffffffff8151cc0e>] schedule+0x25a/0x86a
<Oct/17 03:55 pm>[6853662.604122]  [<ffffffff8144b38b>] lock_sock_nested+0x82/0xe2
<Oct/17 03:55 pm>[6853662.610310]  [<ffffffff81054b6c>] ? autoremove_wake_function+0x0/0x38
<Oct/17 03:55 pm>[6853662.617300]  [<ffffffff81452976>] skb_free_datagram_locked<Oct/17 03:55 pm>
                 <Oct/17 03:55 pm>[6853662.624207]  [<ffffffff8149bbfb>] udp_recvmsg+0x1a5/0x209
<Oct/17 03:55 pm>[6853662.634640]  [<ffffffff8144a75d>] sock_common_recvmsg+0x32/0x47
<Oct/17 03:55 pm>[6853662.641107]  [<ffffffff81448584>] sock_recvmsg+0xc1/0xda
<Oct/17 03:55 pm>[6853662.646911]  [<ffffffff810480a2>] ? lock_timer_base+0x26/0<Oct/17 03:55 pm>
                 <Oct/17 03:55 pm>[6853662.653171]  [<ffffffff810480a2>] ? lock_timer_base+0x26/0x4a
<Oct/17 03:55 pm>[6853662.659453]  [<ffffffff81048174>] ? try_to_del_timer_sync+0xae/0xbb
<Oct/17 03:55 pm>[6853662.666302]  [<ffffffff814485d2>] kernel_recvmsg+0x35/0x41
<Oct/17 03:55 pm>[6853662.672287]  [<ffffffff814f1954>] svc_udp_recvfrom+0xa1/0x2f9
<Oct/17 03:55 pm>[6853662.678573]  [<ffffffff814fc980>] svc_recv+0x758/0x7f5
<Oct/17 03:55 pm>[6853662.684220]  [<ffffffff814fc1d9>] ? svc_send+0x85/0xa1
<Oct/17 03:55 pm>[6853662.689863]  [<ffffffff8103688c>] ? default_wake_function+0x0/0xf
<Oct/17 03:55 pm>[6853662.696496]  [<ffffffff811f0ccf>] ? nfsd+0x0/0x139
<Oct/17 03:55 pm>[6853662.701796]  [<ffffffff811f0d78>] nfsd+0xa9/0x139

<Oct/17 03:55 pm>[6853662.707006]  [<ffffffff810546e2>] kthread+0x7d/0x85
<Oct/17 03:55 pm>[6853662.712380]  [<ffffffff81003794>] kernel_thread_helper+0x4/0x10
<Oct/17 03:55 pm>[6853662.718841]  [<ffffffff81054665>] ? kthread+0x0/0x85
<Oct/17 03:55 pm>[6853662.724331]  [<ffffffff81003790>] ? kernel_thread_helper+0x0/0x10
<Oct/17 03:55 pm>[6853662.730965] Pid: 8429, comm: nfsd Tainted: G      D    2.6.34.4 #3
<Oct/17 03:55 pm>[6853662.737689] Call Trace:
<Oct/17 03:55 pm>[6853662.740581]  <NMI>  [<ffffffff81009b2a>] ? show_regs+0x26/0x2a
<Oct/17 03:55 pm>[6853662.747000]  [<ffffffff81520db8>] nmi_watchdog_tick+0xc3/0x1a5
<Oct/17 03:55 pm>[6853662.753358]  [<ffffffff81520360>] do_nmi+0xb2/0x259
<Oct/17 03:55 pm>[6853662.758721]  [<ffffffff8151fdfa>] nmi+0x1a/0x20
<Oct/17 03:55 pm>[6853662.763747]  [<ffffffff812c08a0>] ? delay_tsc+0x16/0x59
<Oct/17 03:55 pm>[6853662.769476]  <<EOE>>  [<ffffffff81019da9>] ? arch_trigger_all_cpu_backtrace+0x51/0x5d
<Oct/17 03:55 pm>[6853662.778122]  [<ffffffff812c08fe>] __delay+0xa/0xc
<Oct/17 03:55 pm>[6853662.783317]  [<ffffffff812c4e65>] do_raw_spin_lock+0xde/0x147
<Oct/17 03:55 pm>[6853662.789570]  [<ffffffff8151f0dc>] _raw_spin_lock_nested+0x32/0x36
<Oct/17 03:55 pm>[6853662.796193]  [<ffffffff81035934>] ? double_rq_lock+0x4a/0x76
<Oct/17 03:55 pm>[6853662.802397]  [<ffffffff81035934>] double_rq_lock+0x4a/0x76
<Oct/17 03:55 pm>[6853662.808412]  [<ffffffff81037672>] load_balance+0xbb6/0xf63
<Oct/17 03:55 pm>[6853662.814428]  [<ffffffff810b2173>] ? __pagevec_free+0x70/0x85
<Oct/17 03:55 pm>[6853662.820606]  [<ffffffff8151cc0e>] schedule+0x25a/0x86a
<Oct/17 03:55 pm>[6853662.826256]  [<ffffffff8144b38b>] lock_sock_nested+0x82/0xe2
<Oct/17 03:55 pm>[6853662.832446]  [<ffffffff81054b6c>] ? autoremove_wake_function+0x0/0x38

<Oct/17 03:55 pm>[6853662.839433]  [<ffffffff81452976>] skb_free_datagram_locked+0x14/0x2c
<Oct/17 03:55 pm>[6853662.846315]  [<ffffffff8149bbfb>] udp_recvmsg+0x1a5/0x209
<Oct/17 03:55 pm>[6853662.852240]  [<ffffffff8144a75d>] sock_common_recvmsg+0x32/0x47
<Oct/17 03:55 pm>[6853662.858698]  [<ffffffff81448584>] sock_recvmsg+0xc1/0xda
<Oct/17 03:55 pm>[6853662.864513]  [<ffffffff810480a2>] ? lock_timer_base+0x26/0x4a
<Oct/17 03:55 pm>[6853662.870806]  [<ffffffff810480a2>] ? lock_timer_base+0x26/0x4a
<Oct/17 03:55 pm>[6853662.877088]  [<ffffffff81048174>] ? try_to_del_timer_sync+0xae/0xbb
<Oct/17 03:55 pm>[6853662.883912]  [<ffffffff814485d2>] kernel_recvmsg+0x35/0x41
<Oct/17 03:55 pm>[6853662.889907]  [<ffffffff814f1954>] svc_udp_recvfrom+0xa1/0x2f9
<Oct/17 03:55 pm>[6853662.896217]  [<ffffffff814fc980>] svc_recv+0x758/0x7f5
<Oct/17 03:55 pm>[6853662.901866]  [<ffffffff814fc1d9>] ? svc_send+0x85/0xa1
<Oct/17 03:55 pm>[6853662.907543]  [<ffffffff8103688c>] ? default_wake_function+0x0/0xf
<Oct/17 03:55 pm>[6853662.914178]  [<ffffffff811f0ccf>] ? nfsd+0x0/0x139
<Oct/17 03:55 pm>[6853662.919475]  [<ffffffff811f0d78>] nfsd+0xa9/0x139
<Oct/17 03:55 pm>[6853662.924678]  [<ffffffff810546e2>] kthread+0x7d/0x85
<Oct/17 03:55 pm>[6853662.930068]  [<ffffffff81003794>] kernel_thread_helper+0x4/0x10
<Oct/17 03:55 pm>[6853662.936538]  [<ffffffff81054665>] ? kthread+0x0/0x85
<Oct/17 03:55 pm>[6853662.942071]  [<ffffffff81003790>] ? kernel_thread_helper+0x0/0x10
<Oct/17 03:55 pm>[6853662.948679] NMI backtrace for cpu 11
<Oct/17 03:55 pm>[6853662.952710] CPU 11 
<Oct/17 03:55 pm>[6853662.954906] Modules linked in: w83793 hwmon_vid autofs4 i2c_i801 i2c_core iTCO_wdt e1000e ioatdma dca rtc_cmos tg3 libphy e1000 fuse dm_snapshot dm_mirror dm_region_hash dm_log scsi_wait_scan qla2xxx megaraid_sas megaraid_mbox megaraid_mm aacraid 3w_9xxx 3w_xxxx arcmsr aic7xxx aic79xx sr_mod sg ata_piix ahci sata_sis sata_nv sata_via pata_sl82c105 pata_sis
<Oct/17 03:55 pm>[6853662.990658] 

<Oct/17 03:55 pm>[6853662.992571] Pid: 2921, comm: pclient Tainted: G      D    2.6.34.4 #3 S7010/empty
<Oct/17 03:55 pm>[6853663.000773] RIP: 0010:[<ffffffff812c08af>]  [<ffffffff812c08af>] delay_tsc+0x25/0x59
<Oct/17 03:55 pm>[6853663.009330] RSP: 0000:ffff8806552a3dd0  EFLAGS: 00000082
<Oct/17 03:55 pm>[6853663.015136] RAX: 00000000c74054ec RBX: 000000004aa9f22b RCX: 00000000c74054d0
<Oct/17 03:55 pm>[6853663.023002] RDX: 000000000040ee8b RSI: 000000000000000b RDI: 0000000000000001
<Oct/17 03:55 pm>[6853663.030889] RBP: ffff8806552a3dd8 R08: 000000000000000b R09: 000000000000000b
<Oct/17 03:55 pm>[6853663.038775] R10: 0000000000000000 R11: ffff8806552a3c08 R12: ffff8806552b3c00
<Oct/17 03:55 pm>[6853663.046637] R13: 0000000000000000 R14: ffff8803ce096120 R15: ffff8803ce0965a8
<Oct/17 03:55 pm>[6853663.054488] FS:  00007fe0e24916f0(0000) GS:ffff8806552a0000(0000) knlGS:0000000000000000
<Oct/17 03:55 pm>[6853663.063349] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<Oct/17 03:55 pm>[6853663.069634] CR2: 000000000186bb18 CR3: 000000062112d000 CR4: 00000000000006e0
<Oct/17 03:55 pm>[6853663.077491] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
<Oct/17 03:55 pm>[6853663.085315] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
<Oct/17 03:55 pm>[6853663.093179] Process pclient (pid: 2921, threadinfo ffff88036c100000, task ffff8803ce096120)
<Oct/17 03:55 pm>[6853663.102282] Stack:
<Oct/17 03:55 pm>[6853663.104723]  ffffffff81019da9 ffff8806552a3de8 ffffffff812c08fe ffff8806552a3e28
<Oct/17 03:55 pm>[6853663.112649] <0> ffffffff812c4e65 000000009ef2e6b4 ffff8806552b3c00 ffff880023799030
<Oct/17 03:55 pm>[6853663.121327] <0> 0000000000013c00 ffff8806552a3e98 000000000000000b ffff8806552a3e48
<Oct/17 03:55 pm>[6853663.130396] Call Trace:
<Oct/17 03:55 pm>[6853663.133273]  <IRQ> 
<Oct/17 03:55 pm>[6853663.135848]  [<ffffffff81019da9>] ? arch_trigger_all_cpu_backtrace+0x51/0x5d
<Oct/17 03:55 pm>[6853663.143639]  [<ffffffff812c08fe>] __delay+0xa/0xc

<Oct/17 03:55 pm>[6853663.148848]  [<ffffffff812c4e65>] do_raw_spin_lock+0xde/0x147
<Oct/17 03:55 pm>[6853663.155109]  [<ffffffff8151f193>] _raw_spin_lock+0x34/0x39
<Oct/17 03:55 pm>[6853663.161123]  [<ffffffff8102fb6b>] ? task_rq_lock+0x7e/0x9a
<Oct/17 03:55 pm>[6853663.167129]  [<ffffffff8102fb6b>] task_rq_lock+0x7e/0x9a
<Oct/17 03:55 pm>[6853663.172987]  [<ffffffff81057842>] ? hrtimer_wakeup+0x0/0x21
<Oct/17 03:55 pm>[6853663.179106]  [<ffffffff81036578>] try_to_wake_up+0x2e/0x342
<Oct/17 03:55 pm>[6853663.185183]  [<ffffffff81057842>] ? hrtimer_wakeup+0x0/0x21
<Oct/17 03:55 pm>[6853663.191268]  [<ffffffff810368b8>] wake_up_process+0x10/0x12
<Oct/17 03:55 pm>[6853663.197333]  [<ffffffff8105785f>] hrtimer_wakeup+0x1d/0x21
<Oct/17 03:55 pm>[6853663.203348]  [<ffffffff81057933>] __run_hrtimer+0xd0/0x12d
<Oct/17 03:55 pm>[6853663.209372]  [<ffffffff81057b81>] hrtimer_interrupt+0xb5/0<Oct/17 03:55 pm>
                 <Oct/17 03:55 pm>[6853663.215734]  [<ffffffff810197a5>] smp_apic_timer_interrupt+0x82/0x95
<Oct/17 03:55 pm>[6853663.222636]  [<ffffffff81003353>] apic_timer_interrupt+0x13/0x20
<Oct/17 03:55 pm>[6853663.229184]  <EOI> 
<Oct/17 03:55 pm>[6853663.231373] Code: fb 48 ff c8 c9 c3 55 48 89 e5 48 83 ec 08 65 8b 34 25 f0 d1 00 00 66 66 90 0f ae e8 0f 31 89 c1 41 89 f1 66 66 90 0f ae e8 0f 31 <89> c6 48 89 f0 48 29 c8 48 39 f8 73 25 f3 90 65 44 8b 04 25 f0 
<Oct/17 03:55 pm>[6853663.255052] Call Trace:
<Oct/17 03:55 pm>[6853663.257957]  <IRQ>  [<ffffffff81019da9>] ? arch_trigger_all_cpu_backtrace+0x51/0x5d
<Oct/17 03:55 pm>[6853663.266410]  [<ffffffff812c08fe>] __delay+0xa/0xc
<Oct/17 03:55 pm>[6853663.271625]  [<ffffffff812c4e65>] do_raw_spin_lock+0xde/0x147
<Oct/17 03:55 pm>[6853663.277886]  [<ffffffff8151f193>] _raw_spin_lock+0x34/0x39
<Oct/17 03:55 pm>[6853663.283883]  [<ffffffff8102fb6b>] ? task_rq_lock+0x7e/0x9a
<Oct/17 03:55 pm>[6853663.289923]  [<ffffffff8102fb6b>] task_rq_lock+0x7e/0x9a

<Oct/17 03:55 pm>[6853663.295720]  [<ffffffff81057842>] ? hrtimer_wakeup+0x0/0x21
<Oct/17 03:55 pm>[6853663.301839]  [<ffffffff81036578>] try_to_wake_up+0x2e/0x342
<Oct/17 03:55 pm>[6853663.307933]  [<ffffffff81057842>] ? hrtimer_wakeup+0x0/0x21
<Oct/17 03:55 pm>[6853663.314062]  [<ffffffff810368b8>] wake_up_process+0x10/0x12
<Oct/17 03:55 pm>[6853663.320153]  [<ffffffff8105785f>] hrtimer_wakeup+0x1d/0x21
<Oct/17 03:55 pm>[6853663.326159]  [<ffffffff81057933>] __run_hrtimer+0xd0/0x12d
<Oct/17 03:55 pm>[6853663.332166]  [<ffffffff81057b81>] hrtimer_interrupt+0xb5/0x19f
<Oct/17 03:55 pm>[6853663.338545]  [<ffffffff810197a5>] smp_apic_timer_interrupt+0x82/0x95
<Oct/17 03:55 pm>[6853663.345447]  [<ffffffff81003353>] apic_timer_interrupt+0x13/0x20
<Oct/17 03:55 pm>[6853663.351978]  <EOI> Pid: 2921, comm: pclient Tainted: G      D    2.6.34.4 #3
<Oct/17 03:55 pm>[6853663.359785] Call Trace:
<Oct/17 03:55 pm>[6853663.362688]  <NMI>  [<ffffffff81009b2a>] ? show_regs+0x26/0x2a
<Oct/17 03:55 pm>[6853663.369080]  [<ffffffff81520db8>] nmi_watchdog_tick+0xc3/0x1a5
<Oct/17 03:55 pm>[6853663.375441]  [<ffffffff81520360>] do_nmi+0xb2/0x259
<Oct/17 03:55 pm>[6853663.380838]  [<ffffffff8151fdfa>] nmi+0x1a/0x20
<Oct/17 03:55 pm>[6853663.385880]  [<ffffffff812c08af>] ? delay_tsc+0x25/0x59
<Oct/17 03:55 pm>[6853663.391620]  <<EOE>>  <IRQ>  [<ffffffff81019da9>] ? arch_trigger_all_cpu_backtrace+0x51/0x5d
<Oct/17 03:55 pm>[6853663.400956]  [<ffffffff812c08fe>] __delay+0xa/0xc
<Oct/17 03:55 pm>[6853663.406153]  [<ffffffff812c4e65>] do_raw_spin_lock+0xde/0x147
<Oct/17 03:55 pm>[6853663.412415]  [<ffffffff8151f193>] _raw_spin_lock+0x34/0x39
<Oct/17 03:55 pm>[6853663.418428]  [<ffffffff8102fb6b>] ? task_rq_lock+0x7e/0x9a
<Oct/17 03:55 pm>[6853663.424426]  [<ffffffff8102fb6b>] task_rq_lock+0x7e/0x9a

<Oct/17 03:55 pm>[6853663.430250]  [<ffffffff81057842>] ? hrtimer_wakeup+0x0/0x21
<Oct/17 03:55 pm>[6853663.436352]  [<ffffffff81036578>] try_to_wake_up+0x2e/0x342
<Oct/17 03:55 pm>[6853663.442461]  [<ffffffff81057842>] ? hrtimer_wakeup+0x0/0x21
<Oct/17 03:55 pm>[6853663.448562]  [<ffffffff810368b8>] wake_up_process+0x10/0x12
<Oct/17 03:55 pm>[6853663.454664]  [<ffffffff8105785f>] hrtimer_wakeup+0x1d/0x21
<Oct/17 03:55 pm>[6853663.460688]  [<ffffffff81057933>] __run_hrtimer+0xd0/0x12d
<Oct/17 03:55 pm>[6853663.466710]  [<ffffffff81057b81>] hrtimer_interrupt+0xb5/0x19f
<Oct/17 03:55 pm>[6853663.473057]  [<ffffffff810197a5>] smp_apic_timer_interrupt+0x82/0x95
<Oct/17 03:55 pm>[6853663.479960]  [<ffffffff81003353>] apic_timer_interrupt+0x13/0x20
<Oct/17 03:55 pm>[6853663.486507]  <EOI> 
<Oct/17 03:55 pm>[6853663.488714] NMI backtrace for cpu 10
<Oct/17 03:55 pm>[6853663.493165] CPU 10 
<Oct/17 03:55 pm>[6853663.495378] Modules linked in: w83793 hwmon_vid autofs4 i2c_i801 i2c_core iTCO_wdt e1000e ioatdma dca rtc_cmos tg3 libphy e1000 fuse dm_snapshot dm_mirror dm_region_hash dm_log scsi_wait_scan qla2xxx megaraid_sas megaraid_mbox megaraid_mm aacraid 3w_9xxx 3w_xxxx arcmsr aic7xxx aic79xx sr_mod sg ata_piix ahci sata_sis sata_nv sata_via pata_sl82c105 pata_sis
<Oct/17 03:55 pm>[6853663.531191] 
<Oct/17 03:55 pm>[6853663.533121] Pid: 19726, comm: ppImage Tainted: G      D    2.6.34.4 #3 S7010/empty
<Oct/17 03:55 pm>[6853663.541437] RIP: 0010:[<ffffffff812c08a2>]  [<ffffffff812c08a2>] delay_tsc+0x18/0x59
<Oct/17 03:55 pm>[6853663.549958] RSP: 0000:ffff88002c88b7a0  EFLAGS: 00000086
<Oct/17 03:55 pm>[6853663.555807] RAX: 00000000c73c4240 RBX: 000000004aaa3b0b RCX: 00000000c73c41f8
<Oct/17 03:55 pm>[6853663.563691] RDX: 000000000040ee8b RSI: 000000000000000a RDI: 0000000000000001
<Oct/17 03:55 pm>[6853663.571570] RBP: ffff88002c88b7a8 R08: 000000000000000a R09: 000000000000000a
<Oct/17 03:55 pm>[6853663.579465] R10: 0000000000000000 R11: ffff88002c88b5d8 R12: ffff8806552b3c00
<Oct/17 03:55 pm>[6853663.587344] R13: 0000000000000000 R14: ffff880611553090 R15: ffff880611553518

<Oct/17 03:55 pm>[6853663.599736] FS:  000000004243e950(0063) GS:ffff880655280000(0000) knlGS:0000000000000000
<Oct/17 03:55 pm>[6853663.608579] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<Oct/17 03:55 pm>[6853663.614848] CR2: 00007fa3e03daab0 CR3: 00000003e2c88000 CR4: 00000000000006e0
<Oct/17 03:55 pm>[6853663.622703] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
<Oct/17 03:55 pm>[6853663.630591] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
<Oct/17 03:55 pm>[6853663.638462] Process ppImage (pid: 19726, threadinfo ffff88002c88a000, task ffff880611553090)
<Oct/17 03:55 pm>[6853663.647644] Stack:
<Oct/17 03:55 pm>[6853663.650083]  ffffffff81019da9 ffff88002c88b7b8 ffffffff812c08fe ffff88002c88b7f8
<Oct/17 03:55 pm>[6853663.658037] <0> ffffffff812c4e65 000000009ef2e6b4 ffff8806552b3c00 ffff8806552b3c00
<Oct/17 03:55 pm>[6853663.666711] <0> ffff88065528e3f0 0000000000000400 0000000000000800 ffff88002c88b818
<Oct/17 03:55 pm>[6853663.675790] Call Trace:
<Oct/17 03:55 pm>[6853663.678671]  [<ffffffff81019da9>] ? arch_trigger_all_cpu_backtrace+0x51/0x5d
<Oct/17 03:55 pm>[6853663.686458]  [<ffffffff812c08fe>] __delay+0xa/0xc
<Oct/17 03:55 pm>[6853663.691686]  [<ffffffff812c4e65>] do_raw_spin_lock+0xde/0x147
<Oct/17 03:55 pm>[6853663.697939]  [<ffffffff8151f0dc>] _raw_spin_lock_nested+0x32/0x36
<Oct/17 03:55 pm>[6853663.704562]  [<ffffffff81035934>] ? double_rq_lock+0x4a/0x76
<Oct/17 03:55 pm>[6853663.710783]  [<ffffffff81035934>] double_rq_lock+0x4a/0x76
<Oct/17 03:55 pm>[6853663.716797]  [<ffffffff81037672>] load_balance+0xbb6/0xf63
<Oct/17 03:55 pm>[6853663.722819]  [<ffffffff810b3cfb>] ? get_dirty_limits+0x22/0x258
<Oct/17 03:55 pm>[6853663.729253]  [<ffffffff8151cc0e>] schedule+0x25a/0x86a
<Oct/17 03:55 pm>[6853663.734919]  [<ffffffff810480a2>] ? lock_timer_base+0x26/0x4a
<Oct/17 03:55 pm>[6853663.741204]  [<ffffffff8151d559>] schedule_timeout+0x222/0x24c

<Oct/17 03:55 pm>[6853663.747600]  [<ffffffff810482a5>] ? process_timeout+0x0/0xb
<Oct/17 03:55 pm>[6853663.753683]  [<ffffffff8151c6c5>] io_schedule_timeout+0x45/0x5e
<Oct/17 03:55 pm>[6853663.760115]  [<ffffffff810c0abe>] congestion_wait+0x6b/0x87
<Oct/17 03:55 pm>[6853663.766223]  [<ffffffff81054b6c>] ? autoremove_wake_function+0x0/0x38
<Oct/17 03:55 pm>[6853663.773188]  [<ffffffff810b3279>] __alloc_pages_nodemask+0x65a/0x671
<Oct/17 03:55 pm>[6853663.780079]  [<ffffffff810da90f>] alloc_page_vma+0x148/0x14d
<Oct/17 03:55 pm>[6853663.786263]  [<ffffffff810c4d2a>] handle_mm_fault+0x294/0x<Oct/17 03:55 pm>
                 <Oct/17 03:55 pm>[6853663.792505]  [<ffffffff815227e9>] do_page_fault+0x351/0x375
<Oct/17 03:55 pm>[6853663.798588]  [<ffffffff8151e35f>] ? do_nanosleep+0x7b/0xb3
<Oct/17 03:55 pm>[6853663.804578]  [<ffffffff810348d8>] ? finish_task_switch+0x69/0xdd
<Oct/17 03:55 pm>[6853663.811096]  [<ffffffff8103486f>] ? finish_task_switch+0x0/0xdd
<Oct/17 03:55 pm>[6853663.817543]  [<ffffffff81058477>] ? hrtimer_nanosleep+0x97/0x105
<Oct/17 03:55 pm>[6853663.824104]  [<ffffffff8106472c>] ? lockdep_sys_exit+0x20/0x75
<Oct/17 03:55 pm>[6853663.830492]  [<ffffffff8151ef6f>] ? lockdep_sys_exit_thunk+0x35/0x67
<Oct/17 03:55 pm>[6853663.837367]  [<ffffffff810c4dda>] ? handle_mm_fault+0x344/0x73e
<Oct/17 03:55 pm>[6853663.843820]  [<ffffffff8151fb5f>] page_fault+0x1f/0x30
<Oct/17 03:55 pm>[6853663.849469] Code: 2e 0f 1f 84 00 00 00 00 00 48 ff c8 75 fb 48 ff c8 c9 c3 55 48 89 e5 48 83 ec 08 65 8b 34 25 f0 d1 00 00 66 66 90 0f ae e8 0f 31 <89> c1 41 89 f1 66 66 90 0f ae e8 0f 31 89 c6 48 89 f0 48 29 c8 
<Oct/17 03:55 pm>[6853663.872747] Call Trace:
<Oct/17 03:55 pm>[6853663.875643]  [<ffffffff81019da9>] ? arch_trigger_all_cpu_backtrace+0x51/0x5d
<Oct/17 03:55 pm>[6853663.883406]  [<ffffffff812c08fe>] __delay+0xa/0xc
<Oct/17 03:55 pm>[6853663.888600]  [<ffffffff812c4e65>] do_raw_spin_lock+0xde/0x147
<Oct/17 03:55 pm>[6853663.894878]  [<ffffffff8151f0dc>] _raw_spin_lock_nested+0x32/0x36

<Oct/17 03:55 pm>[6853663.901510]  [<ffffffff81035934>] ? double_rq_lock+0x4a/0x76
<Oct/17 03:55 pm>[6853663.907704]  [<ffffffff81035934>] double_rq_lock+0x4a/0x76
<Oct/17 03:55 pm>[6853663.913720]  [<ffffffff81037672>] load_balance+0xbb6/0xf63
<Oct/17 03:55 pm>[6853663.919734]  [<ffffffff810b3cfb>] ? get_dirty_limits+0x22/0x258
<Oct/17 03:55 pm>[6853663.926183]  [<ffffffff8151cc0e>] schedule+0x25a/0x86a
<Oct/17 03:55 pm>[6853663.931840]  [<ffffffff810480a2>] ? lock_timer_base+0x26/0x4a
<Oct/17 03:55 pm>[6853663.938102]  [<ffffffff8151d559>] schedule_timeout+0x222/0<Oct/17 03:55 pm>
                 <Oct/17 03:55 pm>[6853663.944472]  [<ffffffff810482a5>] ? process_timeout+0x0/0xb
<Oct/17 03:55 pm>[6853663.950545]  [<ffffffff8151c6c5>] io_schedule_timeout+0x45/0x5e
<Oct/17 03:55 pm>[6853663.957021]  [<ffffffff810c0abe>] congestion_wait+0x6b/0x87
<Oct/17 03:55 pm>[6853663.963131]  [<ffffffff81054b6c>] ? autoremove_wake_function+0x0/0x38
<Oct/17 03:55 pm>[6853663.970137]  [<ffffffff810b3279>] __alloc_pages_nodemask+0x65a/0x671
<Oct/17 03:55 pm>[6853663.977018]  [<ffffffff810da90f>] alloc_page_vma+0x148/0x14d
<Oct/17 03:55 pm>[6853663.983177]  [<ffffffff810c4d2a>] handle_mm_fault+0x294/0x73e
<Oct/17 03:55 pm>[6853663.989470]  [<ffffffff815227e9>] do_page_fault+0x351/0x375
<Oct/17 03:55 pm>[6853663.995555]  [<ffffffff8151e35f>] ? do_nanosleep+0x7b/0xb3
<Oct/17 03:55 pm>[6853664.001586]  [<ffffffff810348d8>] ? finish_task_switch+0x69/0xdd
<Oct/17 03:55 pm>[6853664.008130]  [<ffffffff8103486f>] ? finish_task_switch+0x0/0xdd
<Oct/17 03:55 pm>[6853664.014623]  [<ffffffff81058477>] ? hrtimer_nanosleep+0x97/0x105
<Oct/17 03:55 pm>[6853664.021175]  [<ffffffff8106472c>] ? lockdep_sys_exit+0x20/0x75
<Oct/17 03:55 pm>[6853664.027553]  [<ffffffff8151ef6f>] ? lockdep_sys_exit_thunk+0x35/0x67
<Oct/17 03:55 pm>[6853664.034454]  [<ffffffff810c4dda>] ? handle_mm_fault+0x344/0x73e

<Oct/17 03:55 pm>[6853664.040900]  [<ffffffff8151fb5f>] page_fault+0x1f/0x30
<Oct/17 03:55 pm>[6853664.046556] Pid: 19726, comm: ppImage Tainted: G      D    2.6.34.4 #3
<Oct/17 03:55 pm>[6853664.053633] Call Trace:
                 <Oct/17 03:55 pm>[6853664.056549]  <NMI>  [<ffffffff81009b2a>] ? show_regs+0x26/0x2a
<Oct/17 03:55 pm>[6853664.062985]  [<ffffffff81520db8>] nmi_watchdog_tick+0xc3/0x1a5
<Oct/17 03:55 pm>[6853664.069319]  [<ffffffff81520360>] do_nmi+0xb2/0x259
<Oct/17 03:55 pm>[6853664.074707]  [<ffffffff8151fdfa>] nmi+0x1a/0x20
<Oct/17 03:55 pm>[6853664.079741]  [<ffffffff812c08a2>] ? delay_tsc+0x18/0x59
<Oct/17 03:55 pm>[6853664.085487]  <<EOE>>  [<ffffffff81019da9>] ? arch_trigger_all_cpu_backtrace+0x51/0x5d
<Oct/17 03:55 pm>[6853664.094123]  [<ffffffff812c08fe>] __delay+0xa/0xc
<Oct/17 03:55 pm>[6853664.099328]  [<ffffffff812c4e65>] do_raw_spin_lock+0xde/0x147
<Oct/17 03:55 pm>[6853664.105592]  [<ffffffff8151f0dc>] _raw_spin_lock_nested+0x32/0x36
<Oct/17 03:55 pm>[6853664.112212]  [<ffffffff81035934>] ? double_rq_lock+0x4a/0x76
<Oct/17 03:55 pm>[6853664.118373]  [<ffffffff81035934>] double_rq_lock+0x4a/0x76
<Oct/17 03:55 pm>[6853664.124387]  [<ffffffff81037672>] load_balance+0xbb6/0xf63
<Oct/17 03:55 pm>[6853664.130420]  [<ffffffff810b3cfb>] ? get_dirty_limits+0x22/0x258
<Oct/17 03:55 pm>[6853664.136880]  [<ffffffff8151cc0e>] schedule+0x25a/0x86a
<Oct/17 03:55 pm>[6853664.142545]  [<ffffffff810480a2>] ? lock_timer_base+0x26/0x4a
<Oct/17 03:55 pm>[6853664.148837]  [<ffffffff8151d559>] schedule_timeout+0x222/0x24c
<Oct/17 03:55 pm>[6853664.155198]  [<ffffffff810482a5>] ? process_timeout+0x0/0xb
<Oct/17 03:55 pm>[6853664.161320]  [<ffffffff8151c6c5>] io_schedule_timeout+0x45/0x5e
<Oct/17 03:55 pm>[6853664.167793]  [<ffffffff810c0abe>] congestion_wait+0x6b/0x87

<Oct/17 03:55 pm>[6853664.173895]  [<ffffffff81054b6c>] ? autoremove_wake_function+0x0/0x38
<Oct/17 03:55 pm>[6853664.180908]  [<ffffffff810b3279>] __alloc_pages_nodemask+0x65a/0x671
<Oct/17 03:55 pm>[6853664.187790]  [<ffffffff810da90f>] alloc_page_vma+0x148/0x14d
<Oct/17 03:55 pm>[6853664.193958]  [<ffffffff810c4d2a>] handle_mm_fault+0x294/0x73e
<Oct/17 03:55 pm>[6853664.200242]  [<ffffffff815227e9>] do_page_fault+0x351/0x375
<Oct/17 03:55 pm>[6853664.206360]  [<ffffffff8151e35f>] ? do_nanosleep+0x7b/0xb3
<Oct/17 03:55 pm>[6853664.212366]  [<ffffffff810348d8>] ? finish_task_switch+0x69/0xdd
<Oct/17 03:55 pm>[6853664.218877]  [<ffffffff8103486f>] ? finish_task_switch+0x0/0xdd
<Oct/17 03:55 pm>[6853664.225333]  [<ffffffff81058477>] ? hrtimer_nanosleep+0x97/0x105
<Oct/17 03:55 pm>[6853664.231885]  [<ffffffff8106472c>] ? lockdep_sys_exit+0x20/0x75
<Oct/17 03:55 pm>[6853664.238222]  [<ffffffff8151ef6f>] ? lockdep_sys_exit_thunk+0x35/0x67
<Oct/17 03:55 pm>[6853664.245120]  [<ffffffff810c4dda>] ? handle_mm_fault+0x344/0x73e
<Oct/17 03:55 pm>[6853664.251569]  [<ffffffff8151fb5f>] page_fault+0x1f/0x30
<Oct/17 03:55 pm>[6853664.257242] NMI backtrace for cpu 9
<Oct/17 03:55 pm>[6853664.261224] CPU 9 
<Oct/17 03:55 pm>[6853664.263307] Modules linked in: w83793 hwmon_vid autofs4 i2c_i801 i2c_core iTCO_wdt e1000e ioatdma dca rtc_cmos tg3 libphy e1000 fuse dm_snapshot dm_mirror dm_region_hash dm_log scsi_wait_scan qla2xxx megaraid_sas megaraid_mbox megaraid_mm aacraid 3w_9xxx 3w_xxxx arcmsr aic7xxx aic79xx sr_mod sg ata_piix ahci sata_sis sata_nv sata_via pata_sl82c105 pata_sis
<Oct/17 03:55 pm>[6853664.299190] 
<Oct/17 03:55 pm>[6853664.301120] Pid: 19611, comm: ppImage Tainted: G      D    2.6.34.4 #3 S7010/empty
<Oct/17 03:55 pm>[6853664.309417] RIP: 0010:[<ffffffff812c08a2>]  [<ffffffff812c08a2>] delay_tsc+0x18/0x59
<Oct/17 03:55 pm>[6853664.317931] RSP: 0000:ffff88050e4897a0  EFLAGS: 00000086
<Oct/17 03:55 pm>[6853664.323761] RAX: 00000000c7382f90 RBX: 000000004aaa1307 RCX: 00000000c7382f48
<Oct/17 03:55 pm>[6853664.331620] RDX: 000000000040ee8b RSI: 0000000000000009 RDI: 0000000000000001

<Oct/17 03:55 pm>[6853664.339500] RBP: ffff88050e4897a8 R08: 0000000000000009 R09: 0000000000000009
<Oct/17 03:55 pm>[6853664.347375] R10: 0000000000000000 R11: ffff88050e4895d8 R12: ffff8806552b3c00
<Oct/17 03:55 pm>[6853664.355254] R13: 0000000000000000 R14: ffff8805c9192060 R15: ffff8805c91924e8
<Oct/17 03:55 pm>[6853664.363142] FS:  00007f02d54d06f0(0000) GS:ffff880655260000(0000) knlGS:0000000000000000
<Oct/17 03:55 pm>[6853664.371995] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<Oct/17 03:55 pm>[6853664.378296] CR2: 00007f02c7852068 CR3: 00000001618d6000 CR4: 00000000000006e0
<Oct/17 03:55 pm>[6853664.386146] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
<Oct/17 03:55 pm>[6853664.394005] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
<Oct/17 03:55 pm>[6853664.401909] Process ppImage (pid: 19611, threadinfo ffff88050e488000, task ffff8805c9192060)
<Oct/17 03:55 pm>[6853664.411110] Stack:
<Oct/17 03:55 pm>[6853664.413558]  ffffffff81019da9 ffff88050e4897b8 ffffffff812c08fe ffff88050e4897f8
<Oct/17 03:55 pm>[6853664.421460] <0> ffffffff812c4e65 000000009ef2e6b4 ffff8806552b3c00 ffff8806552b3c00
<Oct/17 03:55 pm>[6853664.430118] <0> ffff88065526e3f0 0000000000000400 0000000000000800 ffff88050e489818
<Oct/17 03:55 pm>[6853664.439196] Call Trace:
<Oct/17 03:55 pm>[6853664.442073]  [<ffffffff81019da9>] ? arch_trigger_all_cpu_backtrace+0x51/0x5d
<Oct/17 03:55 pm>[6853664.449871]  [<ffffffff812c08fe>] __delay+0xa/0xc
<Oct/17 03:55 pm>[6853664.455063]  [<ffffffff812c4e65>] do_raw_spin_lock+0xde/0x147
<Oct/17 03:55 pm>[6853664.461318]  [<ffffffff8151f0dc>] _raw_spin_lock_nested+0x32/0x36
<Oct/17 03:55 pm>[6853664.467941]  [<ffffffff81035934>] ? double_rq_lock+0x4a/0x76
<Oct/17 03:55 pm>[6853664.474092]  [<ffffffff81035934>] double_rq_lock+0x4a/0x76
<Oct/17 03:55 pm>[6853664.480089]  [<ffffffff81037672>] load_balance+0xbb6/0xf63
<Oct/17 03:55 pm>[6853664.486105]  [<ffffffff810b3cfb>] ? get_dirty_limits+0x22/0x258

<Oct/17 03:55 pm>[6853664.492572]  [<ffffffff8151cc0e>] schedule+0x25a/0x86a
<Oct/17 03:55 pm>[6853664.498209]  [<ffffffff810480a2>] ? lock_timer_base+0x26/0x4a
<Oct/17 03:55 pm>[6853664.504471]  [<ffffffff8151d559>] schedule_timeout+0x222/0x24c
<Oct/17 03:55 pm>[6853664.510868]  [<ffffffff810482a5>] ? process_timeout+0x0/0xb
<Oct/17 03:55 pm>[6853664.516932]  [<ffffffff8151c6c5>] io_schedule_timeout+0x45/0x5e
<Oct/17 03:55 pm>[6853664.523375]  [<ffffffff810c0abe>] congestion_wait+0x6b/0x87
<Oct/17 03:55 pm>[6853664.529459]  [<ffffffff81054b6c>] ? autoremove_wake_function+0x0/0x38
<Oct/17 03:55 pm>[6853664.536454]  [<ffffffff810b3279>] __alloc_pages_nodemask+0x65a/0x671
<Oct/17 03:55 pm>[6853664.543371]  [<ffffffff810da90f>] alloc_page_vma+0x148/0x14d
<Oct/17 03:55 pm>[6853664.549537]  [<ffffffff810c4d2a>] handle_mm_fault+0x294/0x73e
<Oct/17 03:55 pm>[6853664.555796]  [<ffffffff815227e9>] do_page_fault+0x351/0x375
<Oct/17 03:55 pm>[6853664.561896]  [<ffffffff810cab7d>] ? sys_mmap_pgoff+0x107/0x126
<Oct/17 03:55 pm>[6853664.568259]  [<ffffffff8106472c>] ? lockdep_sys_exit+0x20/0x75
<Oct/17 03:55 pm>[6853664.574604]  [<ffffffff8151ef6f>] ? lockdep_sys_exit_thunk+0x35/0x67
<Oct/17 03:55 pm>[6853664.586054]  [<ffffffff810c4dda>] ? handle_mm_fault+0x344/0x73e
<Oct/17 03:55 pm>[6853664.592520]  [<ffffffff8151fb5f>] page_fault+0x1f/0x30
<Oct/17 03:55 pm>[6853664.598173] Code: 2e 0f 1f 84 00 00 00 00 00 48 ff c8 75 fb 48 ff c8 c9 c3 55 48 89 e5 48 83 ec 08 65 8b 34 25 f0 d1 00 00 66 66 90 0f ae e8 0f 31 <89> c1 41 89 f1 66 66 90 0f ae e8 0f 31 89 c6 48 89 f0 48 29 c8 
<Oct/17 03:55 pm>[6853664.621402] Call Trace:
<Oct/17 03:55 pm>[6853664.624299]  [<ffffffff81019da9>] ? arch_trigger_all_cpu_backtrace+0x51/0x5d
<Oct/17 03:55 pm>[6853664.632095]  [<ffffffff812c08fe>] __delay+0xa/0xc
<Oct/17 03:55 pm>[6853664.637314]  [<ffffffff812c4e65>] do_raw_spin_lock+0xde/0x147
<Oct/17 03:55 pm>[6853664.643603]  [<ffffffff8151f0dc>] _raw_spin_lock_nested+0x32/0x36

<Oct/17 03:55 pm>[6853664.650216]  [<ffffffff81035934>] ? double_rq_lock+0x4a/0x76
<Oct/17 03:55 pm>[6853664.656375]  [<ffffffff81035934>] double_rq_lock+0x4a/0x76
<Oct/17 03:55 pm>[6853664.662424]  [<ffffffff81037672>] load_balance+0xbb6/0xf63
<Oct/17 03:55 pm>[6853664.668430]  [<ffffffff810b3cfb>] ? get_dirty_limits+0x22/0x258
<Oct/17 03:55 pm>[6853664.674857]  [<ffffffff8151cc0e>] schedule+0x25a/0x86a
<Oct/17 03:55 pm>[6853664.680468]  [<ffffffff810480a2>] ? lock_timer_base+0x26/0x4a
<Oct/17 03:55 pm>[6853664.686753]  [<ffffffff8151d559>] schedule_timeout+0x222/0x24c
<Oct/17 03:55 pm>[6853664.693116]  [<ffffffff810482a5>] ? process_timeout+0x0/0xb
<Oct/17 03:55 pm>[6853664.699252]  [<ffffffff8151c6c5>] io_schedule_timeout+0x45/0x5e
<Oct/17 03:55 pm>[6853664.705726]  [<ffffffff810c0abe>] congestion_wait+0x6b/0x87
<Oct/17 03:55 pm>[6853664.711819]  [<ffffffff81054b6c>] ? autoremove_wake_function+0x0/0x38
<Oct/17 03:55 pm>[6853664.718816]  [<ffffffff810b3279>] __alloc_pages_nodemask+0x65a/0x671
<Oct/17 03:55 pm>[6853664.725715]  [<ffffffff810da90f>] alloc_page_vma+0x148/0x14d
<Oct/17 03:55 pm>[6853664.731889]  [<ffffffff810c4d2a>] handle_mm_fault+0x294/0x73e
<Oct/17 03:55 pm>[6853664.738166]  [<ffffffff815227e9>] do_page_fault+0x351/0x375
<Oct/17 03:55 pm>[6853664.744268]  [<ffffffff810cab7d>] ? sys_mmap_pgoff+0x107/0x126
<Oct/17 03:55 pm>[6853664.750630]  [<ffffffff8106472c>] ? lockdep_sys_exit+0x20/0x75
<Oct/17 03:55 pm>[6853664.756984]  [<ffffffff8151ef6f>] ? lockdep_sys_exit_thunk+0x35/0x67
<Oct/17 03:55 pm>[6853664.763892]  [<ffffffff810c4dda>] ? handle_mm_fault+0x344/0x73e
<Oct/17 03:55 pm>[6853664.770365]  [<ffffffff8151fb5f>] page_fault+0x1f/0x30
<Oct/17 03:55 pm>[6853664.776014] Pid: 19611, comm: ppImage Tainted: G      D    2.6.34.4 #3
<Oct/17 03:55 pm>[6853664.783097] Call Trace:

<Oct/17 03:55 pm>[6853664.785994]  <NMI>  [<ffffffff81009b2a>] ? show_regs+0x26/0x2a
<Oct/17 03:55 pm>[6853664.792406]  [<ffffffff81520db8>] nmi_watchdog_tick+0xc3/0x1a5
<Oct/17 03:55 pm>[6853664.798810]  [<ffffffff81520360>] do_nmi+0xb2/0x259
<Oct/17 03:55 pm>[6853664.804198]  [<ffffffff8151fdfa>] nmi+0x1a/0x20
<Oct/17 03:55 pm>[6853664.809241]  [<ffffffff812c08a2>] ? delay_tsc+0x18/0x59
<Oct/17 03:55 pm>[6853664.814961]  <<EOE>>  [<ffffffff81019da9>] ? arch_trigger_all_cpu_backtrace+0x51/0x5d
<Oct/17 03:55 pm>[6853664.823563]  [<ffffffff812c08fe>] __delay+0xa/0xc
<Oct/17 03:55 pm>[6853664.828777]  [<ffffffff812c4e65>] do_raw_spin_lock+0xde/0x147
<Oct/17 03:55 pm>[6853664.835039]  [<ffffffff8151f0dc>] _raw_spin_lock_nested+0x32/0x36
<Oct/17 03:55 pm>[6853664.841669]  [<ffffffff81035934>] ? double_rq_lock+0x4a/0x76
<Oct/17 03:55 pm>[6853664.847855]  [<ffffffff81035934>] double_rq_lock+0x4a/0x76
<Oct/17 03:55 pm>[6853664.853870]  [<ffffffff81037672>] load_balance+0xbb6/0xf63
<Oct/17 03:55 pm>[6853664.859893]  [<ffffffff810b3cfb>] ? get_dirty_limits+0x22/0x258
<Oct/17 03:55 pm>[6853664.866396]  [<ffffffff8151cc0e>] schedule+0x25a/0x86a
<Oct/17 03:55 pm>[6853664.872061]  [<ffffffff810480a2>] ? lock_timer_base+0x26/0x4a
<Oct/17 03:55 pm>[6853664.878347]  [<ffffffff8151d559>] schedule_timeout+0x222/0x24c
<Oct/17 03:55 pm>[6853664.884741]  [<ffffffff810482a5>] ? process_timeout+0x0/0xb
<Oct/17 03:55 pm>[6853664.890818]  [<ffffffff8151c6c5>] io_schedule_timeout+0x45/0x5e
<Oct/17 03:55 pm>[6853664.897277]  [<ffffffff810c0abe>] congestion_wait+0x6b/0x87
<Oct/17 03:55 pm>[6853664.903393]  [<ffffffff81054b6c>] ? autoremove_wake_function+0x0/0x38
<Oct/17 03:55 pm>[6853664.910347]  [<ffffffff810b3279>] __alloc_pages_nodemask+0x65a/0x671
<Oct/17 03:55 pm>[6853664.917237]  [<ffffffff810da90f>] alloc_page_vma+0x148/0x14d

<Oct/17 03:55 pm>[6853664.923431]  [<ffffffff810c4d2a>] handle_mm_fault+0x294/0x73e
<Oct/17 03:55 pm>[6853664.929680]  [<ffffffff815227e9>] do_page_fault+0x351/0x375
<Oct/17 03:55 pm>[6853664.935789]  [<ffffffff810cab7d>] ? sys_mmap_pgoff+0x107/0x126
<Oct/17 03:55 pm>[6853664.942163]  [<ffffffff8106472c>] ? lockdep_sys_exit+0x20/0x75
<Oct/17 03:55 pm>[6853664.948541]  [<ffffffff8151ef6f>] ? lockdep_sys_exit_thunk+0x35/0x67
<Oct/17 03:55 pm>[6853664.955442]  [<ffffffff810c4dda>] ? handle_mm_fault+0x344/0x73e
<Oct/17 03:55 pm>[6853664.961887]  [<ffffffff8151fb5f>] page_fault+0x1f/0x30

Last modified 13 years ago Last modified on Oct 18, 2013, 10:29:52 AM
Note: See TracWiki for help on using the wiki.