IPP Software Navigation Tools IPP Links Communication Pan-STARRS Links

Changes between Initial Version and Version 1 of ipp066-20131120-crash.log


Ignore:
Timestamp:
Nov 21, 2013, 2:27:38 PM (13 years ago)
Author:
Mark Huber
Comment:

--

Legend:

Unmodified
Added
Removed
Modified

  • ipp066-20131120-crash.log

    v1 v1  
     1from console
     2{{{
     3<Nov/20 06:50 pm>ipp066 login: [3299102.015236] BUG: unable to handle kernel paging request at 0000000000013cd8
     4<Nov/20 06:50 pm>[3299102.022895] IP: [<ffffffff8102fd29>] update_curr+0xb/0x140
     5<Nov/20 06:50 pm>[3299102.028876] PGD 0
     6<Nov/20 06:50 pm>[3299102.031293] Thread overran stack, or stack corrupted
     7<Nov/20 06:50 pm>[3299102.036712] Oops: 0000 [#1] SMP
     8<Nov/20 06:50 pm>[3299102.040429] last sysfs file: /sys/devices/pci0000:00/0000:00:1f.3/i2c-0/0-002f/temp6_alarm
     9<Nov/20 06:50 pm>[3299102.049421] CPU 0
     10                 <Nov/20 06:50 pm>[3299102.051491] Modules linked in: w83793 hwmon_vid autofs4 i2c_i801 i2c_core iTCO_wdt e1000e ioatdma dca rtc_cmos tg3 libphy e1000 fuse dm_snapshot dm_mirror dm_region_hash dm_log scsi_wait_scan qla2xxx megaraid_sas megaraid_mbox megaraid_mm aacraid 3w_9xxx 3w_xxxx arcmsr aic7xxx aic79xx sr_mod sg ata_piix ahci sata_sis sata_nv sata_via pata_sl82c105 pata_sis
     11<Nov/20 06:50 pm>[3299102.085866]
     12<Nov/20 06:50 pm>[3299102.087732] Pid: 5851, comm: flush-8:0 Tainted: G   M    W  2.6.34.4 #3 S7010/empty
     13<Nov/20 06:50 pm>[3299102.096040] RIP: 0010:[<ffffffff8102fd29>]  [<ffffffff8102fd29>] update_curr+0xb/0x140
     14<Nov/20 06:50 pm>[3299102.104731] RSP: 0018:ffff880002603df8  EFLAGS: 00010006
     15<Nov/20 06:50 pm>[3299102.110503] RAX: 0000000000000246 RBX: 0000000000013c00 RCX: 0000000000000005
     16<Nov/20 06:50 pm>[3299102.118283] RDX: 0000000000000000 RSI: ffff880623826120 RDI: 0000000000013c88
     17<Nov/20 06:50 pm>[3299102.126094] RBP: ffff880002603e18 R08: 0000000000000400 R09: 0000000000000097
     18<Nov/20 06:50 pm>[3299102.133941] R10: ffffffff81038588 R11: 0000000000000000 R12: 0000000000013c88
     19
     20<Nov/20 06:50 pm>[3299102.141733] R13: ffff880623826120 R14: 0000000000013c00 R15: ffff880623826158
     21<Nov/20 06:50 pm>[3299102.149572] FS:  0000000000000000(0000) GS:ffff880002600000(0000) knlGS:0000000000000000
     22<Nov/20 06:50 pm>[3299102.158350] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
     23<Nov/20 06:50 pm>[3299102.164583] CR2: 0000000000013cd8 CR3: 0000000001a8a000 CR4: 00000000000006f0
     24<Nov/20 06:50 pm>[3299102.172384] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
     25<Nov/20 06:50 pm>[3299102.180181] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
     26<Nov/20 06:50 pm>[3299102.187987] Process flush-8:0 (pid: 5851, threadinfo ffff880623796000, task ffff880623826120)
     27<Nov/20 06:50 pm>[3299102.197259] Stack:
     28<Nov/20 06:50 pm>[3299102.199676]  0000000000013c00 0000000000013c88 ffff880623826120 0000000000013c00
     29<Nov/20 06:50 pm>[3299102.207478] <0> ffff880002603e58 ffffffff8102ffb2 000000000260eae0 ffff880623826120
     30<Nov/20 06:50 pm>[3299102.215924] <0> ffff880002613c00 0000000000000000 ffff88000260eae0 ffff88000260ea70
     31<Nov/20 06:50 pm>[3299102.224754] Call Trace:
     32                 <Nov/20 06:50 pm>[3299102.227615]  <IRQ>
     33<Nov/20 06:50 pm>[3299102.230142]  [<ffffffff8102ffb2>] task_tick_fair+0x4c/0x133
     34<Nov/20 06:50 pm>[3299102.236178]  [<ffffffff8103864e>] scheduler_tick+0x103/0x236
     35<Nov/20 06:50 pm>[3299102.242306]  [<ffffffff810483d1>] update_process_times+0x49/0x59
     36<Nov/20 06:50 pm>[3299102.248808]  [<ffffffff810610b4>] tick_sched_timer+0x73/0x9d
     37<Nov/20 06:50 pm>[3299102.254934]  [<ffffffff81061041>] ? tick_sched_timer+0x0/0x9d
     38<Nov/20 06:50 pm>[3299102.261142]  [<ffffffff81057933>] __run_hrtimer+0xd0/0x12d
     39<Nov/20 06:50 pm>[3299102.267106]  [<ffffffff81057b81>] hrtimer_interrupt+0xb5/0x19f
     40<Nov/20 06:50 pm>[3299102.273426]  [<ffffffff810197a5>] smp_apic_timer_interrupt+0x82/0x95
     41<Nov/20 06:50 pm>[3299102.280301]  [<ffffffff81003353>] apic_timer_interrupt+0x13/0x20
     42
     43<Nov/20 06:50 pm>[3299102.286798]  <EOI>
     44<Nov/20 06:50 pm>[3299102.288934] Code: c7 c2 fd fc 02 81 be 01 00 00 00 48 c7 c7 a0 da a9 81 e8 48 51 03 00 5a 59 88 d8 5b 41 5c c9 c3 55 48 89 e5 41 56 41 55 41 54 53 <4c> 8b 67 50 48 89 fb 4c 8b b7 78 07 00 00 4d 85 e4 0f 84 15 01
     45<Nov/20 06:50 pm>[3299102.310627] RIP  [<ffffffff8102fd29>] update_curr+0xb/0x140
     46<Nov/20 06:50 pm>[3299102.316701]  RSP <ffff880002603df8>
     47<Nov/20 06:50 pm>[3299102.320615] CR2: 0000000000013cd8
     48<Nov/20 06:50 pm>[3299102.324618] ---[ end trace b6463113d3410802 ]---
     49<Nov/20 06:50 pm>[3299102.329727] Kernel panic - not syncing: Fatal exception in interrupt
     50<Nov/20 06:50 pm>[3299102.336615] Pid: 5851, comm: flush-8:0 Tainted: G   M  D W  2.6.34.4 #3
     51<Nov/20 06:50 pm>[3299102.343739] Call Trace:
     52<Nov/20 06:50 pm>[3299102.346620]  <IRQ>  [<ffffffff8151c4a8>] panic+0x9e/0x118
     53<Nov/20 06:50 pm>[3299102.352563]  [<ffffffff8103bf8a>] ? kmsg_dump+0xa3/0x12c
     54<Nov/20 06:50 pm>[3299102.358379]  [<ffffffff8103bff9>] ? kmsg_dump+0x112/0x12c
     55<Nov/20 06:50 pm>[3299102.364309]  [<ffffffff81520698>] oops_end+0xaa/0xba
     56<Nov/20 06:50 pm>[3299102.369772]  [<ffffffff81023dbe>] no_context+0x1f3/0x202
     57<Nov/20 06:50 pm>[3299102.375590]  [<ffffffff81023f87>] __bad_area_nosemaphore+0x1ba/0x1e0
     58<Nov/20 06:50 pm>[3299102.382482]  [<ffffffff8102fab1>] ? __wake_up+0x43/0x4f
     59<Nov/20 06:50 pm>[3299102.388237]  [<ffffffff81023fbb>] bad_area_nosemaphore+0xe<Nov/20 06:50 pm>
     60                 <Nov/20 06:50 pm>[3299102.394636]  [<ffffffff81522660>] do_page_fault+0x1c8/0x375
     61<Nov/20 06:50 pm>[3299102.400737]  [<ffffffff8102fc93>] ? cpuacct_charge+0x8e/0x96
     62<Nov/20 06:50 pm>[3299102.406891]  [<ffffffff8102fc21>] ? cpuacct_charge+0x1c/0x96
     63<Nov/20 06:50 pm>[3299102.413065]  [<ffffffff8101c5c2>] ? physflat_send_IPI_mask+0x9/0xb
     64<Nov/20 06:50 pm>[3299102.419782]  [<ffffffff8151fb5f>] page_fault+0x1f/0x30
     65
     66<Nov/20 06:50 pm>[3299102.425442]  [<ffffffff81038588>] ? scheduler_tick+0x3d/0x236
     67<Nov/20 06:50 pm>[3299102.431710]  [<ffffffff8102fd29>] ? update_curr+0xb/0x140
     68<Nov/20 06:50 pm>[3299102.437586]  [<ffffffff81038588>] ? scheduler_tick+0x3d/0x236
     69<Nov/20 06:50 pm>[3299102.443856]  [<ffffffff8102ffb2>] task_tick_fair+0x4c/0x133
     70<Nov/20 06:50 pm>[3299102.449938]  [<ffffffff8103864e>] scheduler_tick+0x103/0x236
     71<Nov/20 06:50 pm>[3299102.456104]  [<ffffffff810483d1>] update_process_times+0x49/0x59
     72<Nov/20 06:50 pm>[3299102.462605]  [<ffffffff810610b4>] tick_sched_timer+0x73/0x9d
     73<Nov/20 06:50 pm>[3299102.468758]  [<ffffffff81061041>] ? tick_sched_timer+0x0/0x9d
     74<Nov/20 06:50 pm>[3299102.475026]  [<ffffffff81057933>] __run_hrtimer+0xd0/0x12d
     75<Nov/20 06:50 pm>[3299102.481027]  [<ffffffff81057b81>] hrtimer_interrupt+0xb5/0x19f
     76<Nov/20 06:50 pm>[3299102.487355]  [<ffffffff810197a5>] smp_apic_timer_interrupt<Nov/20 06:50 pm>
     77                 <Nov/20 06:50 pm>[3299102.494256]  [<ffffffff81003353>] apic_timer_interrupt+0x13/0x20
     78<Nov/20 06:51 pm>[3299102.500770]  <EOI>
     79<Nov/20 06:51 pm>[3299171.844879] BUG: spinlock lockup on CPU#5, kswapd0/608, ffff880002613c00
     80<Nov/20 06:51 pm>[3299171.852510] Pid: 608, comm: kswapd0 Tainted: G   M  D W  2.6.34.4 #3
     81<Nov/20 06:51 pm>[3299171.859398] Call Trace:
     82<Nov/20 06:51 pm>[3299171.862293]  [<ffffffff812c4e9f>] do_raw_spin_lock+0x118/0x147
     83<Nov/20 06:51 pm>[3299171.868641]  [<ffffffff8151f193>] _raw_spin_lock+0x34/0x39
     84<Nov/20 06:51 pm>[3299171.874627]  [<ffffffff8102fb6b>] ? task_rq_lock+0x7e/0x9a
     85<Nov/20 06:51 pm>[3299171.880617]  [<ffffffff8102fb6b>] task_rq_lock+0x7e/0x9a
     86<Nov/20 06:51 pm>[3299171.886419]  [<ffffffff81036578>] try_to_wake_up+0x2e/0x342
     87<Nov/20 06:51 pm>[3299171.892478]  [<ffffffff810368b8>] wake_up_process+0x10/0x12
     88
     89<Nov/20 06:51 pm>[3299171.898522]  [<ffffffff8127e964>] xfsbufd_wakeup+0x49/0x76
     90<Nov/20 06:51 pm>[3299171.904514]  [<ffffffff810b8876>] shrink_slab+0x69/0x148
     91<Nov/20 06:51 pm>[3299171.910320]  [<ffffffff810b9340>] kswapd+0x640/0x8bb
     92<Nov/20 06:51 pm>[3299171.915775]  [<ffffffff810b6cc0>] ? isolate_pages_global+0x0/0x23f
     93<Nov/20 06:51 pm>[3299171.922497]  [<ffffffff81054b6c>] ? autoremove_wake_function+0x0/0x38
     94<Nov/20 06:51 pm>[3299171.933967]  [<ffffffff810b8d00>] ? kswapd+0x0/0x8bb
     95<Nov/20 06:51 pm>[3299171.939459]  [<ffffffff810546e2>] kthread+0x7d/0x85
     96<Nov/20 06:51 pm>[3299171.944818]  [<ffffffff81003794>] kernel_thread_helper+0x4/0x10
     97<Nov/20 06:51 pm>[3299171.951229]  [<ffffffff81054665>] ? kthread+0x0/0x85
     98<Nov/20 06:51 pm>[3299171.956687]  [<ffffffff81003790>] ? kernel_thread_helper+0x0/0x10
     99<Nov/20 06:51 pm>[3299171.963269] sending NMI to all CPUs:
     100<Nov/20 06:51 pm>[3299171.967324] NMI backtrace for cpu 0
     101<Nov/20 06:51 pm>[3299171.971290] CPU 0
     102<Nov/20 06:51 pm>[3299171.973379] Modules linked in: w83793 hwmon_vid autofs4 i2c_i801 i2c_core iTCO_wdt e1000e ioatdma dca rtc_cmos tg3 libphy e1000 fuse dm_snapshot dm_mirror dm_region_hash dm_log scsi_wait_scan qla2xxx megaraid_sas megaraid_mbox megaraid_mm aacraid 3w_9xxx 3w_xxxx arcmsr aic7xxx aic79xx sr_mod sg ata_piix ahci sata_sis sata_nv sata_via pata_sl82c105 pata_sis
     103<Nov/20 06:51 pm>[3299172.009237]
     104<Nov/20 06:51 pm>[3299172.011144] Pid: 5851, comm: flush-8:0 Tainted: G   M  D W  2.6.34.4 #3 S7010/empty
     105<Nov/20 06:51 pm>[3299172.019515] RIP: 0010:[<ffffffff812c08be>]  [<ffffffff812c08be>] delay_tsc+0x34/0x59
     106<Nov/20 06:51 pm>[3299172.028023] RSP: 0018:ffff8800026039b0  EFLAGS: 00000293
     107<Nov/20 06:51 pm>[3299172.033830] RAX: 000000000003a3f8 RBX: 0000000000010c15 RCX: 00000000d8396e38
     108<Nov/20 06:51 pm>[3299172.041666] RDX: 00000000001f56b0 RSI: 00000000d83d1230 RDI: 000000000028b2df
     109<Nov/20 06:51 pm>[3299172.049500] RBP: ffff8800026039b8 R08: 0000000000000000 R09: 0000000000000000
     110<Nov/20 06:51 pm>[3299172.057349] R10: ffffffff8132052f R11: 0000000000000082 R12: 0000000000000009
     111
     112<Nov/20 06:51 pm>[3299172.065180] R13: 0000000000000046 R14: ffff880623826120 R15: 0000000000000000
     113<Nov/20 06:51 pm>[3299172.073020] FS:  0000000000000000(0000) GS:ffff880002600000(0000) knlGS:0000000000000000
     114<Nov/20 06:51 pm>[3299172.081798] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
     115<Nov/20 06:51 pm>[3299172.088041] CR2: 0000000000013cd8 CR3: 0000000001a8a000 CR4: 00000000000006f0
     116<Nov/20 06:51 pm>[3299172.095894] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
     117<Nov/20 06:51 pm>[3299172.103737] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
     118<Nov/20 06:51 pm>[3299172.111603] Process flush-8:0 (pid: 5851, threadinfo ffff880623796000, task ffff880623826120)
     119<Nov/20 06:51 pm>[3299172.120866] Stack:
     120<Nov/20 06:51 pm>[3299172.123315]  0000000000010c15 ffff8800026039c8 ffffffff812c093d ffff8800026039e8
     121<Nov/20 06:51 pm>[3299172.131231] <0> ffffffff8103afa4 0000000000000246 ffffffff817811ba ffff880002603ad8
     122<Nov/20 06:51 pm>[3299172.139918] <0> ffffffff8151c520 0000003000000008 ffff880002603ae8 ffff880002603a18
     123<Nov/20 06:51 pm>[3299172.148956] Call Trace:
     124<Nov/20 06:51 pm>[3299172.151844]  <IRQ>
     125<Nov/20 06:51 pm>[3299172.154421]  [<ffffffff812c093d>] __const_udelay+0x3d/0x3f
     126<Nov/20 06:51 pm>[3299172.160422]  [<ffffffff8103afa4>] panic_blink_one_second+0x4b/0x7b
     127<Nov/20 06:51 pm>[3299172.167114]  [<ffffffff8151c520>] panic+0x116/0x118
     128<Nov/20 06:51 pm>[3299172.172496]  [<ffffffff8103bf8a>] ? kmsg_dump+0xa3/0x12c
     129<Nov/20 06:51 pm>[3299172.178324]  [<ffffffff8103bff9>] ? kmsg_dump+0x112/0x12c
     130<Nov/20 06:51 pm>[3299172.184237]  [<ffffffff81520698>] oops_end+0xaa/0xba
     131<Nov/20 06:51 pm>[3299172.189680]  [<ffffffff81023dbe>] no_context+0x1f3/0x202
     132<Nov/20 06:51 pm>[3299172.195491]  [<ffffffff81023f87>] __bad_area_nosemaphore+0x1ba/0x1e0
     133<Nov/20 06:51 pm>[3299172.202392]  [<ffffffff8102fab1>] ? __wake_up+0x43/0x4f
     134
     135<Nov/20 06:51 pm>[3299172.208104]  [<ffffffff81023fbb>] bad_area_nosemaphore+0xe/0x10
     136<Nov/20 06:51 pm>[3299172.214529]  [<ffffffff81522660>] do_page_fault+0x1c8/0x375
     137<Nov/20 06:51 pm>[3299172.220613]  [<ffffffff8102fc93>] ? cpuacct_charge+0x8e/0x96
     138<Nov/20 06:51 pm>[3299172.226778]  [<ffffffff8102fc21>] ? cpuacct_charge+0x1c/0x96
     139<Nov/20 06:51 pm>[3299172.232949]  [<ffffffff8101c5c2>] ? physflat_send_IPI_mask+0x9/0xb
     140<Nov/20 06:51 pm>[3299172.239666]  [<ffffffff8151fb5f>] page_fault+0x1f/0x30
     141<Nov/20 06:51 pm>[3299172.245273]  [<ffffffff81038588>] ? scheduler_tick+0x3d/0x236
     142<Nov/20 06:51 pm>[3299172.251534]  [<ffffffff8102fd29>] ? update_curr+0xb/0x140
     143<Nov/20 06:51 pm>[3299172.257454]  [<ffffffff81038588>] ? scheduler_tick+0x3d/0x236
     144<Nov/20 06:51 pm>[3299172.263708]  [<ffffffff8102ffb2>] task_tick_fair+0x4c/0x133
     145<Nov/20 06:51 pm>[3299172.269791]  [<ffffffff8103864e>] scheduler_tick+0x103/0x236
     146<Nov/20 06:51 pm>[3299172.275954]  [<ffffffff810483d1>] update_process_times+0x49/0x59
     147<Nov/20 06:51 pm>[3299172.282447]  [<ffffffff810610b4>] tick_sched_timer+0x73/0x9d
     148<Nov/20 06:51 pm>[3299172.288601]  [<ffffffff81061041>] ? tick_sched_timer+0x0/0x9d
     149<Nov/20 06:51 pm>[3299172.294860]  [<ffffffff81057933>] __run_hrtimer+0xd0/0x12d
     150<Nov/20 06:51 pm>[3299172.300875]  [<ffffffff81057b81>] hrtimer_interrupt+0xb5/0x19f
     151<Nov/20 06:51 pm>[3299172.307213]  [<ffffffff810197a5>] smp_apic_timer_interrupt+0x82/0x95
     152<Nov/20 06:51 pm>[3299172.314096]  [<ffffffff81003353>] apic_timer_interrupt+0x13/0x20
     153<Nov/20 06:51 pm>[3299172.320610]  <EOI>
     154                 <Nov/20 06:51 pm>[3299172.322794] Code: 8b 34 25 f0 d1 00 00 66 66 90 0f ae e8 0f 31 89 c1 41 89 f1 66 66 90 0f ae e8 0f 31 89 c6 48 89 f0 48 29 c8 48 39 f8 73 25 f3 90 <65> 44 8b 04 25 f0 d1 00 00 45 39 c1 74 db 48 29 f1 48 01 cf 66
     155<Nov/20 06:51 pm>[3299172.346368] Call Trace:
     156<Nov/20 06:51 pm>[3299172.349253]  <IRQ>  [<ffffffff812c093d>] __const_udelay+0x3d/0x3f
     157
     158<Nov/20 06:51 pm>[3299172.355900]  [<ffffffff8103afa4>] panic_blink_one_second+0x4b/0x7b
     159<Nov/20 06:51 pm>[3299172.362589]  [<ffffffff8151c520>] panic+0x116/0x118
     160<Nov/20 06:51 pm>[3299172.367955]  [<ffffffff8103bf8a>] ? kmsg_dump+0xa3/0x12c
     161<Nov/20 06:51 pm>[3299172.373780]  [<ffffffff8103bff9>] ? kmsg_dump+0x112/0x12c
     162<Nov/20 06:51 pm>[3299172.379686]  [<ffffffff81520698>] oops_end+0xaa/0xba
     163<Nov/20 06:51 pm>[3299172.385157]  [<ffffffff81023dbe>] no_context+0x1f3/0x202
     164<Nov/20 06:51 pm>[3299172.390976]  [<ffffffff81023f87>] __bad_area_nosemaphore+0x1ba/0x1e0
     165<Nov/20 06:51 pm>[3299172.397868]  [<ffffffff8102fab1>] ? __wake_up+0x43/0x4f
     166<Nov/20 06:51 pm>[3299172.403613]  [<ffffffff81023fbb>] bad_area_nosemaphore+0xe/0x10
     167<Nov/20 06:51 pm>[3299172.410021]  [<ffffffff81522660>] do_page_fault+0x1c8/0x375
     168<Nov/20 06:51 pm>[3299172.416123]  [<ffffffff8102fc93>] ? cpuacct_charge+0x8e/0x96
     169<Nov/20 06:51 pm>[3299172.422270]  [<ffffffff8102fc21>] ? cpuacct_charge+0x1c/0x96
     170<Nov/20 06:51 pm>[3299172.428451]  [<ffffffff8101c5c2>] ? physflat_send_IPI_mask+0x9/0xb
     171<Nov/20 06:51 pm>[3299172.435151]  [<ffffffff8151fb5f>] page_fault+0x1f/0x30
     172<Nov/20 06:51 pm>[3299172.440793]  [<ffffffff81038588>] ? scheduler_tick+0x3d/0x236
     173<Nov/20 06:51 pm>[3299172.447044]  [<ffffffff8102fd29>] ? update_curr+0xb/0x140
     174<Nov/20 06:51 pm>[3299172.452963]  [<ffffffff81038588>] ? scheduler_tick+0x3d/0x236
     175<Nov/20 06:51 pm>[3299172.459215]  [<ffffffff8102ffb2>] task_tick_fair+0x4c/0x133
     176<Nov/20 06:51 pm>[3299172.465284]  [<ffffffff8103864e>] scheduler_tick+0x103/0x236
     177<Nov/20 06:51 pm>[3299172.471482]  [<ffffffff810483d1>] update_process_times+0x49/0x59
     178<Nov/20 06:51 pm>[3299172.478002]  [<ffffffff810610b4>] tick_sched_timer+0x73/0x9d
     179<Nov/20 06:51 pm>[3299172.484198]  [<ffffffff81061041>] ? tick_sched_timer+0x0/0x9d
     180
     181<Nov/20 06:51 pm>[3299172.490456]  [<ffffffff81057933>] __run_hrtimer+0xd0/0x12d
     182<Nov/20 06:51 pm>[3299172.496472]  [<ffffffff81057b81>] hrtimer_interrupt+0xb5/0x19f
     183<Nov/20 06:51 pm>[3299172.502826]  [<ffffffff810197a5>] smp_apic_timer_interrupt+0x82/0x95
     184<Nov/20 06:51 pm>[3299172.509718]  [<ffffffff81003353>] apic_timer_interrupt+0x13/0x20
     185<Nov/20 06:51 pm>[3299172.516252]  <EOI> Pid: 5851, comm: flush-8:0 Tainted: G   M  D W  2.6.34.4 #3
     186<Nov/20 06:51 pm>[3299172.524219] Call Trace:
     187<Nov/20 06:51 pm>[3299172.527114]  <NMI>  [<ffffffff81009b2a>] ? show_regs+0x26/0x2a
     188<Nov/20 06:51 pm>[3299172.533524]  [<ffffffff81520db8>] nmi_watchdog_tick+0xc3/0x1a5
     189<Nov/20 06:51 pm>[3299172.539868]  [<ffffffff81520360>] do_nmi+0xb2/0x259
     190<Nov/20 06:51 pm>[3299172.545242]  [<ffffffff8151fdfa>] nmi+0x1a/0x20
     191<Nov/20 06:51 pm>[3299172.550255]  [<ffffffff8132052f>] ? vt_console_print+0x43/0x2d2
     192<Nov/20 06:51 pm>[3299172.556690]  [<ffffffff812c08be>] ? delay_tsc+0x34/0x59
     193<Nov/20 06:51 pm>[3299172.562402]  <<EOE>>  <IRQ>  [<ffffffff812c093d>] __const_udelay+0x3d/0x3f
     194<Nov/20 06:51 pm>[3299172.569901]  [<ffffffff8103afa4>] panic_blink_one_second+0x4b/0x7b
     195<Nov/20 06:51 pm>[3299172.576582]  [<ffffffff8151c520>] panic+0x116/0x118
     196<Nov/20 06:51 pm>[3299172.581946]  [<ffffffff8103bf8a>] ? kmsg_dump+0xa3/0x12c
     197<Nov/20 06:51 pm>[3299172.587742]  [<ffffffff8103bff9>] ? kmsg_dump+0x112/0x12c
     198<Nov/20 06:51 pm>[3299172.593625]  [<ffffffff81520698>] oops_end+0xaa/0xba
     199<Nov/20 06:51 pm>[3299172.599080]  [<ffffffff81023dbe>] no_context+0x1f3/0x202
     200<Nov/20 06:51 pm>[3299172.604890]  [<ffffffff81023f87>] __bad_area_nosemaphore+0x1ba/0x1e0
     201<Nov/20 06:51 pm>[3299172.611783]  [<ffffffff8102fab1>] ? __wake_up+0x43/0x4f
     202<Nov/20 06:51 pm>[3299172.617511]  [<ffffffff81023fbb>] bad_area_nosemaphore+0xe/0x10
     203
     204<Nov/20 06:51 pm>[3299172.623945]  [<ffffffff81522660>] do_page_fault+0x1c8/0x375
     205<Nov/20 06:51 pm>[3299172.630021]  [<ffffffff8102fc93>] ? cpuacct_charge+0x8e/0x96
     206<Nov/20 06:51 pm>[3299172.636191]  [<ffffffff8102fc21>] ? cpuacct_charge+0x1c/0x96
     207<Nov/20 06:51 pm>[3299172.642348]  [<ffffffff8101c5c2>] ? physflat_send_IPI_mask+0x9/0xb
     208<Nov/20 06:51 pm>[3299172.649056]  [<ffffffff8151fb5f>] page_fault+0x1f/0x30
     209<Nov/20 06:51 pm>[3299172.654690]  [<ffffffff81038588>] ? scheduler_tick+0x3d/0x236
     210<Nov/20 06:51 pm>[3299172.660931]  [<ffffffff8102fd29>] ? update_curr+0xb/0x140
     211<Nov/20 06:51 pm>[3299172.666819]  [<ffffffff81038588>] ? scheduler_tick+0x3d/0x236
     212<Nov/20 06:51 pm>[3299172.673078]  [<ffffffff8102ffb2>] task_tick_fair+0x4c/0x133
     213<Nov/20 06:51 pm>[3299172.679171]  [<ffffffff8103864e>] scheduler_tick+0x103/0x236
     214<Nov/20 06:51 pm>[3299172.685309]  [<ffffffff810483d1>] update_process_times+0x49/0x59
     215<Nov/20 06:51 pm>[3299172.691855]  [<ffffffff810610b4>] tick_sched_timer+0x73/0x<Nov/20 06:51 pm>
     216                 <Nov/20 06:51 pm>[3299172.698033]  [<ffffffff81061041>] ? tick_sched_timer+0x0/0x9d
     217<Nov/20 06:51 pm>[3299172.704285]  [<ffffffff81057933>] __run_hrtimer+0xd0/0x12d
     218<Nov/20 06:51 pm>[3299172.710291]  [<ffffffff81057b81>] hrtimer_interrupt+0xb5/0x19f
     219<Nov/20 06:51 pm>[3299172.716622]  [<ffffffff810197a5>] smp_apic_timer_interrupt+0x82/0x95
     220<Nov/20 06:51 pm>[3299172.723546]  [<ffffffff81003353>] apic_timer_interrupt+0x13/0x20
     221<Nov/20 06:51 pm>[3299172.730088]  <EOI>
     222<Nov/20 06:51 pm>[3299172.732263] NMI backtrace for cpu 5
     223<Nov/20 06:51 pm>[3299172.736596] CPU 5
     224<Nov/20 06:51 pm>[3299172.738702] Modules linked in: w83793 hwmon_vid autofs4 i2c_i801 i2c_core iTCO_wdt e1000e ioatdma dca rtc_cmos tg3 libphy e1000 fuse dm_snapshot dm_mirror dm_region_hash dm_log scsi_wait_scan qla2xxx megaraid_sas megaraid_mbox megaraid_mm aacraid 3w_9xxx 3w_xxxx arcmsr aic7xxx aic79xx sr_mod sg ata_piix ahci sata_sis sata_nv sata_via pata_sl82c105 pata_sis
     225<Nov/20 06:51 pm>[3299172.774516]
     226
     227<Nov/20 06:51 pm>[3299172.776425] Pid: 608, comm: kswapd0 Tainted: G   M  D W  2.6.34.4 #3 S7010/empty
     228<Nov/20 06:51 pm>[3299172.784542] RIP: 0010:[<ffffffff812c08af>]  [<ffffffff812c08af>] delay_tsc+0x25/0x59
     229<Nov/20 06:51 pm>[3299172.793078] RSP: 0018:ffff88062380dbd8  EFLAGS: 00000046
     230<Nov/20 06:51 pm>[3299172.798878] RAX: 00000000d8412550 RBX: 0000000000000001 RCX: 00000000d84124d8
     231<Nov/20 06:51 pm>[3299172.806705] RDX: 00000000001f56b0 RSI: 00000000d8412524 RDI: 000000000028b0df
     232<Nov/20 06:51 pm>[3299172.814557] RBP: ffff88062380dbe0 R08: 0000000000000005 R09: 0000000000000005
     233<Nov/20 06:51 pm>[3299172.822397] R10: ffff8800000b83c0 R11: 0000000000000086 R12: ffff880002613c00
     234<Nov/20 06:51 pm>[3299172.830253] R13: 0000000000000000 R14: ffff880623fcb090 R15: ffff880623fcb518
     235<Nov/20 06:51 pm>[3299172.838119] FS:  0000000000000000(0000) GS:ffff8800026a0000(0000) knlGS:0000000000000000
     236<Nov/20 06:51 pm>[3299172.846984] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
     237<Nov/20 06:51 pm>[3299172.853244] CR2: 00000000025891c8 CR3: 00000002d99dd000 CR4: 00000000000006e0
     238<Nov/20 06:51 pm>[3299172.861079] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
     239<Nov/20 06:51 pm>[3299172.868929] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
     240<Nov/20 06:51 pm>[3299172.876779] Process kswapd0 (pid: 608, threadinfo ffff88062380c000, task ffff880623fcb090)
     241<Nov/20 06:51 pm>[3299172.885815] Stack:
     242<Nov/20 06:51 pm>[3299172.892763]  ffff880623fcb518 ffff88062380dbf0 ffffffff812c093d ffff88062380dc10
     243<Nov/20 06:51 pm>[3299172.900677] <0> ffffffff81019da9 000000009ef2fcf8 000000009ef2fcf8 ffff88062380dc50
     244<Nov/20 06:51 pm>[3299172.909323] <0> ffffffff812c4ea4 000000009ef2fcf8 ffff880002613c00 ffff880623820000
     245<Nov/20 06:51 pm>[3299172.918376] Call Trace:
     246<Nov/20 06:51 pm>[3299172.921273]  [<ffffffff812c093d>] __const_udelay+0x3d/0x3f
     247<Nov/20 06:51 pm>[3299172.927274]  [<ffffffff81019da9>] arch_trigger_all_cpu_backtrace+0x51/0x5d
     248<Nov/20 06:51 pm>[3299172.934679]  [<ffffffff812c4ea4>] do_raw_spin_lock+0x11d/0x147
     249
     250<Nov/20 06:51 pm>[3299172.941028]  [<ffffffff8151f193>] _raw_spin_lock+0x34/0x39
     251<Nov/20 06:51 pm>[3299172.947034]  [<ffffffff8102fb6b>] ? task_rq_lock+0x7e/0x9a
     252<Nov/20 06:51 pm>[3299172.953034]  [<ffffffff8102fb6b>] task_rq_lock+0x7e/0x9a
     253<Nov/20 06:51 pm>[3299172.958834]  [<ffffffff81036578>] try_to_wake_up+0x2e/0x342
     254<Nov/20 06:51 pm>[3299172.964896]  [<ffffffff810368b8>] wake_up_process+0x10/0x12
     255<Nov/20 06:51 pm>[3299172.970981]  [<ffffffff8127e964>] xfsbufd_wakeup+0x49/0x76
     256<Nov/20 06:51 pm>[3299172.976970]  [<ffffffff810b8876>] shrink_slab+0x69/0x148
     257<Nov/20 06:51 pm>[3299172.982770]  [<ffffffff810b9340>] kswapd+0x640/0x8bb
     258<Nov/20 06:51 pm>[3299172.988240]  [<ffffffff810b6cc0>] ? isolate_pages_global+0x0/0x23f
     259<Nov/20 06:51 pm>[3299172.994945]  [<ffffffff81054b6c>] ? autoremove_wake_function+0x0/0x38
     260<Nov/20 06:51 pm>[3299173.001929]  [<ffffffff810b8d00>] ? kswapd+0x0/0x8bb
     261<Nov/20 06:51 pm>[3299173.007387]  [<ffffffff810546e2>] kthread+0x7d/0x85
     262<Nov/20 06:51 pm>[3299173.012748]  [<ffffffff81003794>] kernel_thread_helper+0x4/0x10
     263<Nov/20 06:51 pm>[3299173.019174]  [<ffffffff81054665>] ? kthread+0x0/0x85
     264<Nov/20 06:51 pm>[3299173.024651]  [<ffffffff81003790>] ? kernel_thread_helper+0x0/0x10
     265<Nov/20 06:51 pm>[3299173.031259] Code: fb 48 ff c8 c9 c3 55 48 89 e5 48 83 ec 08 65 8b 34 25 f0 d1 00 00 66 66 90 0f ae e8 0f 31 89 c1 41 89 f1 66 66 90 0f ae e8 0f 31 <89> c6 48 89 f0 48 29 c8 48 39 f8 73 25 f3 90 65 44 8b 04 25 f0
     266<Nov/20 06:51 pm>[3299173.054544] Call Trace:
     267<Nov/20 06:51 pm>[3299173.057431]  [<ffffffff812c093d>] __const_udelay+0x3d/0x3f
     268<Nov/20 06:51 pm>[3299173.063423]  [<ffffffff81019da9>] arch_trigger_all_cpu_backtrace+0x51/0x5d
     269<Nov/20 06:51 pm>[3299173.070854]  [<ffffffff812c4ea4>] do_raw_spin_lock+0x11d/0x147
     270<Nov/20 06:51 pm>[3299173.077222]  [<ffffffff8151f193>] _raw_spin_lock+0x34/0x39
     271<Nov/20 06:51 pm>[3299173.083219]  [<ffffffff8102fb6b>] ? task_rq_lock+0x7e/0x9a
     272
     273<Nov/20 06:51 pm>[3299173.089191]  [<ffffffff8102fb6b>] task_rq_lock+0x7e/0x9a
     274<Nov/20 06:51 pm>[3299173.095010]  [<ffffffff81036578>] try_to_wake_up+0x2e/0x342
     275<Nov/20 06:51 pm>[3299173.101070]  [<ffffffff810368b8>] wake_up_process+0x10/0x1<Nov/20 06:51 pm>
     276                 <Nov/20 06:51 pm>[3299173.107173]  [<ffffffff8127e964>] xfsbufd<Nov/20 06:51 pm>
     277                 <Nov/20 06:51 pm>[3299173.113172]  [<ffffffff810b8876>] shrink_slab+0x69/0x148
     278<Nov/20 06:51 pm>[3299173.118964]  [<ffffffff810b9340>] kswapd+0x640/0x8bb
     279<Nov/20 06:51 pm>[3299173.124441]  [<ffffffff810b6cc0>] ? isolate_pages_global+0x0/0x23f
     280<Nov/20 06:51 pm>[3299173.131145]  [<ffffffff81054b6c>] ? autoremove_wake_function+0x0/0x38
     281<Nov/20 06:51 pm>[3299173.138124]  [<ffffffff810b8d00>] ? kswapd+0x0/0x8bb
     282<Nov/20 06:51 pm>[3299173.143607]  [<ffffffff810546e2>] kthread+0x7d/0x85
     283<Nov/20 06:51 pm>[3299173.148976]  [<ffffffff81003794>] kernel_thread_helper+0x4/0x10
     284<Nov/20 06:51 pm>[3299173.155419]  [<ffffffff81054665>] ? kthread+0x0/0x85
     285<Nov/20 06:51 pm>[3299173.160871]  [<ffffffff81003790>] ? kernel_thread_helper+0x0/0x10
     286<Nov/20 06:51 pm>[3299173.167495] Pid: 608, comm: kswapd0 Tainted: G   M  D W  2.6.34.4 #3
     287<Nov/20 06:51 pm>[3299173.174376] Call Trace:
     288<Nov/20 06:51 pm>[3299173.177251]  <NMI>  [<ffffffff81009b2a>] ? show_regs+0x26/0x2a
     289<Nov/20 06:51 pm>[3299173.183671]  [<ffffffff81520db8>] nmi_watchdog_tick+0xc3/0x1a5
     290<Nov/20 06:51 pm>[3299173.190023]  [<ffffffff81520360>] do_nmi+0xb2/0x259
     291<Nov/20 06:51 pm>[3299173.195412]  [<ffffffff8151fdfa>] nmi+0x1a/0x20
     292<Nov/20 06:51 pm>[3299173.200435]  [<ffffffff812c08af>] ? delay_tsc+0x25/0x59
     293<Nov/20 06:51 pm>[3299173.206158]  <<EOE>>  [<ffffffff812c093d>] __const_udelay+0x3d/0x3f
     294<Nov/20 06:51 pm>[3299173.212993]  [<ffffffff81019da9>] arch_trigger_all_cpu_backtrace+0x51/0x5d
     295
     296<Nov/20 06:51 pm>[3299173.220385]  [<ffffffff812c4ea4>] do_raw_spin_lock+0x11d/0x147
     297<Nov/20 06:51 pm>[3299173.226726]  [<ffffffff8151f193>] _raw_spin_lock+0x34/0x39
     298<Nov/20 06:51 pm>[3299173.232724]  [<ffffffff8102fb6b>] ? task_rq_lock+0x7e/0x9a
     299<Nov/20 06:51 pm>[3299173.238688]  [<ffffffff8102fb6b>] task_rq_lock+0x7e/0x9a
     300<Nov/20 06:51 pm>[3299173.244514]  [<ffffffff81036578>] try_to_wake_up+0x2e/0x342
     301<Nov/20 06:51 pm>[3299173.250575]  [<ffffffff810368b8>] wake_up_process+0x10/0x12
     302<Nov/20 06:51 pm>[3299173.256662]  [<ffffffff8127e964>] xfsbufd_wakeup+0x49/0x76
     303<Nov/20 06:51 pm>[3299173.262669]  [<ffffffff810b8876>] shrink_slab+0x69/0x148
     304<Nov/20 06:51 pm>[3299173.268486]  [<ffffffff810b9340>] kswapd+0x640/0x8bb
     305<Nov/20 06:51 pm>[3299173.273963]  [<ffffffff810b6cc0>] ? isolate_pages_global+0x0/0x23f
     306<Nov/20 06:51 pm>[3299173.280678]  [<ffffffff81054b6c>] ? autoremove_wake_function+0x0/0x38
     307<Nov/20 06:51 pm>[3299173.287636]  [<ffffffff810b8d00>] ? kswapd+0x0/0x8bb
     308<Nov/20 06:51 pm>[3299173.293112]  [<ffffffff810546e2>] kthread+0x7d/0x85
     309<Nov/20 06:51 pm>[3299173.298464]  [<ffffffff81003794>] kernel_thread_helper+0x4/0x10
     310<Nov/20 06:51 pm>[3299173.304890]  [<ffffffff81054665>] ? kthread+0x0/0x85
     311<Nov/20 06:51 pm>[3299173.310383]  [<ffffffff81003790>] ? kernel_thread_helper+0x0/0x10
     312
     313}}}