from console -- power cycle stalled a while on starting local but seems to be back up {{{ ippc28 login: [1507747.909968] general protection fault: 0000 [#1] SMP [1507747.912564] last sysfs file: /sys/class/i2c-adapter/i2c-0/0-002f/temp6_alarm [1507747.912564] CPU 0 [1507747.912564] Modules linked in: w83793 hwmon_vid autofs4 e100 i2c_i801 i2c_core iTCO_wdt e1000e tg3 libphy e1000 xfs dm_snapshot dm_mirror dm_region_hash dm_log aacraid 3w_9xxx 3w_xxxx atp870u arcmsr aic7xxx scsi_wait_scan [1507747.912564] Pid: 1671, comm: rpciod/0 Not tainted 2.6.28-rc7-00105-gfeaf384 #4 [1507747.912564] RIP: 0010:[] [] run_workqueue+0x77/0x1ed [1507747.912564] RSP: 0018:ffff88063b293e80 EFLAGS: 00010087 [1507747.912564] RAX: 1aa494a4d74ac649 RBX: ffff88013f277580 RCX: 0000000000000000 [1507747.912564] RDX: ffff8809f4e1b4f8 RSI: ffff88013f2775b0 RDI: ffff88063b233000 [1507747.912564] RBP: ffff88063b233000 R08: 0000000000000001 R09: 0000000000000001 [1507747.912564] R10: 0000000000000000 R11: 0000000000000000 R12: ffff88013f277578 [1507747.912564] R13: cc4a7f6d487a4100 R14: ffffffff808b85a0 R15: 0000000000000000 [1507747.912564] FS: 0000000000000000(0000) GS:ffffffff80872d80(0000) knlGS:0000000000000000 [1507747.912564] CS: 0010 DS: 0018 ES: 0018 CR0: 000000008005003b [1507747.912564] CR2: 0000000000a1c008 CR3: 0000000638454000 CR4: 00000000000006e0 [1507747.912564] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [1507747.912564] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [1507747.912564] Process rpciod/0 (pid: 1671, threadinfo ffff88063b292000, task ffff88063b219e60) [1507747.912564] Stack: [1507747.912564] 0000000000000001 0000000000000046 5f185ad49f1d7d91 95c4cf635cb05640 [1507747.912564] a3d560adfe0c5e41 ffff88063b233000 ffff88063cc6dbf0 ffff88063b233000 [1507748.069655] ffff88063b233040 ffff88063cc6dbf0 ffffffff808b5d60 ffffffff80249511 [1507748.069655] Call Trace: [1507748.069655] [] ? worker_thread+0xd8/0xe3 [1507748.069655] [] ? autoremove_wake_function+0x0/0x2e [1507748.069655] [] ? worker_thread+0x0/0xe3 [1507748.069655] [] ? kthread+0x47/0x73 [1507748.069655] [] ? schedule_tail+0x27/0x5f [1507748.069655] [] ? child_rip+0xa/0x11 [1507748.069655] [] ? kthread+0x0/0x73 [1507748.069655] [] ? child_rip+0x0/0x11 [1507748.069655] Code: 48 8d 7c 24 10 48 8d 73 18 fc 4d 8b 6c 24 18 b9 06 00 00 00 f3 a5 4c 89 a5 80 00 00 00 48 89 ef 48 8b 43 08 48 8b 13 48 89 42 08 <48> 89 10 48 89 5b 08 48 89 1b e8 d1 ed 36 00 48 8b 43 f8 48 83 [1507748.069655] RIP [] run_workqueue+0x77/0x1ed [1507748.069655] RSP [1507748.069655] ---[ end trace d2b44a22042f9805 ]--- [1507761.949892] nfs: server ippc18 not responding, still trying [1507794.116723] ata1.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x6 frozen [1507794.124527] ata1.00: cmd ea/00:00:00:00:00/00:00:00:00:00/a0 tag 0 [1507794.124528] res 40/00:00:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout) [1507794.139389] ata1.00: status: { DRDY } [1507794.143535] ata1: hard resetting link [1507794.480033] ata1: SATA link up 3.0 Gbps (SStatus 123 SControl 300) [1507799.493344] ata1.00: qc timeout (cmd 0xec) [1507799.497890] ata1.00: failed to IDENTIFY (I/O error, err_mask=0x4) [1507799.504459] ata1.00: revalidation failed (errno=-5) [1507799.509803] ata1: hard resetting link [1507799.876696] ata1: SATA link up 3.0 Gbps (SStatus 123 SControl 300) [1507809.902766] ata2.00: exception Emask 0x0 SAct 0x1 SErr 0x0 action 0x6 frozen [1507809.903379] ata1.00: qc timeout (cmd 0xec) [1507809.903386] ata1.00: failed to IDENTIFY (I/O error, err_mask=0x4) [1507809.903388] ata1.00: revalidation failed (errno=-5) [1507809.903392] ata1: hard resetting link [1507809.931076] ata2.00: cmd 60/40:00:b1:05:03/00:00:00:00:00/40 tag 0 ncq 32768 in [1507809.931078] res 40/00:00:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout) [1507809.947109] ata2.00: status: { DRDY } [1507809.951224] ata2: hard resetting link [1507812.902775] ata2: SATA link up 3.0 Gbps (SStatus 123 SCont [1507812.903400] ata1: SATA link up 3.0 Gbps (SStatus 123 SControl 300) [1507817.902768] ata2.00: qc timeout (cmd 0xec) [1507817.907368] ata2.00: failed to IDENTIFY (I/O error, err_mask=0x4) [1507817.914048] ata2.00: revalidation failed (errno=-5) [1507817.919434] ata2: hard resetting link [1507820.902776] ata2: SATA link up 3.0 Gbps (SStatus 123 SCont [1507830.902766] ata2.00: qc timeout (cmd 0xec) [1507830.907323] ata2.00: failed to IDENTIFY (I/O error, err_mask=0x4) [1507830.913908] ata2.00: revalidation failed (errno=-5) [1507830.919250] ata2: hard resetting link [1507830.903013] BUG: spinlock lockup on CPU#0, rpciod/0/1671, [1507830.903013] Pid: 1671, comm: rpciod/0 Tainted: G D 2.6.28-rc7-00105-gfeaf384 #4 [1507830.903013] Call Trace: [1507830.903013] [] _raw_spin_lock+0xfb/0x122 [1507830.903013] [] xprt_init_autodisconnect+0x0/0x98 [1507830.903013] [] _spin_lock_irqsave+0x37/0x3f [1507830.903013] [] __queue_work+0x10/0x32 [1507830.903013] [] __queue_work+0x10/0x32 [1507830.903013] [] xprt_init_autodisconnect+0x0/0x98 [1507830.903013] [] queue_work_on+0x37/0x40 [1507830.903013] [] run_timer_softirq+0x169/0x1dd [1507830.903013] [] __do_softirq+0x7a/0x13d [1507830.903013] [] call_softirq+0x1c/0x28 [1507830.903013] [] do_softirq+0x2c/0x68 [1507830.903013] [] irq_exit+0x3f/0x85 [1507830.903013] [] smp_apic_timer_interrupt+0x93/0xac [1507830.903013] [] apic_timer_interrupt+0x6b/0x70 [1507830.903013] [] _spin_unlock_irq+0x21/0x22 [1507830.903013] [] tty_audit_exit+0x60/0x92 [1507830.903013] [] do_exit+0x1b0/0x74d [1507830.903013] [] oops_begin+0x0/0x8d [1507830.903013] [] error_exit+0x0/0x9a [1507830.903013] [] run_workqueue+0x77/0x1ed [1507830.903013] [] worker_thread+0xd8/0xe3 [1507830.903013] [] autoremove_wake_function+0x0/0x2e [1507830.903013] [] worker_thread+0x0/0xe3 [1507830.903013] [] kthread+0x47/0x73 [1507830.903013] [] schedule_tail+0x27/0x5f [1507830.903013] [] child_rip+0xa/0x11 [1507830.903013] [] kthread+0x0/0x73 [1507830.903013] [] child_rip+0x0/0x11 [1507833.902778] ata2: SATA link up 3.0 Gbps (SStatus 123 SControl 300) [1507842.903354] ata1.00: qc timeout (cmd 0xec) [1507842.907950] ata1.00: failed to IDENTIFY (I/O error, err_mask=0x4) [1507842.914597] ata1.00: revalidation failed (errno=-5) [1507842.920029] ata1.00: disabled [1507842.923489] ata1: hard resetting link [1507845.903359] ata1: SATA link up 3.0 Gbps (SStatus 123 SCont [1507845.910057] ata1: EH complete [1507845.913479] sd 0:0:0:0: [sda] Result: hostbyte=DID_BAD_TARIVER_OK,SUGGEST_OK [1507845.922808] end_request: I/O error, dev sda, sector 96195185 [1507845.928983] end_request: I/O error, dev sda, sector 96195185 [1507845.932307] md: super_written gets error=-5, uptodate=0 [1507845.932307] raid1: Disk failure on sda3, disabling device. [1507845.932307] raid1: Operation continuing on 1 devices. [1507863.902775] ata2.00: qc timeout (cmd 0xec) [1507863.907324] ata2.00: failed to IDENTIFY (I/O error, err_mask=0x4) [1507863.913897] ata2.00: revalidation failed (errno=-5) [1507863.919302] ata2.00: disabled [1507863.922742] ata2: hard resetting link [1507866.902771] ata2: SATA link up 3.0 Gbps (SStatus 123 SControl 300) [1507866.909457] ata2: EH complete [1507866.912951] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK [1507866.922239] end_request: I/O error, dev sdb, sector 198065 [1507866.928211] Read-error on swap-device (8:16:198073) [1507866.933557] Read-error on swap-device (8:16:198081) [1507866.938886] Read-error on swap-device (8:16:198089) [1507866.944238] Read-error on swap-device (8:16:198097) [1507866.949586] Read-error on swap-device (8:16:198105) [1507866.954995] Read-error on swap-device (8:16:198113) [1507866.960403] Read-error on swap-device (8:16:198121) [1507866.965802] Read-error on swap-device (8:16:198129) [1507866.971227] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK [1507866.980626] end_request: I/O error, dev sdb, sector 96195185 [1507866.986850] end_request: I/O error, dev sdb, sector 96195185 [1507866.990174] md: super_written gets error=-5, uptodate=0 [1507866.998796] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK [1507867.008133] end_request: I/O error, dev sdb, sector 96195185 [1507867.014352] end_request: I/O error, dev sdb, sector 96195185 [1507867.017673] md: super_written gets error=-5, uptodate=0 [1507867.026397] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK [1507867.035706] end_request: I/O error, dev sdb, sector 73136425 [1507867.041946] RAID1 conf printout: [1507867.042052] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK [1507867.042057] end_request: I/O error, dev sdb, sector 73136425 [1507867.042085] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK [1507867.042089] end_request: I/O error, dev sdb, sector 73136425 [1507867.042106] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK [1507867.042109] end_request: I/O error, dev sdb, sector 73136425 [1507867.042127] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK [1507867.042131] end_request: I/O error, dev sdb, sector 73136425 [1507867.042147] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK [1507867.042150] end_request: I/O error, dev sdb, sector 73136425 [1507867.042168] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK [1507867.042171] end_request: I/O error, dev sdb, sector 73136425 [1507867.042187] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK [1507867.042191] end_request: I/O error, dev sdb, sector 73136425 [1507867.042208] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK [1507867.042212] end_request: I/O error, dev sdb, sector 73136425 [1507867.042228] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK [1507867.042231] end_request: I/O error, dev sdb, sector 73136425 [1507867.042249] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK [1507867.042252] end_request: I/O error, dev sdb, sector 73136425 [1507867.042268] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK [1507867.042271] end_request: I/O error, dev sdb, sector 73136425 [1507867.042289] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK [1507867.042292] end_request: I/O error, dev sdb, sector 73136425 [1507867.042308] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK [1507867.042311] end_request: I/O error, dev sdb, sector 73136425 [1507867.042329] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK [1507867.042332] end_request: I/O error, dev sdb, sector 73136425 [1507867.042348] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK [1507867.042351] end_request: I/O error, dev sdb, sector 73136425 [1507867.042369] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK [1507867.042372] end_request: I/O error, dev sdb, sector 73136425 [1507867.042388] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK [1507867.042391] end_request: I/O error, dev sdb, sector 73136425 [1507867.042409] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK [1507867.042412] end_request: I/O error, dev sdb, sector 73136425 [1507867.042428] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK [1507867.042432] end_request: I/O error, dev sdb, sector 73136425 [1507867.042449] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK [1507867.042452] end_request: I/O error, dev sdb, sector 73136425 [1507867.042469] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK [1507867.042473] end_request: I/O error, dev sdb, sector 73136425 [1507867.042491] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK [1507867.042494] end_request: I/O error, dev sdb, sector 73136425 [1507867.042510] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK [1507867.042514] end_request: I/O error, dev sdb, sector 73136425 [1507867.042531] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK [1507867.042535] end_request: I/O error, dev sdb, sector 73136425 [1507867.042551] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK [1507867.042554] end_request: I/O error, dev sdb, sector 73136425 [1507867.042572] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK [1507867.042575] end_request: I/O error, dev sdb, sector 73136425 [1507867.042591] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK [1507867.042595] end_request: I/O error, dev sdb, sector 73136425 [1507867.042613] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK [1507867.042625] end_request: I/O error, dev sdb, sector 73136425 [1507867.042643] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK [1507867.042646] end_request: I/O error, dev sdb, sector 73136425 [1507867.042664] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK [1507867.042667] end_request: I/O error, dev sdb, sector 73136425 [1507867.042684] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK [1507867.042687] end_request: I/O error, dev sdb, sector 73136425 [1507867.042706] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK [1507867.042709] end_request: I/O error, dev sdb, sector 73136425 [1507867.042726] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK [1507867.042729] end_request: I/O error, dev sdb, sector 73136425 [1507867.042747] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK [1507867.042750] end_request: I/O error, dev sdb, sector 73136425 [1507867.042766] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK [1507867.042769] end_request: I/O error, dev sdb, sector 73136425 [1507867.042787] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK [1507867.042790] end_request: I/O error, dev sdb, sector 73136425 [1507867.042807] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK [1507867.042810] end_request: I/O error, dev sdb, sector 73136425 [1507867.042828] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK [1507867.042831] end_request: I/O error, dev sdb, sector 73136425 [1507867.042847] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK [1507867.042850] end_request: I/O error, dev sdb, sector 73136425 [1507867.042868] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK [1507867.042871] end_request: I/O error, dev sdb, sector 73136425 [1507867.042888] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK [1507867.042891] end_request: I/O error, dev sdb, sector 73136425 [1507867.042909] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK [1507867.042912] end_request: I/O error, dev sdb, sector 73136425 [1507867.042928] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK [1507867.042931] end_request: I/O error, dev sdb, sector 73136425 [1507867.042949] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK [1507867.042952] end_request: I/O error, dev sdb, sector 73136425 [1507867.042969] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK [1507867.042972] end_request: I/O error, dev sdb, sector 73136425 [1507867.042990] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK [1507867.042993] end_request: I/O error, dev sdb, sector 73136425 [1507867.043010] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK [1507867.043013] end_request: I/O error, dev sdb, sector 73136425 [1507867.043038] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK [1507867.043041] end_request: I/O error, dev sdb, sector 73136425 [1507867.043058] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK [1507867.043061] end_request: I/O error, dev sdb, sector 73136425 [1507867.043079] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK [1507867.043082] end_request: I/O error, dev sdb, sector 73136425 [1507867.043098] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK [1507867.043102] end_request: I/O error, dev sdb, sector 73136425 [1507867.043119] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK [1507867.043123] end_request: I/O error, dev sdb, sector 73136425 [1507867.043139] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK [1507867.043142] end_request: I/O error, dev sdb, sector 73136425 [1507867.043160] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK [1507867.043163] end_request: I/O error, dev sdb, sector 73136425 [1507867.886971] --- wd:1 rd:2 [1507867.890137] disk 0, wo:1, o:0, dev:sda3 [1507867.894539] disk 1, wo:0, o:1, dev:sdb3 [1507869.603310] BUG: soft lockup - CPU#11 stuck for 61s! [events/11:50] [1507869.603316] Modules linked in: w83793 hwmon_vid autofs4 e100 i2c_i801 i2c_core iTCO_wdt e1000e tg3 libphy e1000 xfs dm_snapshot dm_mirror dm_region_hash dm_log aacraid 3w_9xxx 3w_xxxx atp870u arcmsr aic7xxx scsi_wait_scan [1507869.603316] CPU 11: [1507869.603316] Modules linked in: w83793 hwmon_vid autofs4 e100 i2c_i801 i2c_core iTCO_wdt e1000e tg3 libphy e1000 xfs dm_snapshot dm_mirror dm_region_hash dm_log aacraid 3w_9xxx 3w_xxxx atp870u arcmsr aic7xxx scsi_wait_scan [1507869.603316] Pid: 50, comm: events/11 Tainted: G D 2.6.28-rc7-00105-gfeaf384 #4 [1507869.603316] RIP: 0010:[] [] smp_call_function_mask+0x163/0x1d7 [1507869.603316] RSP: 0018:ffff88063ceb7d30 EFLAGS: 00000202 [1507869.603316] RAX: 0000000000000020 RBX: 000000000000000b RCX: 0000000000000020 [1507869.603316] RDX: ffffffff8081a300 RSI: 0000000000000020 RDI: 0000000000000000 [1507869.603316] RBP: 000000000000000b R08: 0000000000000020 R09: ffff88063ceb7cf0 [1507869.603316] R10: 0000000000000000 R11: ffffffff802230fe R12: 0000000000000001 [1507869.603316] R13: ffffffff80257376 R14: 0000000000000069 R15: ffff88063ceac388 [1507869.603316] FS: 0000000000000000(0000) GS:ffff880c3cc52900(0000) knlGS:0000000000000000 [1507869.603316] CS: 0010 DS: 0018 ES: 0018 CR0: 000000008005003b [1507869.603316] CR2: 00007fff666100c0 CR3: 0000000000201000 CR4: 00000000000006e0 [1507869.603316] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [1507869.603316] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [1507869.603316] Call Trace: [1507869.603316] [] ? __lock_acquire+0x788/0x7f3 [1507869.603316] [] ? update_curr+0x93/0xc9 [1507869.603316] [] ? mcheck_check_cpu+0x0/0x2f [1507869.603316] [] ? on_each_cpu+0x10/0x22 [1507869.603316] [] ? mcheck_timer+0x0/0x76 [1507869.603316] [] ? mcheck_timer+0x14/0x76 [1507869.603316] [] ? run_workqueue+0xed/0x1ed [1507869.603316] [] ? run_workqueue+0x97/0x1ed [1507869.603316] [] ? worker_thread+0xd8/0xe3 [1507869.603316] [] ? autoremove_wake_function+0x0/0x2e [1507869.603316] [] ? worker_thread+0x0/0xe3 [1507869.603316] [] ? kthread+0x47/0x73 [1507869.603316] [] ? schedule_tail+0x27/0x5f [1507869.603316] [] ? child_rip+0xa/0x11 [1507869.603316] [] ? kthread+0x0/0x73 [1507869.603316] [] ? child_rip+0x0/0x11 [1507935.103303] BUG: soft lockup - CPU#11 stuck for 61s! [events/11:50] [1507935.103310] Modules linked in: w83793 hwmon_vid autofs4 e100 i2c_i801 i2c_core iTCO_wdt e1000e tg3 libphy e1000 xfs dm_snapshot dm_mirror dm_region_hash dm_log aacraid 3w_9xxx 3w_xxxx atp870u arcmsr aic7xxx scsi_wait_scan [1507935.103310] CPU 11: [1507935.103310] Modules linked in: w83793 hwmon_vid autofs4 e100 i2c_i801 i2c_core iTCO_wdt e1000e tg3 libphy e1000 xfs dm_snapshot dm_mirror dm_region_hash dm_log aacraid 3w_9xxx 3w_xxxx atp870u arcmsr aic7xxx scsi_wait_scan [1507935.103310] Pid: 50, comm: events/11 Tainted: G D 2.6.28-rc7-00105-gfeaf384 #4 [1507935.103310] RIP: 0010:[] [] smp_call_function_mask+0x163/0x1d7 [1507935.103310] RSP: 0018:ffff88063ceb7d30 EFLAGS: 00000202 [1507935.103310] RAX: 0000000000000020 RBX: 000000000000000b RCX: 0000000000000020 [1507935.103310] RDX: ffffffff8081a300 RSI: 0000000000000020 RDI: 0000000000000000 [1507935.103310] RBP: 000000000000000b R08: 0000000000000020 R09: ffff88063ceb7cf0 [1507935.103310] R10: 0000000000000000 R11: ffffffff802230fe R12: 0000000000000001 [1507935.103310] R13: ffffffff80257376 R14: 0000000000000069 R15: ffff88063ceac388 [1507935.103310] FS: 0000000000000000(0000) GS:ffff880c3cc52900(0000) knlGS:0000000000000000 [1507935.103310] CS: 0010 DS: 0018 ES: 0018 CR0: 000000008005003b [1507935.103310] CR2: 00007fff666100c0 CR3: 0000000000201000 CR4: 00000000000006e0 [1507935.103310] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [1507935.103310] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [1507935.103310] Call Trace: [1507935.103310] [] ? __lock_acquire+0x788/0x7f3 [1507935.103310] [] ? update_curr+0x93/0xc9 [1507935.103310] [] ? mcheck_check_cpu+0x0/0x2f [1507935.103310] [] ? on_each_cpu+0x10/0x22 [1507935.103310] [] ? mcheck_timer+0x0/0x76 [1507935.103310] [] ? mcheck_timer+0x14/0x76 [1507935.103310] [] ? run_workqueue+0xed/0x1ed [1507935.103310] [] ? run_workqueue+0x97/0x1ed [1507935.103310] [] ? worker_thread+0xd8/0xe3 [1507935.103310] [] ? autoremove_wake_function+0x0/0x2e [1507935.103310] [] ? worker_thread+0x0/0xe3 [1507935.103310] [] ? kthread+0x47/0x73 [1507935.103310] [] ? schedule_tail+0x27/0x5f [1507935.103310] [] ? child_rip+0xa/0x11 [1507935.103310] [] ? kthread+0x0/0x73