IPP Software Navigation Tools IPP Links Communication Pan-STARRS Links

Changes between Initial Version and Version 1 of ipp014-2012-03-07-crash


Ignore:
Timestamp:
Mar 7, 2012, 12:04:09 PM (14 years ago)
Author:
bills
Comment:

--

Legend:

Unmodified
Added
Removed
Modified

  • ipp014-2012-03-07-crash

    v1 v1  
     1
     2
     3ipp014 became unresponsive with ramping load. The following was on the console.
     4{{{
     5
     6
     7
     8<Mar/07 00:21 am>[2039834.057468] burntool[24771]: segfault at 7f4291404150 ip 000000000040acc8 sp 00007fff9941ef10 error 4 in burntool[400000+1a000]
     9<Mar/07 00:24 am>[2040021.686312] w83793 0-002f: set bank to 2 failed, fall back to bank 0, read reg 0x201 error
     10<Mar/07 00:36 am>[2040734.545943] nfs: server ipp008 not responding, still trying
     11
     12<Mar/07 00:36 am>[2040741.933544] nfs: server ipp008 OK
     13<Mar/07 00:54 am>[2041844.945835] lockd: server ipp009 not responding, still trying
     14<Mar/07 00:55 am>[2041880.900253] lockd: server ipp009 OK
     15<Mar/07 01:00 am>[2042154.555887] lockd: server ipp009 not responding, still trying
     16<Mar/07 01:00 am>[2042194.559362] lockd: server ipp009 not responding, still trying
     17<Mar/07 01:03 am>[2042371.708094] lockd: server ipp009 OK
     18<Mar/07 01:03 am>[2042371.712437] lockd: server ipp009 OK
     19<Mar/07 01:11 am>[2042829.049117] lockd: server ipp009 not responding, still trying
     20<Mar/07 01:15 am>[2043069.059356] lockd: server ipp009 not responding, still trying
     21<Mar/07 01:16 am>[2043151.429837] lockd: server ipp009 OK
     22<Mar/07 01:16 am>[2043153.980385] lockd: server ipp009 OK
     23<Mar/07 01:27 am>[2043791.426853] burntool[3569]: segfault at 7fb990fcf150 ip 000000000040acc8 sp 00007fff98fe8ad0 error 4 in burntool[400000+1a000]
     24<Mar/07 01:30 am>[2043985.125630] lockd: server ipp009 not responding, still trying
     25<Mar/07 01:30 am>[2044011.026558] lockd: server ipp009 OK
     26<Mar/07 01:37 am>[2044397.882252] lockd: server ipp009 not responding, still trying
     27<Mar/07 01:37 am>[2044397.890795] lockd: server ipp009 OK
     28<Mar/07 01:44 am>[2044804.615559] nfs: server ipp009 not responding, still trying
     29<Mar/07 01:44 am>[2044806.812658] nfs: server ipp009 not responding, still trying
     30<Mar/07 01:44 am>[2044806.916741] nfs: server ipp009 OK
     31<Mar/07 01:44 am>[2044806.920685] nfs: server ipp009 OK
     32<Mar/07 01:54 am>[2045404.972359] lockd: server ipp009 not responding, still trying
     33<Mar/07 01:54 am>[2045444.975686] lockd: server ipp009 not responding, still trying
     34
     35<Mar/07 01:58 am>[2045652.641752] lockd: server ipp009 OK
     36<Mar/07 01:58 am>[2045652.645778] lockd: server ipp009 OK
     37<Mar/07 02:05 am>[2046092.528737] lockd: server ipp009 not responding, still trying
     38<Mar/07 02:06 am>[2046144.050026] lockd: server ipp009 OK
     39<Mar/07 02:12 am>[2046489.760351] nfs: server ipp015 not responding, still trying
     40<Mar/07 02:12 am>[2046489.767052] nfs: server ipp015 not responding, still trying
     41<Mar/07 02:12 am>[2046489.773859] nfs: server ipp015 OK
     42<Mar/07 02:12 am>[2046489.778345] nfs: server ipp015 OK
     43<Mar/07 02:13 am>[2046569.385443] lockd: server ipp008 not responding, still trying
     44<Mar/07 02:13 am>[2046584.860384] burntool[11693]: segfault at 7f2604675150 ip 000000000040acc8 sp 00007fff0c68e180 error 4 in burntool[400000+1a000]
     45<Mar/07 02:13 am>[2046591.902133] lockd: server ipp009 not responding, still trying
     46<Mar/07 02:16 am>[2046714.437289] lockd: server ipp009 OK
     47<Mar/07 02:16 am>[2046742.605939] lockd: server ipp008 OK
     48<Mar/07 02:21 am>[2047019.472659] lockd: server ipp009 not responding, still trying
     49<Mar/07 02:21 am>[2047068.842140] lockd: server ipp009 OK
     50<Mar/07 03:09 am>[2049902.026193] burntool[17541]: segfault at 7fb860e4c150 ip 000000000040acc8 sp 00007fff68e65950 error 4 in burntool[400000+1a000]
     51<Mar/07 04:07 am>[2053401.048609] burntool[22815]: segfault at 7ffcc5643150 ip 000000000040acc8 sp 00007fffcd65e150 error 4 in burntool[400000+1a000]
     52<Mar/07 04:58 am>[2056452.322252] burntool[27240]: segfault at 7f6434b23150 ip 000000000040acc8 sp 00007fff3cb3c630 error 4 in burntool[400000+1a000]
     53<Mar/07 05:48 am>[2059455.129657] burntool[31353]: segfault at 7fc625234150 ip 000000000040acc8 sp 00007fff2d24ed40 error 4 in burntool[400000+1a000]
     54<Mar/07 06:55 am>[2063506.595390] burntool[5285]: segfault at 7f57df248150 ip 000000000040acc8 sp 00007fffe7262d50 error 4 in burntool[400000+1a000]
     55<Mar/07 07:48 am>[2066690.493896] burntool[9761]: segfault at 7fef540e5150 ip 000000000040acc8 sp 00007fff5c0fdbf0 error 4 in burntool[400000+1a000]
     56<Mar/07 08:46 am>[2070125.905056] burntool[14723]: segfault at 7f8d9fb45150 ip 000000000040acc8 sp 00007fffa7b60650 error 4 in burntool[400000+1a000]
     57
     58<Mar/07 09:37 am>[2073223.362867] lockd: server ipp009 not responding, still trying
     59<Mar/07 09:39 am>[2073336.126094] lockd: server ipp009 OK
     60                 <Mar/07 09:42 am>[2073518.497426] burntool[20652]: segfault at 7fcf3d875150 ip 000000000040acc8 sp 00007fff45890380 error 4 in burntool[400000+1a000]
     61<Mar/07 09:55 am>[2074305.919359] lockd: server ipp009 not responding, still trying
     62<Mar/07 09:58 am>[2074446.655081] BUG: unable to handle kernel paging request at ffffffff800073d4
     63<Mar/07 09:58 am>[2074446.655894] IP: [<ffffffff80589612>] xprt_autoclose+0x19/0x4c
     64<Mar/07 09:58 am>[2074446.655894] PGD 203067 PUD 207063 PMD 0
     65<Mar/07 09:58 am>[2074446.655894] Oops: 0000 [#1] SMP
     66<Mar/07 09:58 am>[2074446.655894] last sysfs file: /sys/class/i2c-adapter/i2c-0/0-002f/temp6_alarm
     67<Mar/07 09:58 am>[2074446.655894] CPU 0
     68<Mar/07 09:58 am>[2074446.655894] Modules linked in: w83793 hwmon_vid autofs4 i2c_i801 i2c_core iTCO_wdt e1000e tg3 libphy e1000 xfs dm_snapshot dm_mirror dm_region_hash dm_log aacraid 3w_9xxx 3w_xxxx atp870u arcmsr aic7xxx scsi_wait_scan
     69<Mar/07 09:58 am>[2074446.655894] Pid: 1515, comm: rpciod/0 Not tainted 2.6.28-rc7-00105-gfeaf384 #4
     70<Mar/07 09:58 am>[2074446.655894] RIP: 0010:[<ffffffff80589612>]  [<ffffffff80589612>] xprt_autoclose+0x19/0x4c
     71<Mar/07 09:58 am>[2074446.655894] RSP: 0018:ffff88033d367e60  EFLAGS: 00010246
     72<Mar/07 09:58 am>[2074446.655894] RAX: ffffffff80007374 RBX: ffff88010001a578 RCX: 0000000000000000
     73<Mar/07 09:58 am>[2074446.655894] RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffff88010001a000
     74<Mar/07 09:58 am>[2074446.655894] RBP: ffff88033d365a00 R08: 0000000000000001 R09: 0000000000000000
     75<Mar/07 09:58 am>[2074446.655894] R10: 0000000000000000 R11: ffffffff8050a1ce R12: ffff88010001a578
     76<Mar/07 09:58 am>[2074446.655894] R13: ffff88010001a000 R14: ffffffff808b85a0 R15: 0000000000000000
     77<Mar/07 09:58 am>[2074446.655894] FS:  0000000000000000(0000) GS:ffffffff80872d80(0000) knlGS:0000000000000000
     78<Mar/07 09:58 am>[2074446.655894] CS:  0010 DS: 0018 ES: 0018 CR0: 000000008005003b
     79<Mar/07 09:58 am>[2074446.655894] CR2: ffffffff800073d4 CR3: 000000033e52f000 CR4: 00000000000006e0
     80
     81<Mar/07 09:58 am>[2074446.655894] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
     82<Mar/07 09:58 am>[2074446.655894] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
     83<Mar/07 09:58 am>[2074446.655894] Process rpciod/0 (pid: 1515, threadinfo ffff88033d366000, task ffff88033d1fcbf0)
     84<Mar/07 09:58 am>[2074446.655894] Stack:
     85<Mar/07 09:58 am>[2074446.655894]  ffff88033d367e90 ffff88010001a578 ffffffff805895f9 ffffffff80248973
     86<Mar/07 09:58 am>[2074446.655894]  ffffffff8024891d 0000000000000046 0000000000000000 0000000000000000
     87<Mar/07 09:58 am>[2074446.655894]  0000000000000000 ffff88033d365a00 ffff88063e443bf0 ffff88033d365a00
     88<Mar/07 09:58 am>[2074446.655894] Call Trace:
     89<Mar/07 09:58 am>[2074446.655894]  [<ffffffff805895f9>] ? xprt_autoclose+0x0/0x4c
     90<Mar/07 09:58 am>[2074446.655894]  [<ffffffff80248973>] ? run_workqueue+0xed/0x1ed
     91<Mar/07 09:58 am>[2074446.655894]  [<ffffffff8024891d>] ? run_workqueue+0x97/0x1<Mar/07 09:58 am>
     92                 <Mar/07 09:58 am>[2074446.655894]  [<ffffffff80249511>] ? worker_thread+0xd8/0xe3
     93<Mar/07 09:58 am>[2074446.655894]  [<ffffffff8024bfda>] ? autoremove_wake_function+0x0/0x2e
     94<Mar/07 09:58 am>[2074446.655894]  [<ffffffff80249439>] ? worker_thread+0x0/0xe3
     95<Mar/07 09:58 am>[2074446.655894]  [<ffffffff8024becc>] ? kthread+0x47/0x73
     96<Mar/07 09:58 am>[2074446.655894]  [<ffffffff80234852>] ? schedule_tail+0x27/0x5f
     97<Mar/07 09:58 am>[2074446.655894]  [<ffffffff8020cd19>] ? child_rip+0xa/0x11
     98<Mar/07 09:58 am>[2074446.655894]  [<ffffffff804fa23c>] ? menu_reflect+0x0/0x6d
     99<Mar/07 09:58 am>[2074446.655894]  [<ffffffff8024be85>] ? kthread+0x0/0x73
     100<Mar/07 09:58 am>[2074446.655894]  [<ffffffff8020cd0f>] ? child_rip+0x0/0x11
     101<Mar/07 09:58 am>[2074446.655894] Code: 89 f7 e8 59 dd 02 00 89 d8 5b 5d 41 5c 41 5d 41 5e c3 41 55 4c 8d af 88 fa ff ff 41 54 53 48 8b 87 90 fa ff ff 48 89 fb 4c 89 ef <ff> 50 60 f0 80 63 d0 f7 4c 8d a3 98 00 00 00 4c 89 e7 e8 43 de
     102<Mar/07 09:58 am>[2074446.655894] RIP  [<ffffffff80589612>] xprt_autoclose+0x19/0x4c
     103
     104<Mar/07 09:58 am>[2074446.655894]  RSP <ffff88033d367e60>
     105<Mar/07 09:58 am>[2074446.655894] CR2: ffffffff800073d4
     106<Mar/07 09:58 am>[2074446.655894] ---[ end trace 8a138b265fc38a3c ]---
     107<Mar/07 09:59 am>[2074508.862741] lockd: cannot unmonitor ipp065
     108<Mar/07 10:00 am>[2074572.829332] lockd: cannot monitor ipp049
     109<Mar/07 10:01 am>[2074632.832642] lockd: cannot monitor ippc28
     110<Mar/07 10:02 am>[2074692.835977] lockd: cannot unmonitor ippc07
     111<Mar/07 10:03 am>[2074752.839315] lockd: cannot unmonitor ipp059
     112<Mar/07 10:04 am>[2074812.842692] lockd: cannot unmonitor ippc47
     113<Mar/07 10:05 am>[2074872.845991] lockd: cannot unmonitor ippc02
     114<Mar/07 10:06 am>[2074932.849349] lockd: cannot unmonitor ippc49
     115<Mar/07 10:07 am>[2074979.992601] nfs: server ipp047 not responding, still trying
     116<Mar/07 10:07 am>[2074992.852678] lockd: cannot unmonitor ippc03
     117<Mar/07 10:08 am>[2075052.859339] lockd: cannot unmonitor ippc50
     118<Mar/07 10:09 am>[2075112.862661] lockd: cannot unmonitor ippc04
     119                 <Mar/07 10:10 am>[2075172.865939] lockd: cannot unmonitor ipp055
     120<Mar/07 10:11 am>[2075232.869312] lockd: cannot unmonitor ippc59
     121<Mar/07 10:12 am>[2075292.872596] lockd: cannot unmonitor ippc14
     122<Mar/07 10:13 am>[2075352.875919] lockd: cannot unmonitor ipp066
     123<Mar/07 10:14 am>[2075412.879260] lockd: cannot unmonitor ippc09
     124<Mar/07 10:15 am>[2075472.882577] lockd: cannot unmonitor ipp031
     125<Mar/07 10:16 am>[2075532.885955] lockd: cannot unmonitor ipp033
     126
     127<Mar/07 10:17 am>[2075592.889281] lockd: cannot unmonitor ippc57
     128                 <Mar/07 10:18 am>[2075652.892636] lockd: cannot unmonitor ippc58
     129<Mar/07 10:19 am>[2075712.895961] lockd: cannot unmonitor ippc21
     130<Mar/07 10:20 am>[2075772.899213] lockd: cannot unmonitor ipp010
     131<Mar/07 10:21 am>[2075832.902624] lockd: cannot unmonitor ippc22
     132<Mar/07 10:22 am>[2075892.905865] lockd: cannot unmonitor ippc36
     133                 <Mar/07 10:23 am>[2075952.909192] lockd: cannot unmonitor ipp011
     134<Mar/07 10:24 am>[2076012.912566] lockd: cannot unmonitor ippc23
     135<Mar/07 10:25 am>[2076076.715871] lockd: cannot unmonitor ipp039
     136<Mar/07 10:26 am>[2076136.719171] lockd: cannot unmonitor ipp040
     137<Mar/07 10:27 am>[2076202.732481] lockd: cannot unmonitor ippc43
     138<Mar/07 10:28 am>[2076262.735867] lockd: cannot unmonitor ippc44
     139<Mar/07 10:29 am>[2076322.739173] lockd: cannot unmonitor stare01
     140<Mar/07 10:30 am>[2076382.742470] lockd: cannot unmonitor ippc45
     141<Mar/07 10:31 am>[2076442.745826] lockd: cannot unmonitor ippc46
     142<Mar/07 10:32 am>[2076502.749207] lockd: cannot unmonitor stare02
     143<Mar/07 10:33 am>[2076562.752491] lockd: cannot unmonitor ippc25
     144<Mar/07 10:34 am>[2076622.755774] lockd: cannot unmonitor ippc26
     145<Mar/07 10:35 am>[2076683.785866] lockd: cannot monitor ippc20
     146<Mar/07 10:36 am>[2076743.789209] lockd: cannot monitor ippc10
     147<Mar/07 10:37 am>[2076803.792460] lockd: cannot unmonitor ippc06
     148<Mar/07 10:38 am>[2076863.795773] lockd: cannot unmonitor ipp057
     149
     150<Mar/07 10:39 am>[2076923.799100] lockd: cannot unmonitor ippc53
     151<Mar/07 10:40 am>[2076988.135760] lockd: cannot unmonitor ipp054
     152<Mar/07 10:41 am>[2077048.139132] lockd: cannot unmonitor ipp061
     153<Mar/07 10:42 am>[2077108.142416] lockd: cannot unmonitor ipp062
     154<Mar/07 10:43 am>[2077168.145747] lockd: cannot unmonitor ipp034
     155<Mar/07 10:44 am>[2077228.149083] lockd: cannot unmonitor ipp043
     156<Mar/07 10:45 am>[2077288.152376] lockd: cannot unmonitor ipp044
     157<Mar/07 10:46 am>[2077348.155761] lockd: cannot unmonitor ipp046
     158<Mar/07 10:47 am>[2077408.159048] lockd: cannot unmonitor ipp006
     159                 <Mar/07 10:48 am>[2077468.162373] lockd: cannot unmonitor ipp007
     160<Mar/07 10:49 am>[2077528.165702] lockd: cannot unmonitor ipp042
     161<Mar/07 10:50 am>[2077588.169030] lockd: cannot unmonitor ipp051
     162<Mar/07 10:51 am>[2077648.172356] lockd: cannot unmonitor ipp052
     163<Mar/07 10:52 am>[2077708.179012] lockd: cannot unmonitor ipp053
     164                 <Mar/07 10:53 am>[2077768.182346] lockd: cannot unmonitor ipp021
     165<Mar/07 10:54 am>[2077828.185683] lockd: cannot unmonitor ipp015
     166<Mar/07 10:55 am>[2077888.189000] lockd: cannot unmonitor ipp050
     167<Mar/07 10:56 am>[2077948.192358] lockd: cannot monitor ippc41
     168<Mar/07 10:57 am>[2078008.195721] lockd: cannot monitor ippc20
     169<Mar/07 10:58 am>[2078068.199012] lockd: cannot unmonitor ipp028
     170<Mar/07 10:59 am>[2078128.202308] lockd: cannot unmonitor ipp030
     171<Mar/07 11:00 am>[2078188.205639] lockd: cannot unmonitor ipp023
     172
     173<Mar/07 11:01 am>[2078248.209007] lockd: cannot unmonitor ipp035
     174<Mar/07 11:02 am>[2078308.212341] lockd: cannot unmonitor ipp036
     175                 <Mar/07 11:03 am>[2078368.215659] lockd: cannot unmonitor ipp038
     176<Mar/07 11:04 am>[2078428.218919] lockd: cannot monitor ippc28
     177<Mar/07 11:05 am>[2078488.222272] lockd: cannot monitor ipp049
     178<Mar/07 11:06 am>[2078548.225600] lockd: cannot monitor ippc41
     179<Mar/07 11:07 am>[2078608.228926] lockd: cannot monitor ippc10
     180<Mar/07 11:08 am>[2078668.232258] lockd: cannot monitor ipp040
     181<Mar/07 11:09 am>[2078728.235584] lockd: cannot monitor ipp040
     182<Mar/07 11:10 am>[2078788.238955] lockd: cannot monitor ipp049
     183<Mar/07 11:11 am>[2078848.242274] lockd: cannot monitor ipp063
     184<Mar/07 11:12 am>[2078908.245565] lockd: cannot monitor ippc28
     185<Mar/07 11:13 am>[2078968.248962] lockd: cannot monitor ippc20
     186<Mar/07 11:14 am>[2079028.252294] lockd: cannot monitor ippc10
     187<Mar/07 11:15 am>[2079088.255651] lockd: cannot monitor ippc41
     188<Mar/07 11:16 am>[2079148.962286] lockd: cannot monitor ipp063
     189<Mar/07 11:17 am>[2079208.965538] lockd: cannot monitor ipp040
     190<Mar/07 11:18 am>[2079268.968833] lockd: cannot monitor ipp063
     191<Mar/07 11:19 am>[2079328.972194] lockd: cannot monitor ipp049
     192<Mar/07 11:20 am>[2079385.498847] w83793 0-002f: set bank to 0 failed, fall back to bank 2, read reg 0xb2 error
     193<Mar/07 11:20 am>[2079388.975520] lockd: cannot monitor ipp059
     194<Mar/07 11:21 am>[2079448.978901] lockd: cannot monitor ipp059
     195
     196<Mar/07 11:22 am>[2079508.982141] lockd: cannot monitor ippc28
     197<Mar/07 11:23 am>[2079568.985505] lockd: cannot monitor ippc20
     198<Mar/07 11:24 am>[2079628.988804] lockd: cannot unmonitor ippc51
     199<Mar/07 11:25 am>[2079688.992157] lockd: cannot unmonitor ippc13
     200<Mar/07 11:26 am>[2079748.995490] lockd: cannot unmonitor ipp060
     201<Mar/07 11:27 am>[2079808.998807] lockd: cannot unmonitor ippc63
     202<Mar/07 11:28 am>[2079869.002198] lockd: cannot unmonitor ippc29
     203<Mar/07 11:29 am>[2079929.005463] lockd: cannot monitor ippc10
     204<Mar/07 11:30 am>[2079989.008832] lockd: cannot monitor ippc41
     205<Mar/07 11:31 am>[2080049.012123] lockd: cannot monitor ipp059
     206<Mar/07 11:32 am>[2080109.015525] lockd: cannot monitor ipp049
     207<Mar/07 11:33 am>[2080169.018844] lockd: cannot monitor ippc28
     208<Mar/07 11:34 am>[2080229.022098] lockd: cannot monitor ippc20
     209<Mar/07 11:35 am>[2080289.025501] lockd: cannot monitor ippc10
     210<Mar/07 11:36 am>[2080349.028796] lockd: cannot monitor ippc41
     211}}}
     212
     213   
     214