| | 1 | I was exploring a couple of <code>fault = 4</code> ('programming') errors in the |
| | 2 | stacks I'm currently processing, and found that both processes died due |
| | 3 | to memory corruption, in different places in the code. There are no |
| | 4 | known or suspected bugs in the code that would cause this. Then I |
| | 5 | noticed that they both occurred on ipp025, which also recently crashed |
| | 6 | with the do_IRQ error. |
| | 7 | |
| | 8 | I suggest that the memory corruption I experienced is roughly equivalent |
| | 9 | to a memtest86 error, and conclude that ipp025 has bad memory. |
| | 10 | |
| | 11 | [wiki:User:Paul.Price Paul.Price] 13:34, 3 November 2008 (HST) |
| | 12 | |
| | 13 | |
| | 14 | <pre> |
| | 15 | 2008-11-03 21:27:12Z|ipp025 |I|ppStack |
| | 16 | Time to perform rejection on image 7: 20.196237 sec |
| | 17 | Memory Block ID: 16273299 @ 0x7f841cd084c0 |
| | 18 | Previous Block: 0x7f841cd08540 Next Block: 0x7f841cd08400 |
| | 19 | Free function: (nil) |
| | 20 | Size: 8 Reference count: 1 Persistent: No |
| | 21 | Posts: deadbeef deadbeef deadbcef |
| | 22 | Allocated in vectorAlloc at (psVector.c:78) |
| | 23 | by Thread ID 1127573840 |
| | 24 | |
| | 25 | Memory block is corrupted; buffer overflow detected. |
| | 26 | Caught in vectorFree at (psVector.c:44). |
| | 27 | |
| | 28 | p_psMemDecrRefCounter (psMemory.c:800) Unsafe to Continue |
| | 29 | </pre> |
| | 30 | |
| | 31 | <pre> |
| | 32 | 2008-11-03 21:30:29Z|ipp025 |I|psModules.imcombine |
| | 33 | Mean deviation from 98 stamps: 0.188637 +/- 0.197773 |
| | 34 | Memory Block ID: 4541300 @ 0x7f0272aba770 |
| | 35 | Previous Block: 0x7f0272aba900 Next Block: 0x7f0272aba7f0 |
| | 36 | Free function: 0x7f028e2d1210 |
| | 37 | Size: 16 Reference count: 1 Persistent: No |
| | 38 | Posts: deadbeef deadbeef deadbcef |
| | 39 | Allocated in pmSubtractionConvolve at (pmSubtraction.c:1282) |
| | 40 | by Thread ID 139648987760368 |
| | 41 | |
| | 42 | Memory block is corrupted; buffer overflow detected. |
| | 43 | Caught in psArrayElementsFree at (psArray.c:239). |
| | 44 | |
| | 45 | p_psMemDecrRefCounter (psMemory.c:800) Unsafe to Continue |
| | 46 | |
| | 47 | Aborting. Error stack: |
| | 48 | </pre> |
| | 49 | |
| | 50 | <pre> |
| | 51 | mysql> select hostname, path_base from stackSumSkyfile join stackRun |
| | 52 | using(stack_id) where label = 'pap_md_nov2008' and fault = 4; |
| | 53 | +----------+-------------------------------------------------------------------------------------+ |
| | 54 | | hostname | path_base | |
| | 55 | +----------+-------------------------------------------------------------------------------------+ |
| | 56 | | ipp025 | neb://ipp010/md_demo_Nov2008//FIXNS/skycell.02874.13/FIXNS.skycell.02874.13.stk.375 | |
| | 57 | | ipp025 | neb://ipp010/md_demo_Nov2008//FIXNS/skycell.02878.62/FIXNS.skycell.02878.62.stk.434 | |
| | 58 | +----------+-------------------------------------------------------------------------------------+ |
| | 59 | 2 rows in set (0.00 sec) |
| | 60 | </pre> |