IPP Software Navigation Tools IPP Links Communication Pan-STARRS Links

Changes between Version 74 and Version 75 of Processing


Ignore:
Timestamp:
Jul 2, 2010, 11:06:03 AM (16 years ago)
Author:
rhenders
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • Processing

    v74 v75  
    194194}}}
    195195
    196 = Finding and dealing with {{{stdscience}}} errors =
     196= Finding and dealing with errors =
    197197
    198198== Finding log files ==
     
    202202== Reverting ==
    203203
    204 Another column in the same tables is 'fault', which are populated with integer values with the following definitions:
     204Another column in the same tables is 'fault', which is populated with integer values with the following definitions:
    205205
    206206|| '''Code''' || '''Description''' ||
     
    211211|| 5 || Error with data ||
    212212|| 6 || Error due to timeout ||
    213 
    214 Note that {{{magic}}} has fault states greater than 6.
    215 
    216 The faults are often of value '2', which is more often than not an NFS error and can be resolved by 'reverting'.
    217 
    218 == Common issues ==
    219 
    220 This section attempts to outline common issues encountered during processing and how to work through them.
    221 
    222 === stdscience ===
    223 
    224 '''Chip failures''', for example
    225 
    226 {{{
    227   AV Name                     Nrun   Njobs   Ngood Nfail Ntime Command               
    228   ++ chip.imfile.run             0   23536   17755  5781     0 chip_imfile.pl       
    229 }}}
    230 
    231 To investigate the failures, go to
    232 
    233 [http://ipp004.ifa.hawaii.edu/ippMonitor/Login.php ippmonitor]->Science steps->Chip Failed Imfiles
    234 
    235 where you can view the logs by clicking within the 'State' column.
    236 
    237 '''Warp failures'''
    238 
    239 To investigate the failures, go to
    240 
    241 [http://ipp004.ifa.hawaii.edu/ippMonitor/Login.php ippmonitor]->Science steps->Warp Failed Skyfiles
    242 
    243 Filter results by using 'new' in the state column. For the results, check that the values in the 'Fault' column are 2, which denotes an NFS error, in which case we can 'revert' using
     213|| >6 || Reserved for magic ||
     214
     215The faults are often of value '2', which is more often than not an NFS error and can be resolved by 'reverting'. For examples, for warps with a fault 2, you can revert with
    244216
    245217{{{
     
    253225}}}
    254226
    255 === Reverting destreaks ===
     227=== Special case: reverting destreaks ===
    256228
    257229From the {{{distribution}}} panstarks_client
    258230
    259231{{{
    260 
    261232destreak.off
    262233destreak.revert.on
     
    270241}}}
    271242
    272 === Removing a troublesome host ===
     243== Removing a troublesome host ==
    273244
    274245Sometimes a particular machine will act unpredictably and should be taken out of processing. To do this, go to each pantasks server in turn and remove the host, {{{ipp016}}} in the example below
    275246
    276 
    277247{{{
    278248pantasks: controller host off ipp016
     
    280250
    281251We also need to set the same host to a state of 'repair' in nebulous:
    282 
    283252
    284253{{{