IPP Software Navigation Tools IPP Links Communication Pan-STARRS Links

Changes between Version 8 and Version 9 of PS1_IPP_Czarlog_20190401


Ignore:
Timestamp:
Apr 2, 2019, 12:47:36 PM (7 years ago)
Author:
tdeboer
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • PS1_IPP_Czarlog_20190401

    v8 v9  
    3636 That solved the issue. Unfortunately, some exposures were faulting due to ipp127 not being available yet, at the time. I will follow-up more on these, once the rack is back up.
    3737
    38 
    3938 *TdB: Shortly after observing started, we noticed that processing was not proceeding at all, and an IPP hosed was called. Mark, Roger, John and myself met via zoom to solve the issue, around 21:00.
    4039  We noticed the issues was related to missing detrend files, which was somewhat strange since nothing changed since last week, when we did open and data went through fine. Initially, we figured it was due to some machines being down, so we turned those back to repair (ipp057, ippb17,18,21,22) and set the exposures to revert. This did not help.
     
    5655 This solved most of the issue, but we found chip CY05 kept failing on each of the exposures. We inspected the actual file using ds9 and it looked ok. We inspected its replication and checksum using neb-stat -validate, which also looked ok. But then we realised this was in fact a DARKTEST file, which the processing was now using, which had no end_time at all. We stopped the stdscience pantasks while investigating, since this is not a good file to use in the nightly processing.
    5756
    58 The det_id of the file was 866, and we set its end_time to yesterday to prevent it from being grabbed in the future:
     57 The det_id of the file was 866, and we set its end_time to yesterday to prevent it from being grabbed in the future:
    5958{{{
    6059dettool -updatedetrun -dbname gpc1 -set_time_end 2099-04-01T00:00:00.000000 -det_id 866