IPP Software Navigation Tools IPP Links Communication Pan-STARRS Links

Changes between Version 6 and Version 7 of PS1_IPP_Czarlog_20180430


Ignore:
Timestamp:
May 1, 2018, 1:47:16 PM (8 years ago)
Author:
Mark Huber
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • PS1_IPP_Czarlog_20180430

    v6 v7  
    1616  * 15:30 CZW: ippb02 rsyncs are stopped so I can transfer data from the .0 partitions of ippb07/8 to the .1 partitions, because I wrote the commands wrong.
    1717  * 16:30 CZW: The ingestion of GPC1 stacks into the gpc2 database cannot be completed due to a lack of usable w science data from GPC2.  This is needed to construct the reference warp that links the stackRun entry to the rawExp table.  Making a note here so I don't forget to fix this when GPC2 w data exists.
    18  * MEH: reprocessed the remaining 2 exposures still broken from before (o7023g0257o, o7023g0317o) so MOPS can get their data as soon as needed without further delays -- looks like the previous fix was still in an odd not-updated state in warp that pstamp could use, a normal cleanup cycle seems to have cleared and now those exposures with the broken LAP label can fully update.
    19 
     18 * MEH: reprocessed the remaining 2 exposures still broken from before (o7023g0257o, o7023g0317o) so MOPS can get their data as soon as needed without further delays -- looks like the previous fix was still in an odd not-updated state in warp that pstamp could use, a normal cleanup cycle seems to have cleared and now those exposures with the broken LAP label can fully update
     19  * the czars really shouldn't only do fixes partway when there is likelihood of other problems in other chips/skycells and asking them to let us know of problems -- that is an unnecessary delay for MOPS on what is a lower level IPP data problem and things may also then get into odd states for higher levels like the pstamp server, causing even further delays
     20  * MOPS really shouldn't have to be testers by default until all these broken files from the stsci shuffle are fixed -- to reduce as much of the unnecessary delay for MOPS, it would help if the czars would ensure that the entire exposure is clear of problems, the best way to do that is having the entire exposure stage in full state and letting them know after checking that is so -- typically this will involve both chip and warp stages -- reminder that ps_ud% cleanup happens daily 1800-1900 HST
    2021 * MEH: restored /export/ippx029.0/ipp dir after Haydn rebuilt the raid -- needs to be tested in processing
    2122 * MEH: ganglia not reporting on ippx071, ippc127, ippdb03, ippc17, ippc27, ippx052 for well over a week now -- restarted gmond and seem fine now -- if not reporting then cannot see problems, czars need to be regularly checking this
    2223 * MEH: the daily pantasks restart seems to be happening again now
     24
    2325=== Tuesday : YYYY.MM.DD ===
    2426