Changes between Version 25 and Version 26 of PSNSC_MOPS

Timestamp:

Sep 4, 2018, 11:40:05 AM (8 years ago)

Author:

Mark Huber

Comment:

--

PSNSC_MOPS

@@ -45 +45 @@
 Nightly products -- only really need image+mask (uncompressed) so could make normal use of no weight file to save disk use during nightly
  * currently hacked to not produce mask+weight in ippitc due to overuse of itc datanodes with non-nightly products... 
- * compression and hack removed ~201709xx once MOPS installed new node -- 
+ * compression and hack removed ~201709xx once MOPS installed new nodes -- 
 
 WW diffims for MOPS --
  * excess detections causing full rejection of OTA -- typically edges full of junk, unclear if could be improved with other cuts, need real sources on edges as it can still be useful
+ * also excess detections in poor convolution/noise level that needs to be restricted more
 
 centroids poor causing large GCR -- factor of 2 larger..
@@ -56 +57 @@
 
 WS diffims for MOPS -- 
- * image files uncompressed -- either in processing or a copy on ipp032 -- ~10TB/night extra space used (20TB since data kept for 36hrs), processing seems ok (network i/o, dtime, faults were all acceptable)
+ * image files uncompressed -- either in processing or a copy on ipp032 -- ~10TB/night extra space used (20TB since data kept for 36hrs), processing seems ok (network i/o, dtime, faults were all acceptable) -- 
  * update w-band template stacks -- see below
- * could be using WS for i-band always -- need to test detection threshold like done for WWdiff (test chunk is in w-band, but use same threshold for all filters in WWdiff and even that could/should be adjusted at least for dynamic s/n cuts in MOPS)
+ * could be using WS for i-band always -- need to test detection threshold like done for WWdiff (test chunk is in w-band, but use same threshold for all filters in WWdiff and even that could/should be adjusted at least for dynamic s/n cuts in MOPS) -- i-band LAP stacks not the greatest
  * exposure uniqueness required -- update .mops catalog format? (see below) 
- * publishing -- TBD
+ * publishing -- TBD -- started for PS2 as live comparison 20180xxx -- 
 
 ppTranslate/ppMOPS -- .mops catalog format update/revision -- 
- * evaluate contents again, remove/add new columns as needed -- must add skycell_id, chip XY and position
+ * evaluate contents again, remove/add new columns as needed -- must add skycell_id, chip XY position
+  * to further reduce neb+gpc# queries, add additional table with all entires and not just best/primary with ID, skycell_id, chip xy, R to skycell center, path on disk (direct, not neb)
+ * need to re-check if selected detection is best/primary etc
  * merge WW and WSdiff catalog publishing to single product if possible -- conflict with timing possibly
  * -- likely best to just wait for MOPS to switch to directly using the diff CMF files for both -- 
+ * consider java implementation to replace ppMops
 
 w-band stack update for WSdiff -- 
@@ -72 +76 @@
  * by default will use the different LAP.PV3W set RA~7-16hr unless find a problem 
 
-z-band 3PI WWdiffs many more detections over i,w (twilight time) -- 
+z-band 3PI WWdiffs many more detections over i,w (twilight time and now full moon time for Euclid fields) -- 
 
 MOPS test samples --
@@ -94 +98 @@
  * backlog -- particularly w/ QUB stamps, in future may want to split pantasks so MOPS doesn't get behind large numbers of uploaded parse jobs
   * can bump up the parse limit (in pstamp.pro) from 10->20 and should help for short term -- added as config in ipp-20170121r40242 update (see below)
- * unclear which refcat used in returned stamps (ie for astrometry in MOPS recovery work 2MASS/GAIAv1) -- really useful for warps->diffs to also have but PSREFCAT isn't propagated...
-
+ * unclear which refcat used in returned stamps (ie for astrometry in MOPS recovery work 2MASS/GAIAv1) -- really useful for warps->diffs to also have but PSREFCAT isn't propagated... -- keyword added now 2018xxxx
+ * broken files is plaguing precovery of priority events -- 
 
 == Ops tag notes ==