Changes between Version 28 and Version 29 of PSNSC_NCU
- Timestamp:
- Oct 25, 2016, 7:54:13 PM (10 years ago)
Legend:
- Unmodified
- Added
- Removed
- Modified
-
PSNSC_NCU
v28 v29 3 3 4 4 NCU observations -- processing needs to be finished by early morning for MOPS to use for NEO 5 * obs_mode NCU -- new, possibly start Oct. 2 42016 HST (not start after 11/1 HST?)5 * obs_mode NCU -- new, possibly start Oct. 25 2016 HST (not start after 11/1 HST?) 6 6 * 4 chunks (returns) -- NCU1B...NCU4B; NCU1A...NCU4A -- see night sequence below 7 7 * B=color set, sequence wgwrwiwz per chunk (8 targets each) … … 9 9 * 8 targets (fields) -- NCU1.1...NCU1.8; NCU2.1...NCU2.8; NCU3.1...NCU3.8 ; NCU4.1...NCU4.8 (chunk#.target#) 10 10 * 4 above, 4 below ecliptic centered around opposition -- all fields non-overlapping 11 * [[html(<span style="color:red">[sc] targets don't overlap? (iirc)</span>)]] 12 * no dither in a night (no rotation like MD/QUB/FSS) -- offset done each night mean motion retrograde (W along ecliptic, ~0.3 degrees?) 11 * no dither in a night (no rotation like MD/QUB/FSS) -- offset done each night mean motion retrograde (W along ecliptic, ~0.3 degrees/night) 13 12 * exposures per field -- 14 13 * griz -- … … 20 19 * exptimes wgriz: 60,120,120,120,180s 21 20 * run queue sequence then 22 * night 1: NCU1B, NCU 2B, NCU3B, NCU4B -- only color queue -- return to exact position (no dither)23 * night 2: NCU 1A, NCU2A, NCU3A, NCU4A -- only w queue -- return each time to position (no dither), but has target/field center motion shift from previous night24 * night 3: NCU 1A, NCU2A, NCU3A, NCU4A -- only w queue -- return each time to position (no dither), but has target/field center motion shift from previous night25 * night 4: NCU 1A, NCU2A, NCU3A, NCU4A -- only w queue -- return each time to position (no dither), but has target/field center motion shift from previous night26 * chunk name (QUEUEID keyword in headers) -- may want to add filter here like in comment string below21 * night 1: NCU1B, NCU1B, NCU1B, NCU1B -- only color queue -- return to exact position (no dither) 22 * night 2: NCU2A, NCU2A, NCU2A, NCU2A -- only w queue -- return each time to position (no dither), but has target/field center motion shift from previous night 23 * night 3: NCU3A, NCU3A, NCU3A, NCU3A -- only w queue -- return each time to position (no dither), but has target/field center motion shift from previous night 24 * night 4: NCU4A, NCU4A, NCU4A, NCU4A -- only w queue -- return each time to position (no dither), but has target/field center motion shift from previous night 25 * chunk name (QUEUEID keyword in headers) -- added filter with vitit# here 27 26 * target name (OBJECT keyword in headers) -- NCU(chunk#.target#) -- may be enough to trigger for WW per chunk (8 visits) rather than all night (32 visits), may also trigger .multi case too over the multiple nights... 28 27 * comment string -- needs to be descriptive and easily parsed, need to check with MOPS for WS grouping/parsing issues (spaces) -- 29 28 * B: old format: NCU1B w4 NCU1.1 -- (chunk) (filter)(visit# for w only) NCU(chunk#.target#) -- want to add "visit #" at end of comment (1-4), move filter to chunk name 30 * B: new format: (chunk).(filter) NCU(chunk#.target#) visit (visit # in chunk) 29 * ~~B: new format: (chunk).(filter) NCU(chunk#.target#) visit (visit # in chunk)~~ -- adding visit# at end not easily done in B chunk so put on chunk name instead 30 * B: revised new format: (chunk).(filter).(visit# in chunk) NCU(chunk#.target#) 31 * NCU1B only night 1 -- 31 32 {{{ 32 NCU1B.w NCU1.1 visit133 NCU1B.w NCU1.2 visit 133 NCU1B.w.1 NCU1.1 34 NCU1B.w.1 NCU1.2 34 35 ... 35 NCU1B.w NCU1.8 visit 136 NCU1B.g NCU1.1 visit137 NCU1B.g NCU1.2 visit 136 NCU1B.w.1 NCU1.8 37 NCU1B.g.1 NCU1.1 38 NCU1B.g.1 NCU1.2 38 39 ... 39 NCU1B.g NCU1.8 visit 140 NCU1B.w NCU1.1 visit 241 NCU1B.w NCU1.2 visit240 NCU1B.g.1 NCU1.8 41 NCU1B.w.2 NCU1.1 42 NCU1B.w.2 NCU1.2 42 43 ... 43 NCU1B.w NCU1.8 visit 244 NCU1B.r NCU1.1 visit144 NCU1B.w.2 NCU1.8 45 NCU1B.r.1 NCU1.1 45 46 ... 46 NCU1B.r NCU1.8 visit 147 NCU1B.w NCU1.1 visit 347 NCU1B.r.1 NCU1.8 48 NCU1B.w.3 NCU1.1 48 49 ... 49 NCU1B.w NCU1.8 visit 350 NCU1B.i NCU1.1 visit150 NCU1B.w.3 NCU1.8 51 NCU1B.i.1 NCU1.1 51 52 ... 52 NCU1B.i NCU1.8 visit 153 NCU1B.w NCU1.1 visit 453 NCU1B.i.1 NCU1.8 54 NCU1B.w.4 NCU1.1 54 55 ... 55 NCU1B.w NCU1.8 visit 456 NCU1B.z NCU1.8 visit 156 NCU1B.w.4 NCU1.8 57 NCU1B.z.1 NCU1.8 57 58 ... 58 NCU1B.z NCU1.8 visit 1 59 60 61 NCU2B.w NCU2.1 visit 1 62 ... 63 NCU2B.w NCU2.8 visit 1 64 ... 65 NCU2B.z NCU2.8 visit 1 66 67 68 NCU3B.w NCU3.1 visit 1 69 ... 70 NCU3B.w NCU3.8 visit 1 71 ... 72 NCU3B.z NCU3.8 visit 1 73 74 75 NCU4B.w NCU4.1 visit 1 76 ... 77 NCU4B.w NCU4.8 visit 1 78 ... 79 NCU4B.z NCU4.8 visit 1 59 NCU1B.z.1 NCU1.8 80 60 81 61 }}} 82 62 * A: old format: NCU1A NCU1.1 -- (chunk) NCU(chunk#.target#) -- add "w" so similar format for B and "visit #" at end of comment (1-8), move filter to chunk name 83 * A: new format make similar: (chunk).(filter) NCU(chunk#.target#) visit (visit # in chunk) 63 * ~~A: new format make similar: (chunk).(filter) NCU(chunk#.target#) visit (visit # in chunk) ~~ visit (visit#) possible using otis repeat(?) but not for B so don't use 64 * A: revised format also make similar: (chunk).(filter).(visit# in chunk) NCU(chunk#.target#) 84 65 {{{ 85 NCU1A.w NCU1.1 visit 1 86 NCU1A.w NCU1.2 visit 1 87 NCU1A.w NCU1.3 visit 1 66 Night 2: 67 68 NCU2A.w.1 NCU1.1 69 NCU2A.w.1 NCU1.2 70 NCU2A.w.1 NCU1.3 88 71 ... 89 NCU 1A.w NCU1.1 visit 290 NCU 1A.w NCU1.2 visit272 NCU2A.w.2 NCU1.1 73 NCU2A.w.2 NCU1.2 91 74 ... 92 NCU 1A.w NCU1.1 visit 875 NCU2A.w.8 NCU1.1 93 76 ... 94 NCU1A.w NCU1.8 visit 8 77 NCU2A.w.8 NCU1.8 78 79 Night 3: 80 81 NCU3A.w.1 NCU3.1 82 NCU3A.w.1 NCU3.2 83 ... 84 NCU3A.w.8 NCU3.8 85 86 Night 4: 87 88 NCU4A.w.1 NCU4.1 89 NCU4A.w.1 NCU4.2 90 ... 91 NCU4A.w.8 NCU4.8 92 93 }}} 94 95 * '''Keyhole problem''' -- ecliptic nearly overhead for opposition (~same next lunation on other side), then opposition near plane of galaxy, then similar position on other side of galaxy, so maybe 5 months if not done in October 96 * Night 1: (B chunk is color set) -- 97 {{{ 98 do one whole B chunk, 99 then half a B chunk (which will be w,g,w,r), 100 then do something else for about 75 minutes until it comes out of the keyhole, 101 and do the rest of that chunk (which will be w,i,w,z), 102 then do the final whole B chunk. 103 }}} 104 * Nights 2-4: (A chunk all w) -- do A twice, then intermission for the keyhole, then do it twice again 95 105 96 106 97 NCU2A.w NCU2.1 visit 198 NCU2A.w NCU2.2 visit 199 ...100 NCU2A.w NCU2.8 visit 8101 102 ...103 }}}104 105 107 * chip->warp normal stdscience pantasks as with xSS products (ipp-20141024 tag) 106 * WSdiff runs in ~ippmops/stdscience_ws with modified ipp-20141024 ops tag to make uncompressed WSdiffs (like FSS )108 * WSdiff runs in ~ippmops/stdscience_ws with modified ipp-20141024 ops tag to make uncompressed WSdiffs (like FSS, PSNSC_MOPS) 107 109 * larger volume, may need more nodes -- must be uncompressed for MOPS stamps since large data volume 108 110 * default detection @5-sigma while WWdiff is <5-sigma and may need to be adjusted -- default plan will be at normal 5-sigma level unless tuning agreed upon to a lower level (test chunk to be evaluated for this still) -- MD not a great equal comparison since exptimes were longer (but was also 5-sigma) 109 111 110 [[html(<span style="color:red">[sc] Not sure if the catalog size is not going to explode if threshold too small. WWdiff is at 2.5-sigma(not sure?). What was the value for WSdiffs e.g. MDs? We could use the values (in each rgiz) unless something has changed. Not sure about what's used for w-band.</span>)]]112 * [[html(<span style="color:red">[sc] Not sure if the catalog size is not going to explode if threshold too small. WWdiff is at 2.5-sigma(not sure?). What was the value for WSdiffs e.g. MDs? We could use the values (in each rgiz) unless something has changed. Not sure about what's used for w-band.</span>)]] 111 113 112 114 * WSdiff published to IPP-MOPS-WS -- WWdiff possibly later to regular datastore IPP-MOPS-TEST 115 * normal OSS warp-warp diffims will go to IPP-MOPS-TEST during the night 113 116 114 [[html(<span style="color:red">[sc] Please publish WSdiff to IPP-MOPS-WS</span>)]]117 * [[html(<span style="color:red">[sc] We will likely request WWdiff to be published to the regular datastore (IPP-MOPS-TEST) later (manual queuing/processing at the IPP level OK for MOPS)</span>)]] 115 118 116 [[html(<span style="color:red">[sc] We will likely request WWdiff to be published to the regular datastore (IPP-MOPS-TEST) later (manual queuing/processing at the IPP level OK for MOPS)</span>)]]117 119 118 Processing products -- czar needs to make sure all faults and issues are cleared daily (early morning) for NCU.* products120 Processing products -- Mark taking responsibility for 119 121 * chip-cam-warp -- NCU.nightlyscience -- nightly 120 122 * warp-stack diffims -- NCU.WS.nightlyscience -- nightly 121 * publishing of WSdiffs -- (need datastore location to use from MOPS)-- nightly122 * manually done -- not czar responsibility123 * publishing of WSdiffs -- IPP-MOPS-WS -- nightly 124 * manually done -- 123 125 * nightly stack -- NCU.nightlyscience -- nightly, manually done to avoid any poor images 124 126 * stack-stack diffims -- NCU.SS.nightlyscience -- nightly, likely automatic possible once manual stacks made 127 * warp-warp diffims -- TBD -- NCU.nightlyscience 125 128 * stamps -- TBD 126 129 127 130 128 131 Cleanup -- 129 * chip -- 1 day?130 * warp -- 2 days (no specific requirement?)131 * WSdiff -- 1 day? uncompressed so likely will need to limit and just do updates for stamps132 * publishing WSdiff catalog bundles -- 133 * pstamp -- manual cleanup like other ps_ud_% 132 * chip -- likely normal 1 day retention 133 * warp -- indefinite hold 134 * WSdiff -- indefinite hold -- uncompressed so will need to keep eye on and cleanup once MOPS is finished 135 * publishing WSdiff catalog bundles -- TBD/auto? 136 * pstamp -- manual cleanup like other ps_ud_% -- TBD 134 137 135 138 136 139 Tasks to do -- priority order now 137 140 * verify w-band stacks okay along ecliptic for opposition end October -- new reference stacks to be made for the expected area (mix of reprocessing+updates to minimize data load) 138 * WSdiff detection threshold -- 5-sigma default but may need to lower ?141 * WSdiff detection threshold -- 5-sigma default but may need to lower -- preliminary tests shows default 5-sigma should recover WWdiff threshold ~3.5 in MOPS test chunk 139 142 * MOPS interested in SSdiff -- published where? 140 143 * check processing rate/nodes for WSdiffs -- during the night of observations -- likely the normal 3x c2 group is sufficient but will have some nodes available to boost as needed (more c2 and/or m0+1, x3?)
