Changes between Version 18 and Version 19 of MDdeeptest_pixrej_20140403
- Timestamp:
- May 7, 2014, 10:24:39 PM (12 years ago)
Legend:
- Unmodified
- Added
- Removed
- Modified
-
MDdeeptest_pixrej_20140403
v18 v19 9 9 A related issue also to be addressed is the ratty edge of the MD stacks due to poorer edge chips and smaller number of input pixels. Possible alternatives for a solution are 10 10 * hard cut by radius field by field and filter -- difficult to implement 11 * combine MD+3PI to help improve by including better 3PI pixels -- non-trivial setup and unknown config s for both types of data sets11 * combine MD+3PI to help improve by including better 3PI pixels -- non-trivial setup and unknown config behavior for both types of data sets (MD and 3PI) 12 12 * only include pixel if N input > some minimum, similar to the SAFE (N=2) option but with a larger N minimum -- edge may vary greatly between filters 13 13 * often edge is dominated by SUSPECT mask pixels, set those to bad and reject -- potential to impact entire skycell (may be good since SUSPECT may actually be bad) 14 14 * any others options? 15 15 16 The bugs found will apply to all stacks (MD and 3PI). 17 18 Changes are noted in the [wiki:MDdeeptest_pixrej_20140403#ConfigurationMods] section for the MD deep stacks only (so far, suspect will be necessary for refstacks but unclear how night stacks will configured as of yet). 19 16 20 17 21 ---- … … 19 23 = Example Junk and Bugs = 20 24 21 Large regions of high level, bad pixels ( md04s065i -- diffract spike chunks (1129,4256), the moth ( ) ) -- bug25 Large regions of high level, bad pixels ( md04s065i -- diffract spike chunks (1129,4256), the moth (2193,2062) ) -- bug 22 26 23 27 || img || mk || wt || num || exp || expwt || … … 37 41 [[Image(md04s065i_stackjunk_crlike_2_ds9.jpg,300px)]] 38 42 39 * due to bug, showing up after NAN/0 bug fixed, sigma sometimes being set to a negative value resulting in final mean and sigma going NAN (then all inputs again used...)40 41 42 Large region of low level, poor/bad pixels ( md04s013g ( ) ) -- bug? much can be reduced by config mods43 * due to bug, showing up more after NAN/0 bug fixed, sigma sometimes being set to a negative value resulting in final mean and sigma going NAN (then all inputs again used...) 44 45 46 Large region of low level, poor/bad pixels ( md04s013g (1090,5600), md09s013g (1090,5600) ) -- bug? much can be reduced by config mods 43 47 44 48 || img || mk || num || wt || … … 46 50 [[Image(md04s013g_stackjunk_1_ds9.jpg,600px)]] 47 51 48 * greatly reduced by lowering THRESHOLD.MASK 0.5->0.01 (may need further modifications) 49 50 51 Edge junk and '' pomegranate-like webbing'' ( md04s013g (), md09s012g extensive () ) -- bug? much can be reduced by config mods and added N input minimum per pixel (NMINPIX)52 * greatly reduced by lowering THRESHOLD.MASK 0.5->0.01 (may need further modifications) or reject a set of entire inputs as poor/bad if can be identified. 53 54 55 Edge junk and ''webbing'' ( md04s013g (800,2850) ) -- bug? much can be reduced by config mods and added N input minimum per pixel (NMINPIX) 52 56 53 57 || img || mk || num || wt || … … 56 60 57 61 * webbing greatly reduced by lowering THRESHOLD.MASK 0.5->0.3 (may need further modifications, 0.01 even better) 58 * edge junk also cleaned up by rejecting pixels with <20 inputs (NMINPIX 20) 62 * edge junk also cleaned up by rejecting pixels with <20 inputs (NMINPIX 20). Based on the large rise of variance values still, could probably be increased 30-50 inputs generally. 59 63 60 64 [[Image(md04s013g_stackjunk_edgepom_2_n20thr03_ds9.jpg,600px)]] … … 62 66 63 67 64 65 Other case of ''pomegranate-like webbing'' (md09s ) -- different cause, appears to be incomplete rejection of bright, scattered light. Bug? Lower threshold (THRESHOLD.MASK) helps, but likely need to reject a set of entire inputs as poor/bad. 68 Other case of ''webbing'' (md09s012g (1421,4850) ) -- different cause, appears to be incomplete rejection of bright, scattered light. Bug? Lower threshold (THRESHOLD.MASK) helps, but likely need to reject a set of entire inputs as poor/bad. 66 69 67 70 [[Image(.jpg,600px)]] 68 71 69 72 70 Setting SUSPECT input pixels as bad -- 71 72 [[Image(.jpg,600px)]] 73 74 Other streak-like features still (md09s00g () ) -- another bug? Lower threshold (THRESHOLD.MASK) helps but not always. 73 74 75 Other streak-like features still (md09s046g (3235,3368) ) -- suspect another bug. Lower threshold (THRESHOLD.MASK) helps but not completely. 76 77 [[Image(md09s046g_suspectbadstreakthres03to01_x3235y3368.jpg,600px)]] 75 78 76 79 77 80 Edge of skycell junk -- strip along skycell with fair amount of junk ignored, kernel size (all stacks have this problem) 78 81 * KERNEL.SIZE=20 pixels, MD.V3 skycell overlap is 320 pixels -- this area should just be entirely MASKED 82 83 [[Image(md09s046g_badbordermask_x6387y2110.jpg,600px)]] 84 85 86 87 Setting SUSPECT input pixels as bad -- general input loss per pixel ~1-5% over the field but more often than not with shapes that are clearly poor (i.e. ghost rings) 79 88 80 89 [[Image(.jpg,600px)]] … … 126 135 127 136 128 __MD09 example as a shifted center field case__ -- pre-test showed a large amount of junk, so also set SUSPECT input pixels as BAD pixels, edge properties unknown so keep pixels N>10 inputs (NMINPIX)137 __MD09 example as a shifted center field case__ -- [wiki:MD09.PV1_deepstack_tests] and manual re-test showed a large amount of junk, so also set SUSPECT input pixels as BAD pixels, edge properties unknown so keep pixels N>10 inputs (NMINPIX) 129 138 * label: MD09.cut2009n10suspectbaddeeptest.20140427 -- g-band only, full PV2 set after 6/2010 like cut2009 130 139 * THRESHOLD.MASK 0.3 … … 185 194 * THRESHOLD.MASK lower to 0.1 186 195 187 New options added :196 New options added (defaults): 188 197 {{{ 189 NMINPIX S32 30# Minimum input per pixel190 MASK.BLANKBORDER S32 20 # Mask blank border in final stack output198 NMINPIX S32 1 # Minimum input per pixel 199 MASK.BLANKBORDER S32 0 # Mask blank border in final stack output 191 200 }}} 192 201 * NMINPIX 30 appears to be a reasonable starting value, may be better if 40-50. The edges of the MD field is poor, 3PI overlap in that area will often be better.
