IPP Software Navigation Tools IPP Links Communication Pan-STARRS Links

Changes between Version 619 and Version 620 of MD.GR0


Ignore:
Timestamp:
Feb 5, 2013, 7:00:39 PM (13 years ago)
Author:
Mark Huber
Comment:

--

Legend:

Unmodified
Added
Removed
Modified

  • MD.GR0

    v619 v620  
    1010http://ps1sc.ifa.hawaii.edu/PS1wiki/index.php/Medium_Deep_Schedule. After the July, August, September time crunch of a new field a month, there should be time to continue reprocessing and creation of new reference stacks in advance.
    1111
    12 Status:
     12
     13Where to find the data:
     14 * http://datastore.ipp.ifa.hawaii.edu/ps1-md-GR/  (full: pixel images of reprocessed nightly data products like nightly stacks)
     15 * http://datastore.ipp.ifa.hawaii.edu/ps1-md-GR-cat/  (cleaned: catalogs of reprocessed nightly data products like camera SMFs)
     16 * http://datastore.ipp.ifa.hawaii.edu/ps1-md-refstack/  (full: pixel images for updated refstacks and reftest stacks)
     17 * http://datastore.ipp.ifa.hawaii.edu/ps1-md-staticsky/  (cleaned: new staticsky catalogs run on the refstacks typically but may have others like from deep stacks)
     18 * http://datastore.ipp.ifa.hawaii.edu/ps1-deeptest/  (full: pixel images of deep stack tests)
     19 * Postage Stamp Server: 
     20
     21
     22== Status ==
    1323 * ''8/15/11'' minimum mode re-processing for reference stacks after MD10 for evaluation and get next MD field ready (MD01)
    1424 * ''9/30/11'' more success weaving in reprocessing with LAP, started reprocessing for refstacks MD03,04 (with more reprocessed exposures)
    … …  
    2131 * ''12/20/11'' MD06 processing recoverable, refstacks proceeding.  MD08 a mess and will need to be re-run.
    2232 * ''12/31/11'' MD06 finally finished, distributed.
    23  *
     33 
    2434 * ''1/09/12'' MD08, 09 on hold while finishing up full reprocessing and release of V3 products now (i.e., nightly stacks, SMFs, SSdiffs).
    2535 * ''1/11/12'' MD06,04 full camera SMFs on datastore, MD07 later today. MD04 set has some duplicates with both unmagicked SMFs (data_group=MD04.GR0, sub-sample of ~150/filter best seeing) and magicked SMFs (data_group=condor_MD04.V3_01_dg, full sample of ~400/filter loaded into PSPS Nov. 2011)
    … …  
    4454 * 12/06 [wiki:MD05.refstack.20121202] to replace 2011 season refstack
    4555 * 12/21 [wiki:MD06.refstack.20121221] to replace 2011 season refstack
    46  *
     56 
    4757 * 01/09/13 [wiki:MD07.refstack.20130102] to replace 2011 season refstack
    4858 * 01/14 re-running all staticsky+skycal stack photometry on ref+deepstack to incorporate improvements/fixes to Kron code for loading into PSPS soon
    … …  
    5161
    5262
    53 *** '''Feedback is important:''' We would not want to continue full reprocessing of the MD fields if there has been a gross oversight or problem for a particular science interest. Please let us know of any issues you discover with example details. For example
     63== Feedback is important ==
     64We would not want to continue full reprocessing of the MD fields if there has been a gross oversight or problem for a particular science interest. Please let us know of any issues you discover with example details. For example
    5465
    5566 * tessellation and/or offsets issues in any of the fields
    … …  
    6374   * MD10.refstack.20110722 y-band skycell.048 had rejection problem, fixed with MD10.refstack.20110821
    6475 * MD10.refstack.20110722 replacement with proper uncompressed MD10.refstack.20110821 for diffims
    65  * with magic turned off now, MD04+MD06 have 2+ warp inputs for nightly stack creation to test how well boundaries come out and has fault/quality issue with 2-3 input cases showing some inputs have incorrect PSF size values measured/passed. rare (~<1%) but should be looked into (ie stack_id=609561)
    66  * adding quality column to ippMonitor for evaluating missing stack skycells would be useful to have on ''stack processed skyfile''
     76 * with magic turned off now (01/2012), MD04+MD06 have 2+ warp inputs for nightly stack creation to test how well boundaries come out and has fault/quality issue with 2-3 input cases showing some inputs have incorrect PSF size values measured/passed. rare (~<1%) but should be looked into (ie stack_id=609561)
     77 * adding a quality column to ippMonitor for evaluating missing stack skycells would be useful to have on ''stack processed skyfile''
    6778 
    6879DRAVG analysis/feedback
    … …  
    7586
    7687
    77 Where to find the data:
    78  * http://datastore.ipp.ifa.hawaii.edu/ps1-md-GR/  (full: pixel images of reprocessed nightly data products like nightly stacks)
    79  * http://datastore.ipp.ifa.hawaii.edu/ps1-md-GR-cat/  (cleaned: catalogs of reprocessed nightly data products like camera SMFs)
    80  * http://datastore.ipp.ifa.hawaii.edu/ps1-md-refstack/  (full: pixel images for updated refstacks and reftest stacks)
    81  * http://datastore.ipp.ifa.hawaii.edu/ps1-md-staticsky/  (cleaned: new staticsky catalogs run on the refstacks typically but may have others like from deep stacks)
    82  * http://datastore.ipp.ifa.hawaii.edu/ps1-deeptest/  (full: pixel images of deep stack tests)
    83 
    84 
    85 = MD.V3 Tessellation =
     88
     89
     90
     91== MD.V3 Tessellation ==
    8692The new V3 tessellations are defined at [wiki:PS1_Tessellations] with the following general properties:
    8793
    … …  
    99105
    100106
    101 = Reference versus Deep Stack =
     107== Reference versus Deep Stack ==
    102108A requirements description for the needs in the different types of stacks would be useful to summarize.
    103109
    … …  
    262268
    263269
    264 = Summary Results Posters at PS1SC 2012 Durham Meeting =
     270= Summary Stack Result Posters at PS1SC 2012 Durham Meeting =
    265271
    266272Posters include summary map plots of psf_major(minor), 5-sig photometric depth and magnitude @50% recovery of 500 added fakes as well as full field, i-band images of the MD refstacks
    … …  
    276282|| '''MD''' || '''Reference Stack''' || '''Deep Stack''' || '''Night Stack''' || '''Refstack Phot''' || '''Deep Phot''' || '''Night Phot''' || '''Camera/SMFs'''  || '''for PSPS''' ||
    277283|| || || || || || || || || ||
    278 || 01 || 20110820 (U) || x || done (mixed) || staticsky+skycal 20120805 || x || x || done (mixed) || SMF+ref phot TBD ||
    279 || 02 || 20110907 (U) || x || done (mixed) || staticsky+skycal 20120805 || x || x || done (mixed) || SMF+ref phot TBD ||
    280 || 03 || 20111010 (U) || x || done (mixed) || staticsky+skycal 20120805 || x || x || done (mixed) || SMF+ref phot TBD ||
    281 || 04 || 20111012 (U) || deeptest.20120727 || done || staticsky+skycal 20120805 || staticsky+skycal 20120806  || TBD || done || SMF+ref+deep+nightly phot TBD (secondary) ||
    282 || 05 || 20111014 (U) || x || done (mixed and full) || staticsky+skycal 20120805 || x || x || done (mixed) || SMF+ref phot TBD ||
    283 || 06 || 20111122 (U) || x || done || staticsky+skycal 20120805 || x || x || done || SMF+ref phot TBD ||
    284 || 07 || 20111106 (U) || x || done || staticsky+skycal 20120805 || x || x || done || SMF+ref phot TBD ||
    285 || 08 || 20120422 || x || done || staticsky+skycal 20120805 || x || x || done || SMF+ref phot TBD ||
    286 || 09 || 20120603 (replaced by 20120831) || deeptest.20120727 || done  || staticsky+skycal 20120805 || staticsky+skycal 20120806  || done || done || SMF+ref+deep+nightly phot  (priority) ||
    287 || 10 || 20110821 (U) || x || done || staticsky+skycal 20120805 || x || x || done || SMF+ref phot TBD ||
     284|| 01 || 20110820 (U) || x || done (mixed) || staticsky+skycal 20120805 || x || x || done (mixed) || SMF+ref phot ''TBD'' ||
     285|| 02 || 20110907 (U) || x || done (mixed) || staticsky+skycal 20120805 || x || x || done (mixed) || SMF+ref phot ''TBD'' ||
     286|| 03 || 20111010 (U) || x || done (mixed) || staticsky+skycal 20120805 || x || x || done (mixed) || SMF+ref phot ''TBD'' ||
     287|| 04 || 20111012 (U) || deeptest.20120727 || done || staticsky+skycal 20120805 || staticsky+skycal 20120806  || TBD || done || SMF+ref+deep+night phot ''TBD'' (secondary) ||
     288|| 05 || 20111014 (U) || x || done (mixed and full) || staticsky+skycal 20120805 || x || x || done (mixed) || SMF+ref phot ''TBD'' ||
     289|| 06 || 20111122 (U) || x || done || staticsky+skycal 20120805 || x || x || done || SMF+ref phot ''TBD'' ||
     290|| 07 || 20111106 (U) || x || done || staticsky+skycal 20120805 || x || x || done || SMF+ref phot ''TBD'' ||
     291|| 08 || 20120422 || x || done || staticsky+skycal 20120805 || x || x || done || SMF+ref phot ''TBD'' ||
     292|| 09 || 20120831 (R) || deeptest.20120727 || done  || staticsky+skycal 20120805 || staticsky+skycal 20120806  || done || done || SMF+ref+deep+night phot  (primary) ||
     293|| 10 || 20110821 (U) || x || done || staticsky+skycal 20120805 || x || x || done || SMF+ref phot ''TBD'' ||
    288294|| 11 || not to be reprocessed in MD.GR0
    289295|| || || || || || || || || ||
    290296|| || || || || || || || || ||
    291297|| || || || || || || || || ||
    292 || M31 (add to MD.GR0?) ||   ||  x  ||  ?  ||  if made  || ?  || ?  || ?  || ? ||
    293 || STS (add to MD.GR0?)  ||  20120531 (MOPS needs revision) ||  x  || ?  || if made  ||  ? || ?  || ?  || ? ||
     298|| M31 -- was brief discussion to add to MD.GR0 but looks like not
     299|| STS -- was brief discussion to add to MD.GR0  ||  20120531 (MOPS needs revision) ||  x  || ?  || ?  ||  ? || ?  || ?  || ? ||
    294300|| || || || || || || || ||
    295301
    296302Comments:
    297303 * TBD=To Be Done/Decided by priority, X=not planned,  ?=unknown at this time
    298  * the date entries are part of the data_group name to access via the PSS or Datastore, i.e., MD01.refstack.20110820, MD09.deeptest.20120727
    299  * (U) updated -- new 2012/2013 versions exist for these refstack now but staticsky/skycal not necessarily available,
    300  * finish reprocessing all exposures to V3 for warp/nightly stack for MD01+02+03+05 =840+609+1286+1115 = 3850 (2-3 full days of --only-- MD processing)
     304 * 20YYMMDD date entries are part of the data_group name to access via the PSS or Datastore, i.e.,
     305  * MD01.refstack.20110820, MD09.deeptest.20120727 stacks
     306  * staticsky and skycal respectively
     307 * (U) updated -- new 2012/2013 versions exist for these refstack now (see [wiki:MediumDeepFields] but staticsky/skycal not necessarily available for updated version unless going to be used for PSPS.
     308 * (R) replaced -- MD09.refstack.20120603,
     309 * finish reprocessing all exposures to V3 for warp/nightly stack for MD01+02+03+05 =840+609+1286+1115 = 3850 (2-3 full days of --only-- MD processing) -- done
    301310  * MD05 mixed -- old and new processing, so magic mix and photometry catalog mix. see specific section for detail.
    302  * out-of-season/anti-yearly refstacks, yearly refstack: on hold for MD.GR1 (would need to update/reprocess large numbers of exposures to do).
    303  * iq/psf stack on hold for MD.GR1 (like out-of-season, would required many updates) -- should do for MD04,MD09 and requires development/code time.
    304  * deep stack for MD04,09 for MD.GR0 only, all others on hold for MD.GR1 (many many updates..).
    305  * ubercal of pixels start with MD08 in the date sequence, finished processing of MD01,02,03,05 exposures into V3 will also be ubercal'd (including reprocessing of any needed to make the complete nightly stacks)
     311  *
     312 * stacks
     313  * out-of-season/anti-yearly refstacks, yearly refstack: on hold for MD.GR1 (would need to update/reprocess large numbers of exposures to do).
     314  * iq/psf stack on hold for MD.GR1 (like out-of-season, would required many updates) -- should do for MD04,MD09 and requires development/code time.
     315  * deep stack for MD04,09 for MD.GR0 only, all others on hold for MD.GR1 (many many updates..).
     316 * reminder that ubercal of pixels start with MD08 in the date sequence, finished processing of MD01,02,03,05 exposures into V3 will also be ubercal'd (including reprocessing of any needed to make the complete nightly stacks)
    306317
    307318Questions:
    … …  
    314325
    315326----
     327
    316328= MD10.GR0 =
    317329