IPP Software Navigation Tools IPP Links Communication Pan-STARRS Links

Changes between Version 23 and Version 24 of PS1_IPP_Czarlog_20190510


Ignore:
Timestamp:
May 14, 2019, 4:46:19 PM (7 years ago)
Author:
tdeboer
Comment:

--

Legend:

Unmodified
Added
Removed
Modified

  • PS1_IPP_Czarlog_20190510

    v23 v24  
    225225 I made a quick inventory of the predominant labels that have large (>100) numbers of wait state chip exps:
    226226{{{
    227 +--------+-----------------------------------+-----------------------------------+
    228 | waitct | data_group                        | label                             |
    229 +--------+-----------------------------------+-----------------------------------+
     227|     97 | DS.REPAIR.clean                   | ThreePi.20111112                  |
     228|    104 | EUCLID.MD08.PV0.T1                | EUCLID.MD08.PV0.T1                |
    230229|    104 | EUCLID.MD07.PV0.T1                | EUCLID.MD07.PV0.T1                |
    231 |    104 | EUCLID.MD08.PV0.T1                | EUCLID.MD08.PV0.T1                |
    232230|    106 | NULL                              | NULL                              |
    233231|    106 | altaz.20080920                    | altaz.20080920                    |
    234 |    109 | LAP.PV3.20140730.20140918         | goto_cleaned.broke190323          |
    235 |    114 | dropped                           | STS.rp.2013.hold                  |
     232|    109 | goto_cleaned.broke190323          | LAP.PV3.20140730.20140918         |
     233|    114 | STS.rp.2013.hold                  | dropped                           |
    236234|    117 | scitest.200807                    | scitest.200807                    |
    237 |    122 | QUB.20160517                      | QUB.PS16cgx                       |
     235|    122 | QUB.PS16cgx                       | QUB.20160517                      |
     236|    141 | flatcorr.20100124                 | flatcorr.20100124                 |
    238237|    141 | flatcorr.20081109                 | flatcorr.20081109                 |
    239 |    141 | flatcorr.20100124                 | flatcorr.20100124                 |
    240238|    142 | flattest.20091010                 | flattest.20091010                 |
    241 |    142 | MD03.20091228                     | MD03.nightlystack                 |
     239|    142 | MD03.nightlystack                 | MD03.20091228                     |
    242240|    142 | flatcorr.20091003                 | flatcorr.20091003                 |
    243 |    180 | PV2.PHOTO.STD.20141028.i          | PV2.PHOTO.STD.20141028            |
     241|    180 | PV2.PHOTO.STD.20141028            | PV2.PHOTO.STD.20141028.i          |
    244242|    180 | flatcorr.20081107                 | flatcorr.20081107                 |
    245 |    187 | Bright3Pi.20190402                | Bright3Pi.nightlyscience          |
     243|    187 | Bright3Pi.nightlyscience          | Bright3Pi.20190402                |
    246244|    203 | flatcorr.20090509                 | flatcorr.20090509                 |
    247245|    205 | MD07.20091123                     | MD07.20091123                     |
    248 |    209 | OSS.20141207                      | goto_cleaned.broke190312          |
    249 |    214 | opposition.20080922               | cleanup_testing_old               |
     246|    209 | goto_cleaned.broke190312          | OSS.20141207                      |
     247|    214 | cleanup_testing_old               | opposition.20080922               |
    250248|    217 | ThreePi.Run2.z.v0                 | ThreePi.Run2.z.v0                 |
    251 |    231 | mops.fixbrokenLAP.20180323        | mopsfix                           |
     249|    231 | mopsfix                           | mops.fixbrokenLAP.20180323        |
     250|    248 | lanl.SAS.201402114                | czw.SAS.201402114                 |
    252251|    248 | SAS.20151208                      | SAS.20151208                      |
    253 |    248 | czw.SAS.201402114                 | lanl.SAS.201402114                |
    254 |    264 | ThreePi.Run2.i.v0                 | ThreePi.Run2.i.v0                 |
    255 |    282 | ThreePi.yz.20091203               | ThreePi.yz.20091203               |
    256 |    304 | m31.ps1.20170627                  | m31.ps1.20170627.wait             |
    257 |    365 | ThreePi_NorthernRegion.090729     | ThreePi_NorthernRegion.090729     |
    258 |    573 | altaz.20080920                    | goto_purged                       |
    259 |    596 | LAP.PV3.20140730.20141008         | goto_cleaned.broke                |
     252|    573 | goto_purged                       | altaz.20080920                    |
     253|    586 | goto_cleaned.broke                | LAP.PV3.20140730.20141008         |
    260254|    641 | scitest.20090327                  | scitest.20090327                  |
    261 |    788 | LAP.PV3.20140730.20150117         | update.LAP.PV3                    |
    262 |    967 | LAP.PV3.20140730.20140927         | goto_cleaned.redoitchold          |
    263 |   1252 | m31.ps1.20170627                  | m31.ps1.20170627                  |
    264 |   1756 | LAP.PV3.20140730.20140807         | PV3.cleanup                       |
    265 |   2239 | ThreePi.20140126                  | goto_cleaned.redoitc              |
     255|    783 | update.LAP.PV3                    | LAP.PV3.20140730.20150118         |
     256|    927 | goto_cleaned.redoitchold          | LAP.PV3.20140730.20140927         |
     257|   1658 | PV3.cleanup                       | LAP.PV3.20140730.20140810         |
     258|   2239 | goto_cleaned.redoitc              | ThreePi.20140126                  |
    266259|   3360 | PV3.Pole.Reprocess.20180510       | PV3.Pole.Reprocess.20180510       |
    267 |  15431 | STS.rp.2013.20120412              | STS.rp.2013                       |
    268 |  51293 | ThreePi.20100220                  | goto_cleaned.rerun                |
    269 | 166252 | scitest.200808                    | goto_cleaned_redo                 |
    270 | 443284 | altstrip.200806                   | goto_cleaned                      |
     260|  15431 | STS.rp.2013                       | STS.rp.2013.20120412              |
     261|  51293 | goto_cleaned.rerun                | ThreePi.20100220                  |
     262| 165642 | goto_cleaned_redo                 | scitest.200808                    |
     263| 442467 | goto_cleaned                      | altstrip.200806                   |
    271264+--------+-----------------------------------+-----------------------------------+
    272265}}}
    … …  
    763756 ->The testing made the alarm on one of the PDUs in that cabinet go off, indicating it is drawing too much power. To fight that, we turned machines ippx033-ippx036 back to off in the pantask since that machine is plugged to the offending PDU which has the alarm going off.
    764757
     758 ------------------------
     759
     760 Gene had some pstamp jobs stuck. Running the usual query:
     761{{{
     762pstamptool -dbname ippRequestServer -dbserver ipp113 -pendingdependent -simple
     763}}}
     764 shows a bunch of jobs:
     765{{{
     76610977292 new chip 1131144 XY71 gpc1 /data/ipp113.0/pstamp/work/2019/05/14/924082 ps_ud_WEB F 0 0 WEB 924082 /data/ipp113.0/pstamp/work/2019/05/14/924082/checkdep.10977292.log 320
     76710977293 new chip 1131146 XY16 gpc1 /data/ipp113.0/pstamp/work/2019/05/14/924082 ps_ud_WEB F 0 0 WEB 924082 /data/ipp113.0/pstamp/work/2019/05/14/924082/checkdep.10977293.log 320
     76810977294 new chip 1131208 XY71 gpc1 /data/ipp113.0/pstamp/work/2019/05/14/924082 ps_ud_WEB F 0 0 WEB 924082 /data/ipp113.0/pstamp/work/2019/05/14/924082/checkdep.10977294.log 320
     76910977295 new chip 1131210 XY16 gpc1 /data/ipp113.0/pstamp/work/2019/05/14/924082 ps_ud_WEB F 0 0 WEB 924082 /data/ipp113.0/pstamp/work/2019/05/14/924082/checkdep.10977295.log 320
     77010977296 new chip 1130544 XY71 gpc1 /data/ipp113.0/pstamp/work/2019/05/14/924082 ps_ud_WEB F 0 0 WEB 924082 /data/ipp113.0/pstamp/work/2019/05/14/924082/checkdep.10977296.log 320
     77110977297 new chip 1130549 XY71 gpc1 /data/ipp113.0/pstamp/work/2019/05/14/924082 ps_ud_WEB F 0 0 WEB 924082 /data/ipp113.0/pstamp/work/2019/05/14/924082/checkdep.10977297.log 320
     77210977298 new chip 1130551 XY16 gpc1 /data/ipp113.0/pstamp/work/2019/05/14/924082 ps_ud_WEB F 0 0 WEB 924082 /data/ipp113.0/pstamp/work/2019/05/14/924082/checkdep.10977298.log 320
     77310977299 new chip 1130546 XY16 gpc1 /data/ipp113.0/pstamp/work/2019/05/14/924082 ps_ud_WEB F 0 0 WEB 924082 /data/ipp113.0/pstamp/work/2019/05/14/924082/checkdep.10977299.log 320
     77410977304 new chip 1130556 XY61 gpc1 /data/ipp113.0/pstamp/work/2019/05/14/924082 ps_ud_WEB F 0 0 WEB 924082 /data/ipp113.0/pstamp/work/2019/05/14/924082/checkdep.10977304.log 320
     77510977305 new chip 1130560 XY61 gpc1 /data/ipp113.0/pstamp/work/2019/05/14/924082 ps_ud_WEB F 0 0 WEB 924082 /data/ipp113.0/pstamp/work/2019/05/14/924082/checkdep.10977305.log 320
     77610977312 new chip 1131230 XY12 gpc1 /data/ipp113.0/pstamp/work/2019/05/14/924082 ps_ud_WEB F 0 0 WEB 924082 /data/ipp113.0/pstamp/work/2019/05/14/924082/checkdep.10977312.log 320
     77710977313 new chip 1200205 XY12 gpc1 /data/ipp113.0/pstamp/work/2019/05/14/924082 ps_ud_WEB F 0 0 WEB 924082 /data/ipp113.0/pstamp/work/2019/05/14/924082/checkdep.10977313.log 320
     77810977314 new chip 1131246 XY12 gpc1 /data/ipp113.0/pstamp/work/2019/05/14/924082 ps_ud_WEB F 0 0 WEB 924082 /data/ipp113.0/pstamp/work/2019/05/14/924082/checkdep.10977314.log 320
     77910977315 new chip 1131251 XY12 gpc1 /data/ipp113.0/pstamp/work/2019/05/14/924082 ps_ud_WEB F 0 0 WEB 924082 /data/ipp113.0/pstamp/work/2019/05/14/924082/checkdep.10977315.log 320
     78010977318 new chip 1130562 XY10 gpc1 /data/ipp113.0/pstamp/work/2019/05/14/924082 ps_ud_WEB F 0 0 WEB 924082 /data/ipp113.0/pstamp/work/2019/05/14/924082/checkdep.10977318.log 320
     78110977319 new chip 1130564 XY10 gpc1 /data/ipp113.0/pstamp/work/2019/05/14/924082 ps_ud_WEB F 0 0 WEB 924082 /data/ipp113.0/pstamp/work/2019/05/14/924082/checkdep.10977319.log 320
     78210977332 new chip 1130568 XY47 gpc1 /data/ipp113.0/pstamp/work/2019/05/14/924082 ps_ud_WEB F 0 0 WEB 924082 /data/ipp113.0/pstamp/work/2019/05/14/924082/checkdep.10977332.log 320
     78310977333 new chip 1130570 XY47 gpc1 /data/ipp113.0/pstamp/work/2019/05/14/924082 ps_ud_WEB F 0 0 WEB 924082 /data/ipp113.0/pstamp/work/2019/05/14/924082/checkdep.10977333.log 320
     78410977336 new chip 1131257 XY66 gpc1 /data/ipp113.0/pstamp/work/2019/05/14/924082 ps_ud_WEB F 0 0 WEB 924082 /data/ipp113.0/pstamp/work/2019/05/14/924082/checkdep.10977336.log 320
     78510977337 new chip 1131260 XY66 gpc1 /data/ipp113.0/pstamp/work/2019/05/14/924082 ps_ud_WEB F 0 0 WEB 924082 /data/ipp113.0/pstamp/work/2019/05/14/924082/checkdep.10977337.log 320
     78610977338 new chip 1200216 XY21 gpc1 /data/ipp113.0/pstamp/work/2019/05/14/924082 ps_ud_WEB F 0 0 WEB 924082 /data/ipp113.0/pstamp/work/2019/05/14/924082/checkdep.10977338.log 320
     78710977339 new chip 1130573 XY21 gpc1 /data/ipp113.0/pstamp/work/2019/05/14/924082 ps_ud_WEB F 0 0 WEB 924082 /data/ipp113.0/pstamp/work/2019/05/14/924082/checkdep.10977339.log 320
     78810977342 new chip 1131276 XY63 gpc1 /data/ipp113.0/pstamp/work/2019/05/14/924082 ps_ud_WEB F 0 0 WEB 924082 /data/ipp113.0/pstamp/work/2019/05/14/924082/checkdep.10977342.log 320
     78910977343 new chip 1131280 XY63 gpc1 /data/ipp113.0/pstamp/work/2019/05/14/924082 ps_ud_WEB F 0 0 WEB 924082 /data/ipp113.0/pstamp/work/2019/05/14/924082/checkdep.10977343.log 320
     79010977344 new chip 1130575 XY47 gpc1 /data/ipp113.0/pstamp/work/2019/05/14/924082 ps_ud_WEB F 0 0 WEB 924082 /data/ipp113.0/pstamp/work/2019/05/14/924082/checkdep.10977344.log 320
     79110977345 new chip 1130578 XY47 gpc1 /data/ipp113.0/pstamp/work/2019/05/14/924082 ps_ud_WEB F 0 0 WEB 924082 /data/ipp113.0/pstamp/work/2019/05/14/924082/checkdep.10977345.log 320
     79210977348 new chip 1131297 XY47 gpc1 /data/ipp113.0/pstamp/work/2019/05/14/924082 ps_ud_WEB F 0 0 WEB 924082 /data/ipp113.0/pstamp/work/2019/05/14/924082/checkdep.10977348.log 320
     79310977349 new chip 1131303 XY47 gpc1 /data/ipp113.0/pstamp/work/2019/05/14/924082 ps_ud_WEB F 0 0 WEB 924082 /data/ipp113.0/pstamp/work/2019/05/14/924082/checkdep.10977349.log 320
     79410977350 new chip 1131318 XY65 gpc1 /data/ipp113.0/pstamp/work/2019/05/14/924082 ps_ud_WEB F 0 0 WEB 924082 /data/ipp113.0/pstamp/work/2019/05/14/924082/checkdep.10977350.log 320
     79510977351 new chip 1131321 XY65 gpc1 /data/ipp113.0/pstamp/work/2019/05/14/924082 ps_ud_WEB F 0 0 WEB 924082 /data/ipp113.0/pstamp/work/2019/05/14/924082/checkdep.10977351.log 320
     79610977364 new chip 1130587 XY33 gpc1 /data/ipp113.0/pstamp/work/2019/05/14/924082 ps_ud_WEB F 0 0 WEB 924082 /data/ipp113.0/pstamp/work/2019/05/14/924082/checkdep.10977364.log 320
     79710977365 new chip 1130590 XY46 gpc1 /data/ipp113.0/pstamp/work/2019/05/14/924082 ps_ud_WEB F 0 0 WEB 924082 /data/ipp113.0/pstamp/work/2019/05/14/924082/checkdep.10977365.log 320
     79810977373 new chip 1131340 XY33 gpc1 /data/ipp113.0/pstamp/work/2019/05/14/924082 ps_ud_WEB F 0 0 WEB 924082 /data/ipp113.0/pstamp/work/2019/05/14/924082/checkdep.10977373.log 320
     79910977384 new chip 1130596 XY34 gpc1 /data/ipp113.0/pstamp/work/2019/05/14/924082 ps_ud_WEB F 0 0 WEB 924082 /data/ipp113.0/pstamp/work/2019/05/14/924082/checkdep.10977384.log 320
     80010977385 new chip 1130598 XY34 gpc1 /data/ipp113.0/pstamp/work/2019/05/14/924082 ps_ud_WEB F 0 0 WEB 924082 /data/ipp113.0/pstamp/work/2019/05/14/924082/checkdep.10977385.log 320
     80110977386 new chip 1130600 XY34 gpc1 /data/ipp113.0/pstamp/work/2019/05/14/924082 ps_ud_WEB F 0 0 WEB 924082 /data/ipp113.0/pstamp/work/2019/05/14/924082/checkdep.10977386.log 320
     80210977447 new chip 1583717 XY34 gpc1 /data/ipp113.0/pstamp/work/2019/05/14/924082 ps_ud_WEB F 0 0 WEB 924082 /data/ipp113.0/pstamp/work/2019/05/14/924082/checkdep.10977447.log 320
     80310977448 new chip 1583733 XY34 gpc1 /data/ipp113.0/pstamp/work/2019/05/14/924082 ps_ud_WEB F 0 0 WEB 924082 /data/ipp113.0/pstamp/work/2019/05/14/924082/checkdep.10977448.log 320
     80410977481 new chip 1756192 XY01 gpc1 /data/ipp113.0/pstamp/work/2019/05/14/924082 ps_ud_WEB F 0 0 WEB 924082 /data/ipp113.0/pstamp/work/2019/05/14/924082/checkdep.10977481.log 320
     80510977483 new chip 1756719 XY01 gpc1 /data/ipp113.0/pstamp/work/2019/05/14/924082 ps_ud_WEB F 0 0 WEB 924082 /data/ipp113.0/pstamp/work/2019/05/14/924082/checkdep.10977483.log 320
     80610977509 new chip 1795556 XY04 gpc1 /data/ipp113.0/pstamp/work/2019/05/14/924082 ps_ud_WEB F 0 0 WEB 924082 /data/ipp113.0/pstamp/work/2019/05/14/924082/checkdep.10977509.log 320
     80710977510 new chip 1795572 XY04 gpc1 /data/ipp113.0/pstamp/work/2019/05/14/924082 ps_ud_WEB F 0 0 WEB 924082 /data/ipp113.0/pstamp/work/2019/05/14/924082/checkdep.10977510.log 320
     80810977511 new chip 1795588 XY04 gpc1 /data/ipp113.0/pstamp/work/2019/05/14/924082 ps_ud_WEB F 0 0 WEB 924082 /data/ipp113.0/pstamp/work/2019/05/14/924082/checkdep.10977511.log 320
     80910977512 new chip 1795603 XY04 gpc1 /data/ipp113.0/pstamp/work/2019/05/14/924082 ps_ud_WEB F 0 0 WEB 924082 /data/ipp113.0/pstamp/work/2019/05/14/924082/checkdep.10977512.log 320
     81010977517 new chip 1813787 XY40 gpc1 /data/ipp113.0/pstamp/work/2019/05/14/924082 ps_ud_WEB F 0 0 WEB 924082 /data/ipp113.0/pstamp/work/2019/05/14/924082/checkdep.10977517.log 320
     81110977518 new chip 1813804 XY40 gpc1 /data/ipp113.0/pstamp/work/2019/05/14/924082 ps_ud_WEB F 0 0 WEB 924082 /data/ipp113.0/pstamp/work/2019/05/14/924082/checkdep.10977518.log 320
     81210977519 new chip 1813821 XY40 gpc1 /data/ipp113.0/pstamp/work/2019/05/14/924082 ps_ud_WEB F 0 0 WEB 924082 /data/ipp113.0/pstamp/work/2019/05/14/924082/checkdep.10977519.log 320
     81310977520 new chip 1813838 XY40 gpc1 /data/ipp113.0/pstamp/work/2019/05/14/924082 ps_ud_WEB F 0 0 WEB 924082 /data/ipp113.0/pstamp/work/2019/05/14/924082/checkdep.10977520.log 320
     81410977541 new chip 1936305 XY75 gpc1 /data/ipp113.0/pstamp/work/2019/05/14/924082 ps_ud_WEB F 0 0 WEB 924082 /data/ipp113.0/pstamp/work/2019/05/14/924082/checkdep.10977541.log 320
     81510977542 new chip 1936310 XY01 gpc1 /data/ipp113.0/pstamp/work/2019/05/14/924082 ps_ud_WEB F 0 0 WEB 924082 /data/ipp113.0/pstamp/work/2019/05/14/924082/checkdep.10977542.log 320
     81610977543 new chip 1936321 XY75 gpc1 /data/ipp113.0/pstamp/work/2019/05/14/924082 ps_ud_WEB F 0 0 WEB 924082 /data/ipp113.0/pstamp/work/2019/05/14/924082/checkdep.10977543.log 320
     81710977544 new chip 1936326 XY01 gpc1 /data/ipp113.0/pstamp/work/2019/05/14/924082 ps_ud_WEB F 0 0 WEB 924082 /data/ipp113.0/pstamp/work/2019/05/14/924082/checkdep.10977544.log 320
     81810977545 new chip 1936337 XY75 gpc1 /data/ipp113.0/pstamp/work/2019/05/14/924082 ps_ud_WEB F 0 0 WEB 924082 /data/ipp113.0/pstamp/work/2019/05/14/924082/checkdep.10977545.log 320
     81910977546 new chip 1936342 XY01 gpc1 /data/ipp113.0/pstamp/work/2019/05/14/924082 ps_ud_WEB F 0 0 WEB 924082 /data/ipp113.0/pstamp/work/2019/05/14/924082/checkdep.10977546.log 320
     82010977547 new chip 1936353 XY75 gpc1 /data/ipp113.0/pstamp/work/2019/05/14/924082 ps_ud_WEB F 0 0 WEB 924082 /data/ipp113.0/pstamp/work/2019/05/14/924082/checkdep.10977547.log 320
     82110977548 new chip 1936358 XY01 gpc1 /data/ipp113.0/pstamp/work/2019/05/14/924082 ps_ud_WEB F 0 0 WEB 924082 /data/ipp113.0/pstamp/work/2019/05/14/924082/checkdep.10977548.log 320
     82210977565 new chip 2055192 XY45 gpc1 /data/ipp113.0/pstamp/work/2019/05/14/924082 ps_ud_WEB F 0 0 WEB 924082 /data/ipp113.0/pstamp/work/2019/05/14/924082/checkdep.10977565.log 320
     82310977566 new chip 2055202 XY45 gpc1 /data/ipp113.0/pstamp/work/2019/05/14/924082 ps_ud_WEB F 0 0 WEB 924082 /data/ipp113.0/pstamp/work/2019/05/14/924082/checkdep.10977566.log 320
     82410977567 new chip 2055212 XY45 gpc1 /data/ipp113.0/pstamp/work/2019/05/14/924082 ps_ud_WEB F 0 0 WEB 924082 /data/ipp113.0/pstamp/work/2019/05/14/924082/checkdep.10977567.log 320
     82510977568 new chip 2055222 XY45 gpc1 /data/ipp113.0/pstamp/work/2019/05/14/924082 ps_ud_WEB F 0 0 WEB 924082 /data/ipp113.0/pstamp/work/2019/05/14/924082/checkdep.10977568.log 320
     82610977585 new chip 2088153 XY34 gpc1 /data/ipp113.0/pstamp/work/2019/05/14/924082 ps_ud_WEB F 0 0 WEB 924082 /data/ipp113.0/pstamp/work/2019/05/14/924082/checkdep.10977585.log 320
     82710977586 new chip 2088169 XY34 gpc1 /data/ipp113.0/pstamp/work/2019/05/14/924082 ps_ud_WEB F 0 0 WEB 924082 /data/ipp113.0/pstamp/work/2019/05/14/924082/checkdep.10977586.log 320
     82810977587 new chip 2088185 XY34 gpc1 /data/ipp113.0/pstamp/work/2019/05/14/924082 ps_ud_WEB F 0 0 WEB 924082 /data/ipp113.0/pstamp/work/2019/05/14/924082/checkdep.10977587.log 320
     82910977588 new chip 2088201 XY34 gpc1 /data/ipp113.0/pstamp/work/2019/05/14/924082 ps_ud_WEB F 0 0 WEB 924082 /data/ipp113.0/pstamp/work/2019/05/14/924082/checkdep.10977588.log 320
     830}}}
     831
     832We can find their chip_id and warp_id, using:
     833{{{
     834mysql -hscidbm -uippuser -pippuser gpc1 -B -e ' SELECT rawExp.exp_name,rawExp.exp_id,chipRun.chip_id,chipRun.state,chipRun.label,warpRun.warp_id,warpRun.state,warpRun.label,warpRun.data_group FROM  warpSkyfile, warpRun, fakeRun, camRun, chipRun, rawExp WHERE  warpSkyfile.warp_id = warpRun.warp_id AND warpRun.fake_id = fakeRun.fake_id AND fakeRun.cam_id = camRun.cam_id AND camRun.chip_id = chipRun.chip_id AND chipRun.exp_id = rawExp.exp_id AND  chipRun.chip_id in (1131144,1131146,1131208,1131210,1130544,1130549,1130551,1130546,1130556,1130560,1131230,1200205,1131246,1131251,1130562,1130564,1130568,1130570,1131257,1131260,1200216,1130573,1131276,1131280,1130575,1130578,1131297,1131303,1131318,1131321,1130587,1130590,1131340,1130596,1130598,1130600,1583717,1583733,1756192,1756719,1795556,1795572,1795588,1795603,1813787,1813804,1813821,1813838,1936305,1936310,1936321,1936326,1936337,1936342,1936353,1936358,2055192,2055202,2055212,2055222,2088153,2088169,2088185,2088201) group by chip_id;'
     835}}}
     836 And then send the chips to cleanup and then back to update.
     837 ->In fact, as soon as I sent them to cleanup, the pstamp server re-set their states to update and proceeded with making the stamps (since the warprun.state was full). Therefore, they are likely not properly cleaned at all....
     838
    765839=== Wednesday : 2019.05.15 ===
    766840