Changes between Version 8 and Version 9 of LAPintoPSPS

Timestamp:

Dec 12, 2012, 10:21:02 AM (14 years ago)

Author:

heather

Comment:

--

LAPintoPSPS

@@ -116 +116 @@
 }}}
 
+== Statistics on slice18 ==
+
+Here are the statistics on slice18. I wanted to see how ingesting slice18 compares to ingesting the same area of sky with various older surveys (same area of sky)
+
+The sql query : 
+
+{{{
+mysql> select minidvodb_group, stage,count(*), sum(dtime_addstar)/60/60/24., avg(dtime_addstar) from addProcessedExp join addRun using (add_id) join camRun on cam_id = stage_id join chipRun using (chip_id) join rawExp using (exp_id) where stage = 'cam' and addRun.state = 'full' and ra > 270*3.14/180 and ra < 300*3.14/180 and decl > 20*3.14/180 and decl < 90*3.14/180 group by minidvodb_group;
+}}}
+
+The result
+{{{
++-----------------------------+-------+----------+------------------------------+--------------------+
+| minidvodb_group             | stage | count(*) | sum(dtime_addstar)/60/60/24. | avg(dtime_addstar) |
++-----------------------------+-------+----------+------------------------------+--------------------+
+| (null)                      | cam   |      100 |            0.066631944444444 |              57.57 | 
+| CNP.V2                      | cam   |      154 |            0.026168981376621 |    14.681818122988 | 
+| LAP.slice18.cam             | cam   |     8951 |              7.0359143898167 |    67.914535055319 | 
+| LAP.ThreePi.20110809        | cam   |     5288 |              4.6303240664607 |    75.654311524622 | 
+| LAP.ThreePi.20110809.V2.Cam | cam   |     6532 |              4.4052546205013 |    58.269136437739 | 
+| ThreePi                     | cam   |     2547 |              1.3785300813919 |    46.762857884672 | 
+| ThreePi.V2                  | cam   |      639 |             0.27396990687759 |    37.043818394716 | 
+| ThreePi.V3                  | cam   |     6095 |              3.0291319420299 |    42.939622607282 | 
++-----------------------------+-------+----------+------------------------------+--------------------+
+}}}
+
+So what can we take from this? First of all it thinks it took ~7 days to ingest almost all of LAP.slice18 (there are about 40 pending).  It feels longer than that, more like 1.5 weeks. Heather suspects that this is due to only having dtime_addstar, and not dtime_total in the addProcessedExp table - so I think it should be longer,but not sure by how much. However, this affect will be throughout all processing (they are all processed the same way). What we can see is that slice18 so far has taking the longest time, but the average time is similar to LAP (V2 and 20110809). So it's a bit slower than ThreePi.V3 (I'm not sure why), but not that much slower... One thing that might cause problems is that ipp005 was both creating the dvo db and serving out files for ipptopsps - does this slow it down?