Changes between Version 1 and Version 2 of ippToPspsIngestof3piPV2

Timestamp:

Jan 17, 2015, 8:11:21 PM (12 years ago)

Author:

heather

Comment:

--

ippToPspsIngestof3piPV2

@@ -22 +22 @@
 
 
-== step 2 queue ==
+=== step 2 queue ===
 
-basically, loader name is 
+there is a script in queueLoaders in ~heather - I need to add to tools.
+
+basically, loader name is PSPS_PV2_OB_SLICE00
+
+=== step 3 set up stare nodes because they are cranky ? ===
+
+* various iterations of start/restart of mysql , manually deleting locked tables and reconfiguring the scratch databases
+
+=== step 4 add stsci loaders ===
+
+magic incantations to configure mysql
+
+{{{
+    mysql -u root
+    CREATE USER 'dvo'@'%' IDENTIFIED BY 'dvo';
+    GRANT ALL on *.* to 'dvo'@'%';    
+grant all privileges on *.* to dvo@localhost identified by 'dvo';
+flush privileges;
+    show grants; 
+exit
+mysql -h localhost -u dvo -pdvo -e "create database ipptopsps2_scratch"
+ippjython setupScratchDb.py
+}}}
 
 
-== step 3 set up stare nodes because they are cranky ? ==
+== Current configuration of loaders ==
 
+The current configuration of loading— ippdvo@ippc18, several screens (screen -r stare00 etc)
+
+This is the configuration of loaders
+
+stare00 — slice 00
+stare01 — slice 06
+stare03 — slice 02
+stare04 — slice 04
+ipp006  —slice 08
+stsci00 — slice 10
+stsci01 — slice 12
+stsci02 — slice 14
+stsci04 — slice 16
+stsci05 —slice 18
+stsci06 —slice 20
+stsci07 —slice 22
+
+== Checking/Restarting logs ==
+
+===Saturday, 7:30am===  
+ all loaders are dead. restarted them
+
+===Saturday, 12:00pm=== 
+ 3 loaders needed to be restarted (stsci/ipp006)
+
+===Saturday, 8:00pm=== 
+{{{
+ select count(*), datastore_product from pending join box on box_id = id join skychunk on name = skychunk where name like 'PSPS%' group by datastore_product ;select count(*) from batch where dvo_db like 'LAP.PV2%' and loaded_to_datastore =1; select count(*), datastore_product from pending join box on box_id = id join skychunk on name = skychunk where name like 'PSPS%';
++----------+---------------------+
+| count(*) | datastore_product   |
++----------+---------------------+
+|     6500 | PSPS_PV2_OB_SLICE00 | 
+|     7444 | PSPS_PV2_OB_SLICE02 | 
+|    10242 | PSPS_PV2_OB_SLICE04 | 
+|    10468 | PSPS_PV2_OB_SLICE06 | 
+|     8023 | PSPS_PV2_OB_SLICE08 | 
+|     6486 | PSPS_PV2_OB_SLICE10 | 
+|     7043 | PSPS_PV2_OB_SLICE12 | 
+|     9248 | PSPS_PV2_OB_SLICE14 | 
+|    11355 | PSPS_PV2_OB_SLICE16 | 
+|    12498 | PSPS_PV2_OB_SLICE18 | 
+|    12606 | PSPS_PV2_OB_SLICE20 | 
+|     9308 | PSPS_PV2_OB_SLICE22 | 
++----------+---------------------+
+12 rows in set (0.21 sec)
+
++----------+
+| count(*) |
++----------+
+|    11662 | 
++----------+
+1 row in set (0.01 sec)
+
++----------+---------------------+
+| count(*) | datastore_product   |
++----------+---------------------+
+|   111221 | PSPS_PV2_OB_SLICE00 | 
++----------+---------------------+
+1 row in set (0.13 sec)
+}}}
+Loaders which needed restart:
+
+stare00 - restart
+stare03 -restart
+ipp006 -restart
+stsci00, 01, 03, 6, 7 -restart
+