Just in case there is any problem and I can't do things myself in case of a moped accident. Moreover searching for "moped" should be a good way to find this wiki page in a couple of years...

 * '''DONE''' change apache configuration files on ippc01-ippc10

== At 8pm ==
 * '''DONE''' (scripts are finished): check if jt scripts are still running on ippc19: make them asleep
 * '''DONE''' make sure ippdb02 is synchronized with ippdb00, stop slave on ippb02
 * '''DONE''' ''apache is connected to ippdb02 (check process table)'' restart apache servers on ippc01-ippc10
 * '''N/A''' wake up jt scripts: if crashed, tell jt
 * '''DONE''' kill nebdiskd on ippdb00
 * '''DONE''' start nebulous mysqldump on ippdb00 (use pbzip2)

   2012-05-29 19:00 The dump is running in screen session named nebulous_dump (i.e. reconnect on ippdb00 with {{{screen -r nebulous_dump}}}). The dump file is /export/ippdb00.0/nebulous_dumps/nebulous-20120529_fulldump.sql.bz2 and should have a size of 26G (according to the last successful dump of ippdb02 started this morning)

   2012-05-29 19:50 The dump should be finished around 0:30 (6 GB in 54 minutes / 27 GB for the complete dump).

   2012-05-29 21:50 The dumping rate has dropped down to about 10 GB in 3 hours... Should be finished around 3am?

   2012-05-30 06:20 Dump finished for info:
{{{
# stat nebulous-20120529_fulldump.sql.bz2
  File: `nebulous-20120529_fulldump.sql.bz2'
  Size: 27041967453     Blocks: 52816600   IO Block: 4096   regular file
Device: 804h/2052d      Inode: 1257        Links: 1
Access: (0644/-rw-r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2012-05-29 18:54:50.019218105 -1000
Modify: 2012-05-30 04:27:42.479312043 -1000
Change: 2012-05-30 04:27:42.479312043 -1000
}}}
== Once neb dump is finished ==
 * '''RUNNING''' copy reference dump to ippdb02 (and from ippdb02 to ipp001)

{{{
   /export/ippdb02.0/mysql-dumps/backups/20120529/nebulous-20120529_fulldump.sql.bz2
   /export/ipp001.0/ipp/mysql-dumps/Archives/2012/nebulous-20120529_fulldump.sql.bz2
}}}

 * delete nebulous database on ippdb00
 * stop the mysql server
 * clean mysql on ippdb00 (remove old binlogs, move and compress mysql logs)
 * start the mysql server
 * ingest neb dump on ippdb00
 * write down new master coordinates
 * move and compress nebdiskd log
 * restart nebdiskd

== Once ingestion is finished ==
 * restore apache configuration files on ippc01-ippc10 (backup files are called /etc/apache2/modules.d/apache2-mod_perl-startup.pl.20120529_ippdb00)
 * restart apache servers (jt scripts should be finished?!)
 * restart ipp
 * delete neb dump on ippdb00

== Once everything seems fine, *** on ippb02 *** ==
 * comment neb dump in root crontab
 * delete nebulous database
 * stop the mysql server
 * clean mysql (remove old binlogs, move and compress mysql logs)
 * start the mysql server
 * ingest neb dump
 * set master coordinates
 * start slave
 * activate neb dump in root crontab
 * delete neb dump