Notes on the IPP Database Backup scheme

(Up to IPP for PS1)

Updated: We have recently (2009.09.16) changed the dump scheme to only dump the specific databases of interest. The text below reflects this change.

Thanks to efforts by Cindy & Gavin, regular dumps of the mysql databases being used by the processing system on Maui are now being made available via the rsync server on ipp0002.ifa.hawaii.edu. The three databases distributed are the gpc1 processing database, the isp processing database, and the ippadmin database describing the database schema. These are dumped every 4 hours, and are available from the rsync location:

rsync://ipp0002.ifa.hawaii.edu/ippdb

if you want all three, give a command like:

rsync -auv rsync://ipp0002.ifa.hawaii.edu/ippdb/ ippdb/

The databases have names of the form ippdb01-DBNAME.dump.gz, eg, the GPC1 processing information is in the file called ippdb01-gpc1.dump.gz. The GPC1 gzip file is currently 0.8GB, so be careful. They are also dumped every 4 hours, and the new one replaces the old name. If people are unable to retrieve these in less than the 4 hour period (~0.4 Mbit/sec), then we can adjust the naming to keep more than one old version around.

If you want just one of the above databases, use a command like:

rsync rsync://ipp0002.ifa.hawaii.edu/ippdb/ippdb01-gpc1.dump.gz .

For reference, all of the various ipp mysql databases are now being dumped to /data/ipp000.0/ipp/mysql-dumps (had been /data/ipp000.0/jhoblitt/mysql-dumps), and these above two are linked into the 'distribution' subdirectory, which is made visible via the rsync server. We are moving to a system of keeping a sampling of the old databases with the following timescales:

all backups for the past 5 days (~20 copies) one backup per day for the previous ~10 days (~10 copies) one backup every 10 days for the previous 100 days (~10 copies) one backup every 100 days for the lifetime of the project (~15 copies)

(currently, we are keeping all of the old copies with a linear spacing...)