MinidvodbCopy
This describes how to use minidvodbcopy.pro (with pantasks) to automatically transfer the minidvodbs to a machine.
Using the module in pantasks
General description
This module is intended to be loaded into the addstar pantasks running on ipp004. This is because this is where the minidvodbs live. Like addstar.pro/minidvodb.pro, this does not need controller host add (host).
server module minidvodbcopy.pro
To queue up and start the copy process:
add.minidvodb.copy (minidvodb_group) (host) (minidvodbcopy_path)
This will queue up all the minidvodbs in 'merged' state, will copy them to /data/ipp022.0/minidvodbs/ThreePi.V1.Mini on ipp022(2). Note that we are using ipp0222 in order for the transfer to work. Also note that ipp004 must be able to ssh/rsync to ipp022 without a password. There are checks on this for each minidvodb queued (there must be 97% or less used, and there must be enough space to transfer the minidvodb onto the host). This is similar to the queueing mechanism for creating dvodbs - it will only queue minidvodbs that have not been queued yet for ipp022(2), there should only be 1 copy of a minidvodb on a machine.
Then, you need to tell pantasks which hosts you want to turn on the transfer - this is a bit like adding labels to determine what is processed:
add.minidvodb.host (host)
Example
An example of how to queue up and copy ThreePi.V1 onto ipp022(2):
add.minidvodb.copy ThreePi ipp0222 /data/ipp022.0/gpc1/mini-catdirs/ThreePi add.minidvodb.host ipp0222
Other useful tasks
| del.minidvodb.host (host) | deletes the host from the list of hosts to transfer |
| show.minidvodb.host | shows the current hosts used for transfer |
| del.minidvodb.copy (host) (minidvodb_group) | removes (host) (minidvodb_group) from queuing/loading/copying |
| show.minidvodb.copy | shows which (host).(minidvodbs) are in pantasks |
| minidvodb.copy.on | turns on/off all the minidvodb.copy tasks |
| minidvodb.copy.off | |
| minidvodb.copy.load.on | turns off individual copy tasks |
| minidvodb.copy.load.off | |
| minidvodb.copy.queue.on | |
| minidvodb.copy.queue.off | |
| minidvodb.copy.run.on | |
| minidvodb.copy.run.off |
MinidvodbCopy table
Information about the state of the minidvodb transfers are available on the minidvodbCopy table:
| Field | Type | Null | Key | Default | Extra | Comment |
| minidvodbcopy_id | bigint(20) | NO | PRI | NULL | auto_increment | |
| minidvodb_id | bigint(20) | YES | MUL | NULL | which minidvodb_id to be copied | |
| minidvodb_rsync_path | varchar(255) | NO | NULL | where to copy it (path) | ||
| destination_host | varchar(255) | NO | NULL | where to copy it (host) | ||
| fault | smallint(6) | NO | NULL | were there faults copying it | ||
| state | varchar(64) | NO | NULL | should be either 'new' or 'full' | ||
| epoch | timestamp | NO | CURRENT_TIMESTAMP | Time of copy (or when the table was last updated | ||
| dtime | float | YES | NULL | Total time the transfer took |
minidvodbtool
You can use minidvodbtool to find out information about the minidvodbCopy table, to revert/queue/update items. minidvodbtool -h gives the available options. Currently there is no revert task - I need to think about how to do this, which ones can and should be reverted. For now, if there are faults, please investigate carefully why they failed, then revert by hand using -updateminidvodbcopy
minidvodbcopy logs
Please, if you run into problems, or do unusual/irregular things, please update the log.
http://svn.pan-starrs.ifa.hawaii.edu/trac/ipp/wiki/MinidvodbCopyLog
