Setting up and Running the IPP Data Distribution Client

(back to IPP for PS1; back to GPC1 Data Distribution)

This page shows how to set up and run the IPP Data Distribution Client. In this example, a working simtest distribution set may be downloaded from the IfA IPP installation.

• Build a current ipp and insure that it works by running simtest. See IPP_Installation and SimtestNew.

• Create an empty database in mysql with some name. This example uses rc_test:

  dbadmin project somehost someaccount rc_test 
  

• Populate the tables: pxadmin -dbname rc_test -create-mirror

• Add the source (a reference to our data store) to the database

  receivetool -dbname rc_test -definesource    \
     -source  http://datastore.ipp.ifa.hawaii.edu/ \
     -product ps1-md \
     -workdir $WORKDIR \
     -comment 'simtest data source'
  

This works for ifaps1:

receivetool -dbname ifaps1_0 -definesource    \
   -source  http://datastore.ipp.ifa.hawaii.edu/  \
   -product ps1-md \
   -workdir /data/ifaps1.0/ps1data/MD/ \
   -comment 'ifa md repository'

• workdir is the local directory to download the files to
• dbname is the name of the database you chose (ifaps1 uses ifaps1_0)

Where $WORKDIR is the top level directory for the data to be received

• Start up pantasks and issue the following comands

  module pantasks.pro
  module receive.pro
  add.database rc_test
  controller host add somehost
  run

replacing somehost with the name of the hosts to run the pantasks jobs.

Note: These hosts must have internet access to make http requests to the data store and ssh must be set up to allow ssh login without supplying a password (see simtest notes).

Multiple hosts may be specified as usual.

Summary information such as faults and copy/extract times can be found in the receiveResult table of the mySQL database, file names and byte size (and other information) in receiveFile:

  select * from receiveResult where fault>0;
  select * from receiveResult;

Example list of files successfully downloaded (fault=0), size, download time, and extraction time:

SELECT receiveResult.file_id,receiveFile.file,receiveResult.dtime_copy,receiveResult.dtime_extract,receiveFile.bytes 
FROM receiveResult,receiveFile 
WHERE receiveResult.fault=0 and receiveResult.file_id=receiveFile.file_id;

Is/will there be functionality in ippMonitor to access/monitor this information?

When faults happen, how can the download be redone? This seems to work (is it the proper method?)

receivetool -dbname rc_test -revert -fault 1

What if files were added to a -product (is this possible?), how would an update be made? -revert -product dist0 ?