Changes between Version 65 and Version 66 of ippToPsps

Timestamp:

May 18, 2010, 1:09:51 PM (16 years ago)

Author:

rhenders

Comment:

--

ippToPsps

@@ -124 +124 @@
 Currently, the IPP to PSPS interface is a 'one-way' system. Batches are created by {{{ippToPsps}}} and posted on an IPP instance of the datastore. These batches are collected by the {{{DXLayer}}} on the PSPS side. As a basis for a future recovery system, the IPP urgently requires some feedback from PSPS as to which batches have succeeded and which have failed, and the reason why. With this information data can be either deleted, or regenerated accordingly. This is important simply because, with such large data volumes, we cannot afford the high levels of redundancy currently in place. At present, for a given batch, the following copies exist within the pipeline:
 
-- a copy exists on the IPP cluster after generation by ippToPsps program
-- a copy exists on the IPP datastore after publication by ippToPsps
-- the DXLayer retains a copy after it has sent the csv version to the ODM
-- the DXLayer also keeps a copy of these (larger) csv files
+ - a copy exists on the IPP cluster after generation by ippToPsps program
+ - a copy exists on the IPP datastore after publication by ippToPsps
+ - the DXLayer retains a copy after it has sent the csv version to the ODM
+ - the DXLayer also keeps a copy of these (larger) csv files
 
 We therefore need to quickly implement the basic framework of a feedback loop such that the IPP can quickly learn if a given batch has been successfully merged into the PSPS database or not. This will enable it to safely delete the data files and remove the copy from the datastore.