Changes between Initial Version and Version 1 of pstamp_web_interface_features

Timestamp:

Dec 23, 2009, 12:48:51 PM (17 years ago)

Author:

bills

Comment:

--

pstamp_web_interface_features

@@ +1 @@
+
+== Notes on a PS1 postage stamp server web interface ==
+
+
+The postage stamp server (PSS) is primarily designed to handle requests for many postage
+stamps by science clients.
+
+It is also desired to have a user friendly web interface that allows a user
+to input parameters for a request (or requests), submit them to the server for
+processing, and retrieve the results.
+
+This note discusses features that such a web interface might have. 
+
+First we provide some technical context by describing the operation of the postage stamp server and
+the prototype web interface. Those interested only in potential web interface
+features may skip ahead to the section "[Features for the IPP PSS Web Interface]"
+
+
+
+== Overview of the operation of the IPP Postage Stamp Server ==
+
+The postage stamp server consists of a set of IPP programs that run in under pantasks.
+The sequence of requests and jobs is managed with the use of a mysql database.
+
+The nominal mode of operation is to query a set of predefined web sites for new request files.
+These web sites follow the data store interface.
+
+When a new fileset on a datastore is discovered the postage stamp server creates a new
+request in the database for each file in the fileset. All files are assumed to be
+fits binary tables with EXTNAME=PS1_PS_REQUEST. The definition of this files may be
+found in the postage stamp server interface control document on the wiki.
+
+    http://svn.pan-starrs.ifa.hawaii.edu/trac/ipp/wiki/PostageStampServer
+
+
+Each row in a request file is a "request specification".
+The columns in the request specification server three purposes
+
+    * to select the images of interest
+    * to define the region of interest (the pixels in the input image that will be
+      included in the stamp)
+    * to select which data products that are included in the results (e.g. image stamp, mask stamp,
+      variance stamp, psf, background model)
+
+Sometime after the requests have been queued, they are processed. The request file is copied to
+the PSS network and parsed.  The parser reads each row in the request file and creates zero
+or more postage stamp jobs.
+Currently each job creates one stamp but there is an expectation that in the future jobs request
+specifications that reference the same images of interest will be processed by a single job.
+
+When all jobs have finished, the postage stamp server builds a fits binary table that describes
+the results of each job. Then that file and the results of the jobs are packaged into a fileset on
+a PSS data store product specific to the requestor.
+
+
+
+== Web Interface ==
+
+Now all of the above is very good for science clients but not at all convienent for random
+user requests.
+
+What we need is some sort of form based web pages that users may use to create requests, submit
+them for processing, and "get" the results.
+
+We have a "prototype" web interface which is very simple, feature incomplete, and still not particularly
+friendly. It's a set of php forms. I do not expect that any of it will be used in the "real web interface".
+
+The way the prototype interacts with the PSS is worth reviewing.
+
+The site is http://pstamp.ipp.ifa.hawaii.edu/request.php
+
+It is password protected with the "standard" PS1 science consortium username and password.
+
+The form allows users to input some of the supported parameters.
+At the top we have data for the images of interest
+For example:
+
+
+{{{
+    Select Images by:   Exposure Name
+    Image Type:         warp
+    ID:                 o5013g0086o
+    Sky     RA:         293.3888     DEC:                16.04312
+    Pixels  width       200          height: 200
+}}}
+
+
+If the user hits submit, the apache web server hosting the page executes a cgi script that
+runs a program that creates a request table file containing a single row. Then the cgi script runs a command that enters 
+a postage stamp request into the database.
+
+On the form the radio button "Get Status" gets selected. If the submit button is pushed again it
+shows the status of the request (and any previous requests entered in the given http session.
+
+Once all of the jobs from the request have finished processing 
+part of the the status line turns into a link to the data store fileset where the results
+have been posted.  For example: http://datastore.ipp.ifa.hawaii.edu/pstampresults/web_373/
+
+If we go to this location we find two files: results.fits and parse_error.txt
+Unfortunately this example generated postage stamp images.
+
+results.fits is the results table.  If we download it and examine it with fv we find that it
+contains 4 rows all of which have an error code set. 
+ *  The first two rows have ERROR_CODE=24 PSTAMP_NOT_DESTREAKED.
+ *  The other two rows have ERROR_CODE=25 PSTAMP_NOT_AVAILABLE.
+
+The first error is somewhat self explanatory. The images haven't been run through magic and thus can't
+be released. PSTAMP_NOT_AVAILABLE is the error that happens when the input images have been
+cleaned up (deleted to recover disk space).
+
+If these images hadn't been cleaned up postage stamp images would have been placed in the fileset as well.
+
+Soon the postage stamp server will (optionally by user request) queue update runs which will ask the IPP 
+to regenerate the images. This will of course increase the latency for processing requests significantly.
+    
+parse_error.txt gives any errors printed to stderr by the request file parser.
+In this example it contains the lines
+
+{{{
+    skipping non-magicked image neb://ipp035.0/gpc1/ThreePi_NorthernRegion.090729//o5013g0086o.83651/o5013g0086o.83651.wrp.9992.skycell.1662.134.fits
+    skipping non-magicked image neb://ipp023.0/gpc1/ThreePi_NorthernRegion.090729//o5013g0086o.836515013g0086o.83651.wrp.9992.skycell.1748.006.fits
+}}}
+
+
+These lines correspond to the jobs in the results file with ERROR_CODE=24.
+
+
+Desired Features for the IPP PSS Web Interface
+----------------------------------------------
+
+Form elements for all parameters. 
+---------------------------------
+
+The elements presented to the user should be context sensitive.  For example the form elements
+where the ID and COMPONENT are to be entered should be labeled depending on the REQ_TYPE:
+if the REQ_TYPE is "byexp" then the ID element should be labeled "Exposure ID".
+Similar considerations apply to the COMPONENT parameter.
+
+This suggests using some code that operates on the client side in the implementation
+(e.g. Javascript or Java).
+
+Form elements need not follow the exact names in the request specification.
+
+Parameter validation
+--------------------
+Where efficient, parameters should be validated as they are entered and the user should
+be notified of errors. (The prototype delays what checking it does until submission).
+Some parameters that are "invalid" on the server side but legal in the request interface
+should be caught and either corrected or raised as an error. 
+For example o5013g0086o is a valid GPC1 exposure name but o50130086o is not.
+
+That said perhaps the user should be allowed to enter values that map to the exp_name.
+
+{{{
+    MJD: 55013  SEQ: 86
+}}}
+
+(Note this would require either the ability to read the ipp database or some other table that
+maps from date and sequence number to exposure name).
+
+
+== User Accounts ==
+ 
+
+We want to define a set of authorized users each with it's own password. The PSS server will
+be enhanced to include the identity of the user in the submitted requests. The PSS server
+may use this to ration usage or prioritize the processing of jobs.
+
+Since requests can have significant latency it would be useful to allow users to "save" the
+state of a session so that it can be resumed later.
+
+
+== User experience for Request submission and status ==
+
+One of the biggest limitations of the prototype interface is the interaction with the server.
+The whole push a button to get status is rather primitive. 
+
+
+== Presentation of Results ==
+
+When a request completes, the results file should be parsed and importand data presented
+to the user in an easy to understand way.
+
+It might be useful to optionally display jpeg images of the stamps. (An option to create these should
+be added to the PSS request specification).
+
+A way to convenient way to download the fits files is desired.
+
+