The purpose of psched is to manage the automatic construction and execution of inter-related (often repetative) operations. psched uses a set of rules to define UNIX commands, and their corresponding command-line arguments, to be performed on some regular, repeated basis. The utility of psched is that it can easily define an analysis system which is completely state-based, as opposed to an event-driven system.
Consider, for example, a telescope which obtains a collection of images over the course of a night. Every minute or two, it takes an image and writes the image to some disk. An event-driven analysis system would involve having the telescope initiate a process at the end of the exposure. This process would perform an analysis, write some output, then send trigger another process. This type of operation works very well for a simple set up with reliable hardware. Such a system becomes more difficult to maintain when hardware failures occur or when multiple systems need to interact with each other. When failures occur, the triggering information (the events) is easily lost, thus some mechanisms are needed to detect these failures and either re-send the trigger or send an alternative failure-mode trigger. Or, if two systems need to interact, one or the other system must block for results from the first. Stopping and restarting such an analysis system is very delicate since the appropriate triggers must be set up some how, eg by noticing which images have not succeeded and restarting them at the appropriate stage. All of these types of methods of handling complexity and failures are essentially state-based rules. psched allows the easy definition of a totally state-based analysis system.
In a state-based system, some mechanism examines the state of the system and decides which actions to perform based on the current state. In the illustration above, the mechanism could examine the images available (either by examining the disk or by examining the state of a data table) and decide to perform an operation based on what images are available. This makes it very easy to handle complexity and errors. If an analysis fails, the state either is not successfully updated or the error state is recorded, both situations being easy to detect and easy to handle. Restarting the system simply involves starting the state-monitoring mechanism. Combining results from multiple input sources simply involves watching for the multiple inputs to be available. psched provides a mechanism to define state monitors, and to define the actions which are performed when those states occur. psched action consist of initiating UNIX commands, where the arguments of those commands may depend on the results of the state tests.
The primary function of psched is to repeatedly perform tasks, and execute jobs on the basis of those tasks. A task consists of a set of rules which describe system state tests to perform on a regular time scale. Based on the results of those state tests, the task will then choose whether or not to construct a job. The task also defines actions to perform upon the completion of a job, based upon the output and exit status of the job. A task thus defines the repeat period. It may optionally define valid or invalid time ranges (eg, Mon-Fri or 10:00-17:00, etc). The task may also specify that the job be run locally (ie, in the background on the same computer as psched) or remotely by the parallel process controller (pcontrol). A job may even be restricted to a specific computer managed by pcontrol. An example of a simple tasks is given below.
task datalist
command ls /data/foo
periods -exec 5.0
periods -timeout 50.0
periods -poll 1.0
task.exit 0
queueprint stdout
queuedelete stdout
end
task.exit 1
queuepush failure "task failed"
end
end
This task does not perform any system state tests; it is simply constructs a new job every 5.0 seconds. The job in this case is always the same: ls /data/foo . When the job finished, if the job exit status is 0 (normal UNIX success status), the resulting output is printed to the screen. If the job returns an exit status of 1 (a failure), the failure queue receives a single entry. Although they are not defined in this case, it is also possible to specify the action to be taken if the job crashes (does not exit normally) or if it times out (runs beyond the specified timeout period). A slightly more complex task which performs a state test and constructs a command based on that test is shown below
task datalist
periods -exec 5.0
periods -timeout 50.0
periods -poll 1.0
task.exec
$file = `next.file`
if ($file == "none")
break
end
command cp /data/foo/$file /data/bar
end
task.exit 0
queueprint stdout
queuedelete stdout
queuepush copied $file
end
task.exit 1
queuepush failure $file
end
end
The task.exec macro is executed by psched every 5.0
seconds. This macro executes a (hypothetical user-defined) UNIX
command (next.file) which examines the system state, return
either a filename or the word "none". If the result of this test is
"none", the task does nothing: no job is constructed. Otherwise, a
job is constructed using the name of the file returned by the state
test. Successful jobs have the filename added to the 'copied'
queue, while failed jobs add the filename to the 'failure' queue.