| | 124 | |
| | 125 | * C-level. This would require script options that instead of executing the final command (ppImage/psastro/pswarp/etc), those commands would be written to an output file and the script would terminate. As we do database updates after that command, we would need to also write out a set of completion commands to update the database with the result. |
| | 126 | * This requires significant changes in the current scripts. |
| | 127 | * It may be difficult to keep the database synchronized at all. Once a task has been sent to the supercomputing cluster, we need a way to flag the job as "executing" so we don't launch a second run. The alternative is that the scripts will simply wait until the job is done. |
| | 128 | * Keeping all the database interactions in Hawaii, and just letting the computational part execute in the supercomputer gives a solution to the detrend issue. We can construct the command to manually specify which detrend to use. Gene had the idea to fold this determination into chiptool, such that the returned metadata includes the detrends that should be used. |
| | 129 | * Script level. This would send entire perl script level commands to the computing cluster (chip_imfile.pl/etc), with the database interaction disabled (--no-update). |
| | 130 | * Although this shouldn't require major changes to the scripts, we still need something to check the results and issue the proper database commands in Hawaii. |
| | 131 | * This also prevents the detrend fix from above working. In fact, it would require some sort of script change to get the construction of the final command done correctly. Currently that's handled by passing only the stage_id to the script, which then asks the database for information about the object described by the stage_id. |
| | 132 | * Stage level. An entire stage worth of final commands is constructed for a given label. |
| | 133 | * This solves the synchronization issue of the C-level by knowing that we'll only queue the commands once. |
| | 134 | * Allows for a scan of the results before generating the commands for the next stage. |
| | 135 | * Requires a large amount of data to be synchronized at each step, although this could be rolled over once we understand the data transfer vs. computation times. |
| | 136 | * Project level. All stages are queued at the same time, with some logic used in moab to determine which jobs are dependent on others. |
| | 137 | * Even larger synchronization issues. |
| | 138 | * Requires some |
| | 139 | * Many stages take metadata objects as inputs. If some components fail permanently (i.e., single bad chip failures, poorly populated skycells), we need to not include them in the input metadata. This prevents all metadata from being pre-calculated. |
| | 140 | |
