IPP Software Navigation Tools IPP Links Communication Pan-STARRS Links
wiki:Pantasks_server_mode

Version 1 (modified by trac, 17 years ago) ( diff )

--

The pantasks "server" mode allows to pantasks to run using a server/client interface. This is especially handy if you want to be able to control pantasks from multiple locations (e.g., check processing from home) or accounts (e.g., multiple people controlling or eavesdropping on the processing).

There is a small convenience issue when running pantasks in the "server" mode: some commands which you would usually run in a standalone pantasks are only available when wrapped inside a macro; see below.

Configuration

Ensure the following is in your <code>~/.ptolemyrc</code> file: <pre> PANTASKS_SERVER XXXXX.ifa.hawaii.edu PASSWORD XXXXX #PANTASKS_SERVER_STDOUT pantasks.stdout.log #PANTASKS_SERVER_STDERR pantasks.stderr.log </pre>

  • <code>PANTASKS_SERVER</code> defines the name of the server for the client to connect to. There's no real need to update this, since we'll use a script to set this on the fly, allowing multiple pantasks servers to be used simultaneously.
  • <code>PASSWORD</code> provides minimal access control for the pantasks server. Set this to a unique word.
  • <code>PANTASKS_SERVER_STDOUT</code> and <code>PANTASKS_SERVER_STDERR</code> define where the output goes that is usually displayed in the pantasks session. Since setting these also affects the standalone version of pantasks, I recommend leaving them commented out; we'll use a script to set them on the fly.

If you are using Nebulous, you will also need to set up an ipphosts.config file. This config file has METADATA cards that assign a particular host to each chip on the camera, so that nebulous can distribute every exposure consistently, without piling too many chips onto the same machine. There is also a metadata card for indicating which hosts are to be used for skycell processing. Here is an example of how this config file should look:

<pre> ipphosts MULTI

ipphosts METADATA

camera STR skycell count S32 8 sky00 STR ipp006 sky01 STR ipp007 sky02 STR ipp008 sky03 STR ipp009 sky04 STR ipp010 sky05 STR ipp011 sky06 STR ipp020 sky07 STR ipp021

END

ipphosts METADATA

camera STR GPC1

XY01 STR ipp014 XY02 STR ipp014 XY03 STR ipp014 XY04 STR ipp014

XY05 STR ipp023 XY06 STR ipp023 XY10 STR ipp023 XY11 STR ipp023

XY12 STR ipp024 XY13 STR ipp024

< etc. ... >

XY75 STR ipp015 XY76 STR ipp015

END </pre>

Helpful scripts

Here are some convenient scripts for starting the pantasks server and client.

  • Save the following script as start_pantasks_server in some directory on your path

<pre> #!/bin/bash # Start up a pantasks server # Output is redirected to files server_redirect="-D PANTASKS_SERVER_STDOUT pantasks.stdout.log -D PANTASKS_SERVER_STDERR pantasks.stderr.log" echo pantasks_server $server_redirect pantasks_server $server_redirect </pre>

  • Save this one as start_pantasks_client on your path

<pre> #!/bin/bash # start up pantasks_client # 1st argument is the name of the host to connect to server_arg="" server_host=$1 if [ $server_host ]; then

server_arg="-D PANTASKS_SERVER $server_host" echo pantasks_client $server_arg pantasks_client $server_arg

else

echo "Please provide the hostname of the pantasks server."

fi </pre>

Starting pantasks

Starting the server is as easy as: <pre> % start_pantasks_server bound to port: 2000 </pre> You can start it in the background if you want: it's not going to print anything more to the terminal.

In a different window (or machine or user), start the client: <pre> % start_pantasks_client mymachine </pre> where <code>mymachine</code> is the name of the machine on which the server is running.

You should be presented with the usual pantasks environment. However, some things are subtly different: the environment you see before you is particular to the client, not the server, and it's the server that runs everything. This is important to keep in mind, and it means that you should bury within macros some commands that you would ordinarily just type on the pantasks command-line.

Setting up

Here are some useful definitions for controlling pantasks using the server mode:

  • Save this as server.pro in your pantasks modules directory

<pre> macro setup

$LABEL = "my_label" $DBNAME = "mydatabase" module pantasks.pro module.tasks all.off

# NEXT LINES ARE ONLY NEEDED IF USING NEBULOUS

$default_host = machine1 queueload tmp -x "cat ipphosts.config" ipptool2book tmp ipphosts -key camera

# END NEBULOUS ONLY

add.database $DBNAME echo Using database $DBNAME echo Current active label is $LABEL

end macro change_label

$LABEL = $1 echo Current active label is $LABEL

end macro print_label

echo Current active label is $LABEL

end

macro machines

controller host add machine1 -threads $1 controller host add machine2 -threads $1 controller host add machine3 -threads $1 controller host add machine4 -threads $1

end </pre>

  • If you're not using Nebulous, you can remove the lines that are specific to Nebulous use (these send jobs to particular machines).
  • Ensure the <code>my_label</code> is changed to the label of the data you're processing (or <code>"NONE"</code> including quotes if none).
  • Ensure the <code>my_database</code> is changed to the database you're using.
  • Ensure the <code>machineN</code> are changed according to your system.

So, now in the pantasks client, you can say: <pre> pantasks: server input server.pro </pre> to load the macro file. Note that the server has to do this command, hence the prefix <code>server</code>.

Here is how you would set up for processing, and add hosts with a maximum of 4 threads each: <pre> pantasks: setup pantasks: machines 4 </pre>

Now you should be able to simply say: <pre> pantasks: run </pre> and pantasks will run as usual. The <code>status</code>, <code>stop</code> and <code>halt</code> commands are also available as usual (no need to prepend <code>server</code> or anything like that). But if you want to change the label, then you should do: <pre> pantasks: change_label "new_label" </pre> whereas in the standalone pantasks, you could have done: <pre> pantasks: $LABEL = "new_label" </pre> Such commands now have to be buried within macros.

Shutdown

You can quit out of the client at any time by saying: <pre> pantasks: quit </pre>

This does not stop the server from continuing to process. You can reconnect to the server as you please.

When you're all done, you can force the server to quit by issuing the following command from the client: <pre> pantasks: shutdown now </pre>

Note: See TracWiki for help on using the wiki.