Objective
=========

The purpose of this software is to give details about how the IPP
cluster is being used. 

Dependencies
============
psutil (0.6.0): http://code.google.com/p/psutil/

Installation
============
1) Have a look at the installation script, read the associated
comment, and change the value of:
BIN_TARGET_DIRECTORY
HTML_TARGET_DIRECTORY
TARGET_NODENAMES_FILENAME

2) Synchronize the contents of TARGET_NODENAMES_FILENAME and the nodes
array in the update_all_nodes.py script (I know it's awful).

3) Run the deploy.sh script

4) Add an entry to your crontab

Constraints and Insights
========================

The update_all_nodes.py script is supposed to be run through a crontab
(or any scheduling mechanism resembling crontab) and produces a set of
HTML pages which are then rendered through a php script.

The user running the script must then have the right to execute cron
jobs. This user must also be able to ssh to any node that needs to be
supervised.

The products (that is html pages) are written to a location that has
to be writable to that user from every node where the (typically a
shared partition of some sort like a NFS one). If the pages are to be
seen thanks to a web browser, they also need to be accessible from
apache (see the "In the IPP context" paragraph for a practical case).

Each instance of the get_processes_information.py generates its own
html page. It scans 5 times every .5s the process table on the host
and filters out some information according to the rules defined in the 
Users class of this script, namely:
* ignore: the users who should always be ignored
* expected: the users who are expected to run something on the
  cluster. Any other user that are not in 'ignore' and are not
  'expected' will be displayed as a potential problem (i.e. in red).
* always: the users for whom all processes must be displayed (even if
  they don't use any resource)
* cmdline_ignore_re_progs_from_pattern: list of regular expressions of
  some programs we don't want to be displayed (e.g. condor daemons)
* max_age_in_seconds: set to 6 hours. Any process older than that will
  be displayed as a potential problem, i.e. in red.
* min_cpu (percentage): any process using more than this quantity will
  be displayed as a potential problem, i.e. in red.
* min_memory: any process using more than this quantity will
  be displayed as a potential problem, i.e. in red.

In the IPP context
==================

The update_all_nodes.py script is run from ippc18 by the ippm user
every 5 minutes. An independent thread is run for each node,
connecting through ssh to that node where the process analysis has to
be performed using the get_processes_information.py script.

The nodes list is in update_all_nodes.py (but must be the same as the
list found in the file named TARGET_NODENAMES_FILENAME in the
deploy.sh script).

TODO / Known bugs
=================
* Make the nodes names list unique
* There are likely hard-coded paths, values and that kind of stuff in
  the scripts
