IPP Software Navigation Tools IPP Links Communication Pan-STARRS Links

Changeset 38258


Ignore:
Timestamp:
May 13, 2015, 9:19:19 AM (11 years ago)
Author:
bills
Message:

Working version of czartool with plots.
raw/ipp.czar.dat

Remove 'czartool - ipptopsps link' from Czar left panel menu

raw/ipp.php

Get czardb name and host from site.php

raw/czartool_getplot.php

czartool plots are now in project specific directory
get base directory from

raw/czartool_labels.php

XXX: a couple of testing hacks to remove
Include proj in arguments to links to czartool_getplot.php

czartool/czarpoll.pl

Support for gathering data from multiple project databases.
czartool::Plotter now has an optional project database member. 3 Plotter
instances now.

plotter0 is for non project specific plots.
plotter1 is for gpc1
plotter1 is for gpc2

Keep track and print out how long each poll iteration takes. If it takes
less than the supplied period, sleep for the remainder. Currently takes
about 140 seconds which is larger than the default 60 second period.

Turn off the ippToPsps data gathering by nulling out the stage list.

czartool/czarplot.pl

Support for multiple projects

czartool/czartool/Metrics.pm

Metrics has a projectDb pass to Plotter constructor.
hardcoded member gpc1Db is deleted

czartool/czartool/DayMetrics.pm

projectDb isntead of hard coded gpc1Db
pass dbname to StageMetrics and CzarDb methods.

czartool/czartool/Config.pm

Added support for 2 project databases

czartool/czartool/CzarDb.pm

Operations on the stage tables now take "dbname" parameter.
This is passed to StageMetrics constructor.
XXX: should we split query for getTimeMinMaxDiff by project.

czartool/czartool/Plotter.pm

Add _projectDb member. Save dbname to _project for use in many method
invocations

czartool/czartool/StageMetrics.pm

Add dbname parameter.
XXX: Actually I'm not sure this is used any longer. This may have been
from a previous iteration. I've since moved the DB info to Plotter and Metrics.
Leaving in for now since this version is fully tested.

czartool/czartool/MySQLDb.pm

Add method to get _dbName

czartool/czartool/MetricsIndex.pm

Add database paramter to constructor for superclass Metrics. (It used to
have hard coded gpc1 db

Location:
trunk/ippMonitor
Files:
16 edited

Legend:

Unmodified
Added
Removed

  • trunk/ippMonitor/czartool/czarplot.pl

    r34249 r38258  
    3030my $log = undef;
    3131my $psps = undef;
     32my $project = 'gpc1';
    3233
    3334GetOptions (
    3435        "label|l=s" => \$label,
     36        "project=s" => \$project,
    3537        "stage|s=s" => \$stage,
    3638        "timeinpast|p=s" => \$timeinpast,
    … …  
    110112my $newDayTime =  $config->getMetricsStartTime();
    111113
     114my $projectDb = $config->getDbInstanceForProject($project);
     115die "Failed to open database for $project.\n" unless $projectDb;
     116
     117
    112118my $plotter = undef;
    113119if ($savingToFile) {
    114120
    115     $plotter = czartool::Plotter->new_file($config, $path, $save_temps);
     121    $plotter = czartool::Plotter->new_file($config, $projectDb, $path, $save_temps);
    116122}
    117123else {
    118124
    119     $plotter = czartool::Plotter->new_display($config, $save_temps);
     125    $plotter = czartool::Plotter->new_display($config, $projectDb, $save_temps);
    120126}
    121127

  • trunk/ippMonitor/czartool/czarpoll.pl

    r38251 r38258  
    2626
    2727GetOptions (
    28         "period|p=s" => \$period, # TODO more Db args
     28        "period|p=s" => \$period,
    2929        );
    3030
    … …  
    3535my $nebulous = new czartool::Nebulous($czarDb);
    3636my $pantasks = new czartool::Pantasks();
    37 my $plotter = czartool::Plotter->new_file($config, $config->getGnuplotPath(), $save_temps);
    38 
     37
     38# We have a top level directory that gets general plots and then one for each database
     39my $plotter0 = czartool::Plotter->new_file($config, undef, $config->getGnuplotPath(), $save_temps);
     40
     41# for now hardcode two databases. Later I'll look into handling a variable number
     42my $plotter1 = czartool::Plotter->new_file($config, $gpc1Db, $config->getGnuplotPath() . "/gpc1", $save_temps);
     43my $plotter2 = czartool::Plotter->new_file($config, $gpc2Db, $config->getGnuplotPath() . "/gpc2", $save_temps);
    3944
    4045my @stages = (
    … …  
    5964        "merged",
    6065        "deleted_local");
     66
     67# The ippToPsps pages are no longer operational
     68@ippToPspsStages = ();
    6169
    6270my @allStages = (@stages, @ippToPspsStages);
    … …  
    167175    while (1) {
    168176
     177        my $iterationStartTime = time();
     178
    169179        # sort out times
    170180        $today = strftime('%Y-%m-%d', localtime);
    … …  
    172182        $end = $czarDb->getNowTimestamp();
    173183
    174         # if before 18:00 today, then start plots from 18:00 yesterday
    175         if ($czarDb->isBefore($end, "$today $newDayTime")) {$begin = "$yesterday $newDayTime";}
    176         # if after 18:00 today, then perform some daily tasks and start plots from 18:00 today
    177         else {
     184        if ($czarDb->isBefore($end, "$today $newDayTime")) {
     185            # if before $newDayTime today (configured to 18:00), then start plots from $newDayTime yesterday
     186            $begin = "$yesterday $newDayTime";
     187        } else {
     188            # if after $newDayTime today (configured to 18:00), then perform some daily tasks and start plots from $newDayTime  today
    178189
    179190            $begin = "$today $newDayTime";
    … …  
    186197                # create metrics for last 24 hours
    187198                print "* Creating metrics for last 24 hours\n";
    188                 my $dayMetrics = new czartool::DayMetrics($config, 1, 0, $today);
     199                my $dayMetrics = new czartool::DayMetrics($config, $gpc1Db, 1, 0, $today);
    189200                $dayMetrics->writeHTML();
    190201
    191202                # now update metrics index page
    192                 my $metricsIndex = new czartool::MetricsIndex($config, 1, 0);
     203                my $metricsIndex = new czartool::MetricsIndex($config, $gpc1Db, 1, 0);
    193204                $metricsIndex->writeHTML();
    194205
    … …  
    210221        print "* Checking Nebulous\n";
    211222        $nebulous->updateClusterSpaceInfo();
    212         $plotter->plotStorageTimeSeries($czarDb->subtractInterval($begin, "1 WEEK") , $end);
    213         $plotter->plotDiskUsageHistogram();
    214         $plotter->plotDiskUsageHistogramLarge();
     223        $plotter0->plotStorageTimeSeries($czarDb->subtractInterval($begin, "1 WEEK") , $end);
     224        $plotter0->plotDiskUsageHistogram();
     225        $plotter0->plotDiskUsageHistogramLarge();
    215226        updateServerStatus();
    216227
    … …  
    239250                foreach $row ( @{$labels} ) {
    240251                    my ($label) = @{$row};
     252                    # for now use priorities from gpc1 database
    241253                    $priority = $gpc1Db->getPriority($label);
    242254                    $czarDb->setLabelPriority($label, $priority);
    … …  
    244256
    245257                updateAllStages($thisServer, $newState, $labels, $begin, $end);
    246                 if ($thisServer eq "stdscience") {createPlots($thisServer, $labels, $begin, $end);}
     258                if ($thisServer eq "stdscience") {
     259                    createPlots($thisServer, $plotter1, $labels, $begin, $end);
     260                    createPlots($thisServer, $plotter2, $labels, $begin, $end);
     261                }
    247262            }
    248263            else { print "* WARNING: no $thisServer labels found in Db\n";}
    249264        }
    250265
     266        my $iterationTime = time() - $iterationStartTime;
     267
     268        print "done with czarpoll iteration after $iterationTime seconds\n";
     269
    251270        print "--------------------------------------------------------------------------\n";
    252         #print "* Going to sleep\n";
    253         #sleep($period);
    254         #print "* Waking up\n";
     271        if ($iterationTime < $period) {
     272            print "* Going to sleep\n";
     273            sleep($period - $iterationTime);
     274            print "* Waking up\n";
     275        }
    255276    };
    256277}
    … …  
    262283###########################################################################
    263284sub createPlots {
    264     my ($server, $rows, $begin, $end) = @_;
     285    my ($server, $plotter, $rows, $begin, $end) = @_;
    265286
    266287    my $stage = undef;
    267288    my $row = undef;
    268289
    269     print "* Generating plots\n";
    270 
    271     # get ippToPsps labels
    272     my $ippToPspsLabels = undef;
    273     $czarDb->getCurrentIppToPspsLabels($begin, $end, \$ippToPspsLabels);
    274     $plotter->setIppToPspsMode();
    275 
    276     # create ippToPsps labels
    277     foreach $stage (@ippToPspsStages) {
     290    my $project = $plotter->getProject();
     291    print "* Generating plots for $project\n";
     292
     293    # XXX The ippToPsps stuff is not project specific so it should be done somewhere
     294    # else
     295    if (scalar @ippToPspsStages) {
     296        # get ippToPsps labels
     297        my $ippToPspsLabels = undef;
     298        $czarDb->getCurrentIppToPspsLabels($begin, $end, \$ippToPspsLabels);
     299        $plotter->setIppToPspsMode();
     300
     301        # create ippToPsps labels
     302        foreach $stage (@ippToPspsStages) {
     303            foreach $row ( @{$ippToPspsLabels} ) {
     304                my ($label) = @{$row};
     305
     306                chomp($label);
     307                $plotter->createTimeSeries($label, $stage, $begin, $end, 1, 0, 1);
     308            }
     309        }
     310
     311        # create plots for each label for all stages for ippToPsps
    278312        foreach $row ( @{$ippToPspsLabels} ) {
    279313            my ($label) = @{$row};
    280314
    281             chomp($label);
    282             $plotter->createTimeSeries($label, $stage, $begin, $end, 1, 0, 1);
    283         }
    284     }
    285 
    286     # create plots for each label for all stages for ippToPsps
    287     foreach $row ( @{$ippToPspsLabels} ) {
    288         my ($label) = @{$row};
    289 
    290         $plotter->createTimeSeries($label, undef, $begin, $end, 1, 1, 1);
    291     }
    292 
     315            $plotter->createTimeSeries($label, undef, $begin, $end, 1, 1, 1);
     316        }
     317    }
    293318
    294319    # create plots for each label for each stage
    295320    $plotter->setStandardMode();
     321
    296322    foreach $stage (@stages) {
    297323        foreach $row ( @{$rows} ) {
    298324            my ($label) = @{$row};
    299325
     326            # skip update labels for now. The plots are not relevant.
     327            next if $label =~ 'ps_ud';
     328
    300329            chomp($label);
    301330            $plotter->createTimeSeries($label, $stage, $begin, $end, 1, 0, 1);
    … …  
    308337    foreach $row ( @{$rows} ) {
    309338        my ($label) = @{$row};
     339
     340        # skip update labels for now. The plots are not relevant.
     341        next if $label =~ 'ps_ud';
    310342
    311343        $plotter->createTimeSeries($label, undef, $begin, $end, 1, 1, 1);
    … …  
    362394            chomp($label);
    363395
     396            # Move this to a function and pass in the database
    364397            $new = $gpc1Db->countExposures($label, $stage, $newState);
    365398            $full = $gpc1Db->countExposures($label, $stage, "full");

  • trunk/ippMonitor/czartool/czartool/Config.pm

    r38250 r38258  
    126126###########################################################################
    127127#
     128# Returns an instance of a database for the given project
     129#
     130###########################################################################
     131sub getDbInstanceForProject {
     132    my ($self, $project) = @_;
     133
     134    if ($project eq 'gpc1') {
     135        return $self->getGpc1Instance();
     136    } elsif ($project eq 'gpc2') {
     137        return $self->getGpc2Instance();
     138    } else {
     139        return undef;
     140    }
     141}
     142
     143###########################################################################
     144#
    128145# Returns the interval used in czarDB cleanup
    129146#

  • trunk/ippMonitor/czartool/czartool/CzarDb.pm

    r38250 r38258  
    214214###########################################################################
    215215#
    216 # Updates current_labels table
     216# Update server_dates table
    217217#
    218218###########################################################################
    … …  
    380380    my ($self, $label, $stage, $fromTime, $toTime, $minX, $maxX, $timeDiff) = @_;
    381381
     382    # XXX: I'm not splitting this by project (dbname).
     383    # Is that ok? Seems like a good idea because this will make the time
     384    # range of the plots for both projects the same.
     385
    382386    my $query = $self->{_db}->prepare(<<SQL);
    383387    SELECT
    … …  
    411415###########################################################################
    412416sub getDayTimings {
    413     my ($self, $label, $startStage, $endStage, $begin, $started, $finished, $timeTaken) = @_;
     417    my ($self, $label, $dbname, $startStage, $endStage, $begin, $started, $finished, $timeTaken) = @_;
    414418
    415419    my $end = $self->addInterval($begin, "1 DAY");
    416420
    417     my $startStageMetrics = new czartool::StageMetrics($startStage, $label, $begin, $end);
    418     my $endStageMetrics = new czartool::StageMetrics($endStage, $label, $begin, $end);
    419 
    420     if (!$self->runAnalysis($startStageMetrics)) {return 0;}
    421     if (!$self->runAnalysis($endStageMetrics)) {return 0;}
     421    my $startStageMetrics = new czartool::StageMetrics($startStage, $label, $dbname, $begin, $end);
     422    my $endStageMetrics = new czartool::StageMetrics($endStage, $label, $dbname, $begin, $end);
     423
     424    if (!$self->runAnalysis($startStageMetrics, $dbname)) {return 0;}
     425    if (!$self->runAnalysis($endStageMetrics, $dbname)) {return 0;}
    422426
    423427    ${$started} = $startStageMetrics->getStarted();
    … …  
    436440###########################################################################
    437441sub runAnalysis {
    438     my ($self, $stageMetrics) = @_;
     442    my ($self, $stageMetrics, $dbname) = @_;
    439443
    440444    if (!$stageMetrics) {return 0;}
    … …  
    445449    my $toTime = $stageMetrics->getEndTime();
    446450
     451
    447452    $toTime = $self->addInterval($toTime, "20 MINUTE"); # TODO dodgy
    448453
    … …  
    452457        FROM $stage
    453458        WHERE label LIKE '$label'
     459        AND dbname LIKE '$dbname'
    454460        AND timestamp >= '$fromTime' AND timestamp <= '$toTime'
    455461        ORDER BY timestamp;
    … …  
    656662###########################################################################
    657663sub createTimeSeriesData {
    658     my ($self, $label, $stage, $fromTime, $toTime, $minX, $maxX, $timeDiff, $linDataFile, $logDataFile, $ratDataFile) = @_;
     664    my ($self, $label, $dbname, $stage, $fromTime, $toTime, $minX, $maxX, $timeDiff, $linDataFile, $logDataFile, $ratDataFile) = @_;
    659665
    660666    # get total number of data points
    … …  
    663669        FROM $stage
    664670        WHERE label LIKE '$label'
     671        AND dbname LIKE '$dbname'
    665672        AND timestamp >= '$fromTime' AND timestamp <= '$toTime';
    666673SQL
    … …  
    680687        FROM $stage
    681688        WHERE label LIKE '$label'
     689        AND dbname LIKE '$dbname'
    682690        AND timestamp >= '$fromTime' AND timestamp <= '$toTime'
    683691        ORDER BY timestamp;
    … …  
    854862###########################################################################
    855863sub countFaultsInPast {
    856     my ($self, $label, $stage, $interval) = @_; # TODO use time not interval
     864    my ($self, $label, $dbname, $stage, $interval) = @_; # TODO use time not interval
    857865
    858866    # get earliest time for this label
    … …  
    860868    SELECT GREATEST(MIN(timestamp), now() - INTERVAL $interval)
    861869        FROM $stage
    862         WHERE label LIKE '$label';
     870        WHERE label LIKE '$label'
     871        AND dbname LIKE '$dbname';
    863872SQL
    864873    $query->execute;
    … …  
    870879        FROM $stage
    871880        WHERE label LIKE '$label'
     881        AND dbname LIKE '$dbname'
    872882        AND timestamp <= '$timestamp' LIMIT 1;
    873883SQL
    … …  
    883893###########################################################################
    884894sub countPendingNow {
    885     my ($self, $label, $stage) = @_;
     895    my ($self, $label, $dbname, $stage) = @_;
    886896
    887897    my $query = $self->{_db}->prepare(<<SQL);
    … …  
    889899        FROM $stage
    890900        WHERE label LIKE '$label'
     901        AND dbname LIKE '$dbname'
    891902        ORDER BY timestamp DESC LIMIT 1;
    892903SQL
    … …  
    902913###########################################################################
    903914sub countProcessedPendingAndFaults {
    904     my ($self, $label, $stage, $fromTime, $toTime, $processed, $pending, $faults) = @_;
     915    my ($self, $label, $dbname, $stage, $fromTime, $toTime, $processed, $pending, $faults) = @_;
    905916
    906917    my $query = $self->{_db}->prepare(<<SQL);
    … …  
    909920        FROM $stage
    910921        WHERE label LIKE '$label'
     922        AND dbname like '$dbname'
    911923        AND timestamp >= '$fromTime' AND timestamp <= '$toTime'
    912924        ORDER BY timestamp DESC LIMIT 1;
    … …  
    917929    (${$pending}, ${$faults}) = $query->fetchrow_array();
    918930
    919     ${$processed} = $self->countProcessed($label, $stage, $fromTime, $toTime);
     931    ${$processed} = $self->countProcessed($label, $dbname, $stage, $fromTime, $toTime);
    920932
    921933    if (!${$pending}) {${$pending} = 0;}
    … …  
    10661078###########################################################################
    10671079sub createProcessingRateData {
    1068     my ($self, $stage, $label, $begin, $end, $interval, $dataFile, $minX, $maxX, $timeDiff) = @_;
     1080    my ($self, $stage, $label, $dbname, $begin, $end, $interval, $dataFile, $minX, $maxX, $timeDiff) = @_;
    10691081
    10701082    my $startTime = $begin;
    … …  
    10891101        if (!$self->isBefore($startTime, $end)) {last;}
    10901102        $endTime = $self->addInterval($startTime, $interval);
    1091         if (!$self->countProcessedPendingAndFaults($label, $stage, $startTime, $endTime, \$processed, \$pending, \$faults)) {}
     1103        if (!$self->countProcessedPendingAndFaults($label, $dbname, $stage, $startTime, $endTime, \$processed, \$pending, \$faults)) {}
    10921104        $timestamp = $self->getFormattedDate($endTime);
    10931105        if (!$processed) {$processed = 0;}
    … …  
    11081120###########################################################################
    11091121sub countProcessed {
    1110     my ($self, $label, $stage, $begin, $end) = @_;
     1122    my ($self, $label, $dbname, $stage, $begin, $end) = @_;
    11111123
    11121124    my $query = $self->{_db}->prepare(<<SQL);
    … …  
    11151127        FROM $stage
    11161128        WHERE label LIKE '$label'
     1129        AND dbname LIKE '$dbname'
    11171130        AND timestamp >= '$begin'
    11181131        AND timestamp < '$end'

  • trunk/ippMonitor/czartool/czartool/DayMetrics.pm

    r38250 r38258  
    2929    my $self = $class->SUPER::new(
    3030            $_[1],  # config object
    31             $_[2],  # verbose
    32             $_[3]   # save_temps
     31            $_[2],  # projectDb
     32            $_[3],  # verbose
     33            $_[4]   # save_temps
    3334            );
    3435
    35     $self->{day} = $_[4];
     36    $self->{day} = $_[5];
    3637
    3738    my $yesterday =  $self->{czarDb}->subtractInterval($self->{day}, "1 DAY");
    … …  
    6869
    6970    # summit and chip exposures
    70     my $summitExposures = $self->{gpc1Db}->countScienceExposuresFromLastNight($self->{day});
    71     my $chipMetrics = new czartool::StageMetrics("chip", "all_stdscience_labels", $self->{begin}, $self->{chipEnd});
    72     $self->{czarDb}->runAnalysis($chipMetrics);
     71    my $summitExposures = $self->{projectDb}->countScienceExposuresFromLastNight($self->{day});
     72    my $dbname = $self->{projectDb}->getDbName();
     73    my $chipMetrics = new czartool::StageMetrics("chip", "all_stdscience_labels", $dbname, $self->{begin}, $self->{chipEnd});
     74    $self->{czarDb}->runAnalysis($chipMetrics, $dbname);
    7375
    7476    my $previousDay = $self->{czarDb}->subtractInterval($self->{day}, "1 DAY");
    … …  
    136138        foreach my $stage ( @stages ){
    137139
    138             my $stageMetrics = new czartool::StageMetrics($stage, $label, $self->{begin}, ($stage eq "chip") ? $self->{chipEnd} : $self->{end});
    139             if (!$self->{czarDb}->runAnalysis($stageMetrics)) {next;}
     140            my $stageMetrics = new czartool::StageMetrics($stage, $label, $dbname, $self->{begin}, ($stage eq "chip") ? $self->{chipEnd} : $self->{end});
     141            if (!$self->{czarDb}->runAnalysis($stageMetrics, $dbname)) {next;}
    140142
    141143            if (!$stageMetrics->anythingProcessed()) {next;}
    … …  
    268270    print $htmlFile "  </tr>\n";
    269271
     272    my $dbname = $self->{projectDb}->getDbName();
     273
    270274    # OSS survey
    271275    my ($started, $finished, $timeTaken) = undef;
    272     $self->{czarDb}->getDayTimings("OSS.nightlyscience", "chip", "diff", $self->{begin}, \$started, \$finished, \$timeTaken);
     276    $self->{czarDb}->getDayTimings("OSS.nightlyscience", $dbname, "chip", "diff", $self->{begin}, \$started, \$finished, \$timeTaken);
    273277    $self->printSurveyDetails("OSS", $started, $finished, $timeTaken);
    274278
    … …  
    295299    foreach $mdf (@mdfs) {
    296300
    297         if ($self->{czarDb}->getDayTimings($mdf, "chip", "dist", $self->{begin}, \$started, \$finished, \$timeTaken)) {
     301        if ($self->{czarDb}->getDayTimings($mdf, $dbname, "chip", "dist", $self->{begin}, \$started, \$finished, \$timeTaken)) {
    298302
    299303            if ($started && $self->{czarDb}->isBefore($started, $earliest)) {$earliest = $started; $haveStart = 1;}
    … …  
    309313    $self->printSurveyDetails("MDF", $earliest, $latest, $timeTaken);
    310314
    311     $self->{czarDb}->getDayTimings("all_stdscience_labels", "chip", "dist", $self->{begin}, \$started, \$finished, \$timeTaken);
     315    $self->{czarDb}->getDayTimings("all_stdscience_labels", $dbname, "chip", "dist", $self->{begin}, \$started, \$finished, \$timeTaken);
    312316    $self->printSurveyDetails("All", $started, $finished, $timeTaken);
    313317

  • trunk/ippMonitor/czartool/czartool/Gpc1Db.pm

    r38206 r38258  
    11#!/usr/bin/perl i-w
     2
     3# This class is now misnamed. Should be ProjectDb or something like that.
    24
    35package czartool::Gpc1Db;

  • trunk/ippMonitor/czartool/czartool/Metrics.pm

    r32093 r38258  
    2525    my $self = {
    2626        config => shift,
     27        projectDb => shift,
    2728        verbose => shift,
    2829        save_temps => shift,
    2930    };
    3031
    31     $self->{gpc1Db} = $self->{config}->getGpc1Instance();
     32    # $self->{gpc1Db} = $self->{config}->getGpc1Instance();
     33
    3234    $self->{czarDb} = $self->{config}->getCzarDbInstance();
    3335    $self->{baseDir} = $self->{config}->getMetricsSaveLocation();
    … …  
    3941    $self->{plotter} = czartool::Plotter->new_file(
    4042            $self->{config},
     43            $self->{projectDb},
    4144            ".",
    4245            $self->{save_temps});

  • trunk/ippMonitor/czartool/czartool/MetricsIndex.pm

    r32224 r38258  
    2929    my $self = $class->SUPER::new(
    3030            $_[1],  # config object
    31             $_[2],  # verbose
    32             $_[3]   # save_temps
     31            $_[2],  # database
     32            $_[3],  # verbose
     33            $_[4]   # save_temps
    3334            );
    3435

  • trunk/ippMonitor/czartool/czartool/MySQLDb.pm

    r32093 r38258  
    6565}                                                                               
    6666
     67sub getDbName {
     68    my( $self ) = @_;                                                       
     69    return $self->{_dbName};                                                 
     70}                                                                               
     71
    6772###########################################################################
    6873#

  • trunk/ippMonitor/czartool/czartool/Plotter.pm

    r38214 r38258  
    4646    my $self = {
    4747        _config => shift,
     48        _projectDb => shift,
    4849        _dateFormat => shift,
    4950        _outputFormat => shift,
    … …  
    5253    };
    5354
     55
    5456    bless $self, $class;
    5557    $self->init();
    … …  
    6668    my $self = {
    6769        _config => shift,
     70        _projectDb => shift,
    6871        _outputPath => shift,
    6972        _save_temps => shift,
    … …  
    8689    my $self = {
    8790        _config => shift,
     91        _projectDb => shift,
    8892        _save_temps => shift,
    8993    };
    … …  
    128132    my ($self) = @_;
    129133
    130     $self->{_gpc1Db} = $self->{_config}->getGpc1Instance();
     134    if ($self->{_projectDb}) {
     135        $self->{_project} = $self->{_projectDb}->getDbName();
     136    } else {
     137        $self->{_project} = undef;
     138    }
     139       
     140
    131141    $self->{_czarDb} = $self->{_config}->getCzarDbInstance();
    132142    $self->setStandardMode();
     143}
     144
     145###########################################################################
     146#
     147# Get the project (dbname) associated with this Plotter
     148#
     149###########################################################################
     150sub getProject {
     151    my ($self) = @_;
     152    return $self->{_project};
    133153}
    134154
    … …  
    209229                    $stage,
    210230                    $label,
     231                    $self->{_project},
    211232                    $beginTime,
    212233                    $endTime,
    … …  
    301322        if($self->{_czarDb}->createTimeSeriesData(
    302323                    $label,
     324                    $self->{_project},
    303325                    $stage,
    304326                    $beginTime,
    … …  
    403425        if (!$self->{_czarDb}->countProcessedPendingAndFaults(
    404426                $label,
     427                $self->{_project},
    405428                $stage,
    406429                $beginTime,
    … …  
    547570    $title .= " for '$label'\\nFrom '$fromTime' to '$toTime' HST";
    548571
    549     open (GP, "|$GNUPLOT -persist") or die "no gnuplot";
     572    open (GP, "|$GNUPLOT -persist > /dev/null 2>&1") or die "no gnuplot";
    550573    use FileHandle;
    551574    GP->autoflush(1);
    … …  
    747770
    748771    my $fracs;
    749     if (!$self->{_gpc1Db}->getMagicMaskFraction($exp_id, \$fracs)) {return 0;}
     772    if (!$self->{_projectDb}->getMagicMaskFraction($exp_id, \$fracs)) {return 0;}
    750773
    751774    open (GNUDAT, ">".$tmpFile->filename);
    … …  
    809832    # get exposures between these dates
    810833    my $exp_ids = undef;
    811     if (!$self->{_gpc1Db}->getProcessedExposures($begin, $end, \$exp_ids)) {return 0;}
     834    if (!$self->{_projectDb}->getProcessedExposures($begin, $end, \$exp_ids)) {return 0;}
    812835    my $totalExp = @{$exp_ids};
    813836    if ($totalExp < 1) {return 0;}
    … …  
    844867        $exp_id = @{$row}[0];
    845868
    846         if (!$self->{_gpc1Db}->getMagicMaskStats($exp_id, \$meanMask, \$sumMask, \$chipCount )) {next;}
     869        if (!$self->{_projectDb}->getMagicMaskStats($exp_id, \$meanMask, \$sumMask, \$chipCount )) {next;}
    847870
    848871

  • trunk/ippMonitor/czartool/czartool/StageMetrics.pm

    r32093 r38258  
    1717        stage => shift,
    1818        label => shift,
     19        dbname => shift,
    1920        startTime => shift,
    2021        endTime => shift
    … …  
    111112sub getEndTime() {my $self = shift; return $self->{endTime};}
    112113sub getLabel() {my $self = shift; return $self->{label};}
     114sub getDbname() {my $self = shift; return $self->{dbname};}
    113115sub getStage() {my $self = shift; return $self->{stage};}
    114116sub getStarted() {my $self = shift; return $self->{started};}

  • trunk/ippMonitor/raw/czartool_exposures.php

    r30563 r38258  
    66
    77if (! $ID['proj']) { projectform ($ID); }
    8 $gpc1db = dbconnect($ID['proj']);
     8$projectdb = dbconnect($ID['proj']);
    99
    1010$PATH = getenv("PATH");
    … …  
    4646echo "</form>\n";
    4747
    48 createTableForThisSurvey($gpc1db, $date, $survey);
     48createTableForThisSurvey($projectdb, $date, $survey);
    4949
    5050menu_end();

  • trunk/ippMonitor/raw/czartool_getplot.php

    r35391 r38258  
    1818$plottype = $_GET[plottype];
    1919
    20 $path = "/data/ipp009.0/czartool_plots"; # TODO horrible hard-coded path - read from config
     20$proj = $_GET[proj];
     21
     22$path = $CZARPLOTDIR;
    2123
    2224if ($label == "all") $label = "all_".$mode."_labels";
    2325
    2426if ($type=="t" || $type=="r" || $type=="rt" || $type=="rh")
    25 $filePath = "$path/czarplot_".$plottype."_".$label."_".$stage."_".$type.".png";
     27$filePath = "$path/$proj/czarplot_".$plottype."_".$label."_".$stage."_".$type.".png";
    2628else if ($type=="h")
    2729$filePath = "$path/czarplot_".$plottype."_".$label."_all_stages_".$type.".png";

  • trunk/ippMonitor/raw/czartool_labels.php

    r38257 r38258  
    88if (! $ID['proj']) { projectform ($ID); }
    99
    10 //$db = dbconnect($ID['proj']);
    11 $czardb = dbconnect("czardb"); // HACK to connect to czarDb
     10// connect to czarDb
     11$czardb = dbconnect($CZARDBNAME);
    1212$lastUpdateTime = getLastUpdateTime($czardb);
    1313
    14 $gpc1db = dbconnect($ID['proj']);
     14$projectdb = dbconnect($ID['proj']);
    1515
    1616$PATH = getenv("PATH");
    … …  
    9999$menu = $ID['menu'];
    100100
    101 $selectedMode = $_GET[mode];
     101// The Update mode of czartool is obsolete since we don't use the update
     102// pantasks.
     103// $selectedMode = $_GET[mode];
     104$selectedMode = "stdscience";
     105
    102106$selectedStage = $_GET[stage];
    103107$selectedLabel = $_GET[label];
    … …  
    135139$labels = getLabels($czardb, $selectedMode);
    136140$distLabels = getLabels($czardb, "distribution");
     141// publishing is in stdscience now
    137142# $pubLabels = getLabels($czardb, "publishing");
    138143$pubLabels = $labels;
    … …  
    145150}
    146151
     152// The Update mode of czartool is obsolete. selectedMode is hardcoded to stdscience above.
    147153$isUpdate = ($selectedMode == "update");
    148 $modeText = $isUpdate ? "Showing update processing" : "Showing standard processing";
    149 $modeLinkText = $isUpdate ? "standard" : "update";
    150 $modeLink = $isUpdate ? "stdscience" : "update";
    151 $link = "czartool_labels.php?pass=".$pass."&proj=".$proj."&mode=".$modeLink."&label=all&stage=" . $selectedStage . "&revertstage=" . $stage . "&plottype=$plotType";
     154$isUpdate = 0;
     155
     156// hijack the text and link at the top to create a link to the czartool monitoring the other project database
     157// I.E. Switch from gpc1 to gpc2 or gpc2 to gpc1
     158
     159$modeText = "Monitoring $proj";
     160if ($proj == 'gpc1') {
     161    $newproj = 'gpc2';
     162} else {
     163    $newproj = 'gpc1';
     164}
     165$link = "czartool_labels.php?pass=".$pass."&mode=".$selectedMode."&label=all&stage=" . $selectedStage . "&revertstage=" . $stage . "&plottype=$plotType";
    152166
    153167// set up the form
    … …  
    164178echo "<td colspan=\"3\" style=\"background-color:#0080c0;\">";
    165179echo "<h1 align=\"middle\">Czartool</h1>";
    166 echo "<h4 align=\"middle\">$modeText (change to <a href=\"$link\"><font color=\"blue\">$modeLinkText</font></a>)</h4>";
     180echo "<h2 align=\"middle\">$modeText (change to <a href=\"$link&proj=$newproj\"><font color=\"blue\">$newproj</font></a>)</h2>";
    167181echo "</td>";
    168182echo "</tr>";
    169183
     184// PLOTS
    170185echo "<tr valign=\"top\">";
    171186echo "<td style=width:100px;text-align:top;\">";
    172187 # time series plot
    173 echo "<br><img src=\"czartool_getplot.php?mode=$selectedMode&type=t&label=$selectedLabel&stage=$selectedStage&plottype=$plotType\"><br>";
     188echo "<br><img src=\"czartool_getplot.php?mode=$selectedMode&type=t&label=$selectedLabel&proj=$proj&stage=$selectedStage&plottype=$plotType\"><br>";
    174189 # rate time series plot
    175 echo "<br><img src=\"czartool_getplot.php?mode=$selectedMode&type=rt&label=$selectedLabel&stage=$selectedStage&plottype=linear\"><br>";
     190echo "<br><img src=\"czartool_getplot.php?mode=$selectedMode&type=rt&label=$selectedLabel&proj=$proj&stage=$selectedStage&plottype=linear\"><br>";
    176191 # rate stacked histo plot
    177 # echo "<img src=\"czartool_getplot.php?mode=$selectedMode&type=rh&label=$selectedLabel&stage=$selectedStage&plottype=linear\"><br>";
     192# echo "<img src=\"czartool_getplot.php?mode=$selectedMode&type=rh&label=$selectedLabel&proj=$proj&stage=$selectedStage&plottype=linear\"><br>";
    178193 # histogram plot
    179 # echo "<img src=\"czartool_getplot.php?mode=$selectedMode&type=h&label=$selectedLabel&stage=$selectedStage&plottype=linear\"><br>";
     194# echo "<img src=\"czartool_getplot.php?mode=$selectedMode&type=h&label=$selectedLabel&proj=$proj&stage=$selectedStage&plottype=linear\"><br>";
    180195 # storage plot
    181196echo "<br><a href=\"czartool_getplot.php?type=sl\"><img title=\"Click to enlarge me\" src=\"czartool_getplot.php?type=s\"></a><br>";
    … …  
    186201 if ($selectedServer && !$serverCmd) showServerStatus($selectedServer);
    187202echo "</td>";
     203
     204
    188205echo "<td style=height:200px;width:10px;text-align:top;\">";
    189206echo "<td style=height:200px;width:400px;text-align:top;\">";
    … …  
    240257      createServersTable($pass, $proj,$selectedMode,$czardb, $servers, $selectedLabel, $selectedStage, $plotType);
    241258echo "<br>";
    242       createSummitDataTable($gpc1db);
     259      createSummitDataTable($projectdb);
    243260echo "<br>";
    244261      createDatesTable($czardb);
    … …  
    253270      createTableColumnHeader("Status (sec behind master)");
    254271      showReplicationsStatus($REPL_HOST_GPC1, $REPL_USER_GPC1, $REPL_PASSWORD_GPC1, $REPL_DBNAME_GPC1);
     272
     273// XXX: avoid replication status checking errors due to missing passwords from my test host
     274// XXX: TODO: remove this conditional before committing changes
     275if (0) {
    255276      # XXX EAM : problem showReplicationsStatus($REPL_HOST_NEBULOUS, $REPL_USER_NEBULOUS, $REPL_PASSWORD_NEBULOUS, $REPL_DBNAME_NEBULOUS);
    256277      showReplicationsStatus($REPL_HOST_PSTAMP, $REPL_USER_PSTAMP, $REPL_PASSWORD_PSTAMP, $REPL_DBNAME_PSTAMP);
    … …  
    258279      showReplicationsStatus($REPL_HOST_ISP, $REPL_USER_ISP, $REPL_PASSWORD_ISP, $REPL_DBNAME_ISP);
    259280      showReplicationsStatus($REPL_HOST_NEBULOUS_SECONDARY, $REPL_USER_NEBULOUS_SECONDARY, $REPL_PASSWORD_NEBULOUS_SECONDARY, $REPL_DBNAME_NEBULOUS_SECONDARY);
     281}
    260282      echo "</table>";
    261283
    … …  
    315337#
    316338###########################################################################
    317 function createSummitDataTable($gpc1db) {
     339function createSummitDataTable($projectdb) {
    318340
    319341    $date = date("Y-m-d");
    320342    $sql = "SELECT DISTINCT exp_type FROM summitExp WHERE dateobs > '$date'";
    321343
    322     $qry = $gpc1db->query($sql);
     344    $qry = $projectdb->query($sql);
    323345    if (dberror($qry)) {
    324346        echo "<b>error reading newExp table</b><br>\n";
    … …  
    343365
    344366        $sql = "SELECT COUNT(*) FROM summitExp WHERE dateobs > '$date' AND exp_type = '$expType[0]'";
    345         $qry2 = $gpc1db->query($sql);
     367        $qry2 = $projectdb->query($sql);
    346368        $qry2->fetchInto($summit);
    347369        $sql = "SELECT COUNT(*) FROM summitExp JOIN rawExp ON summitExp.exp_name = rawExp.exp_name WHERE summitExp.dateobs > '$date' AND summitExp.exp_type = '$expType[0]'";
    348         $qry2 = $gpc1db->query($sql);
     370        $qry2 = $projectdb->query($sql);
    349371        $qry2->fetchInto($mhpcc);
    350372
    … …  
    373395function getLabels($db, $server) {
    374396
    375     # order by descending priority as set in gpc1 database
     397    # order by descending priority as set in project database
    376398    $sql = "SELECT label FROM current_labels WHERE server LIKE '$server' ORDER BY priority DESC";
    377399    if ($debug) {echo "$sql<br>";}
    … …  
    490512          $myTable    = $mainTable[$stage];
    491513          $myFailures = $getFailures[$stage];
    492           # echo "myTable: $myTable";
     514          # echo "myTable: $myTable";
    493515
    494516          if ("$myFailures" == "NONE") {
    … …  
    501523            }
    502524          }
    503           getStateAndFaults($db, $thisLabel, $selectedState, $stage, $str, $anyFaults);
     525          getStateAndFaults($db, $thisLabel, $proj, $selectedState, $stage, $str, $anyFaults);
    504526
    505527          $cellcolor = 0;
    … …  
    647669#
    648670###########################################################################
    649 function getStateAndFaults($db, $label, $state, $stage, &$str, &$anyFaults) {
     671function getStateAndFaults($db, $label, $proj, $state, $stage, &$str, &$anyFaults) {
    650672
    651673    $anyFaults = false;
    652     $sql = "SELECT pending, faults FROM $stage WHERE label LIKE '$label' ORDER BY timestamp DESC LIMIT 1";
     674    $sql = "SELECT pending, faults FROM $stage WHERE label LIKE '$label' AND dbname LIKE '$proj' ORDER BY timestamp DESC LIMIT 1";
    653675    if ($debug) {echo "$sql<br>";}
    654676

  • trunk/ippMonitor/raw/ipp.czar.dat

    r33846 r38258  
    44
    55menulink   | menuselect      | link    | czartool - standard          | czartool_labels.php
    6 menulink   | menuselect      | link    | czartool - ipptopsps         | czartool_ipptopsps.php
    76menulink   | menuselect      | link    | mask stats                   | maskStats.php
    87menulink   | menuselect      | link    | night summary                | nightSummary.php

  • trunk/ippMonitor/raw/ipp.php

    r37983 r38258  
    6969  global $DBUSER;
    7070  global $DBPASS;
     71  global $CZARDBNAME;
     72  global $CZARDBHOST;
    7173
    7274  $success = 0;
    … …  
    8688  // connect to the database
    8789  if ($DBI == "DB") {
    88     $db = DB::connect("mysql://$DBUSER:$DBPASS@$DBHOST/$database");
     90    if ($database == $CZARDBNAME) {
     91        $db = DB::connect("mysql://$DBUSER:$DBPASS@$CZARDBHOST/$database");
     92    } else {
     93        $db = DB::connect("mysql://$DBUSER:$DBPASS@$DBHOST/$database");
     94    }
    8995  }
    9096  if ($DBI == "MDB2") {
    … …  
    192198function checkID () {
    193199
     200
    194201  $VERBOSE = 0;
     202  if ($VERBOSE > 1) {
     203      phpinfo(32);
     204  }
    195205
    196206  // check for valid server method
Note: See TracChangeset for help on using the changeset viewer.