| | 14 | * 07:00 Bill lots of red on czartool. Summit copy and registration are far behind. The problem appears to be that nebulous is overloaded and not responding in a timely fashion. Stopped most processing to investigate. Sample error messages |
| | 15 | {{{ |
| | 16 | 500 Can't connect to ippdb00.ifa.hawaii.edu:80 (connect: timeout) at /home/panstarrs/ipp/psconfig//ipp-20110218.lin64/lib/Nebulous/Client.pm line 852 |
| | 17 | Unable to perform neb-locate: 9 at /home/panstarrs/ipp/psconfig//ipp-20110218.lin64/bin/summit_copy.pl line 233. |
| | 18 | |
| | 19 | |
| | 20 | Unable to stat Nebulous handle neb://ipp030.0/gpc1/20110315/o5635g0395o.310644/o5635g0395o.310644.reg.ota37.log at /home/panstarrs/ipp/psconfig//ipp-20110218.lin64/bin/register_imfile.pl line 61 |
| | 21 | }}} |
| | 22 | * cleanup has a very large backlog of jobs. It looks like it's in the mode where it takes 10 minutes per (sparse) chipRun. Replication was running as well. With just summit copy, registration, and cleanup running the errors have stopped. |
| | 23 | * 07:09 restarting stdscience and distribution. |