PS1 IPP Czar Logs for the week 2019.05.10 - 2019.05.16

(Up to PS1 IPP Czar Logs)

Friday : 2019.05.10

CZAR HANDOVER by CCL

• Haydn mentioned he will check them on 5/13 (remind him maybe)
• ipp128: Device: /dev/bus/0 [megaraid_disk_15], Read SMART Self-Test Log Failed
• ipp139: Device: /dev/bus/0 [megaraid_disk_37], Read SMART Self-Test Log Failed
• ippc72 is down: details mentioned above, not sure what the next action will be (another hardware issue)
• ippc129, ippc135, ippc145, ippc149, ippc150: drive issue

• update.LAP.PV3: 15k warp prcessing jobs on ippmonitor, some of them might have been processed to be "full" state by queue their data_state in the warpSkyfile table. Their data quality is not equal to 0 which may affect the state keep "update".
• QUB have put in a lot of stamp requests, you may need to restart pstamp a couple more times today (when jobs done >100k).
• these huge jobs may cause ipp restart unsuccessfully (stdscience happened twice), make sure all server are running by check_system.sh

Saturday : 2019.05.11

Sunday : 2019.05.12

Monday : 2019.05.13

Tuesday : 2019.05.14

Wednesday : 2019.05.15

Thursday : 2019.05.16