Changes between Version 4 and Version 5 of PS1_IPP_Czarlog_20170918

Timestamp:

Sep 23, 2017, 10:13:26 AM (9 years ago)

Author:

eugene

Comment:

--

PS1_IPP_Czarlog_20170918

@@ -17 +17 @@
 === Friday : YYYY.MM.DD ===
 
-=== Saturday : YYYY.MM.DD ===
+=== Saturday : 2017.09.23 ===
+ * 10:15 EAM : some problems with nebulous / neb-host and incorrect config.  Mark sent the following:
+{{{
+strangely it looked like ipp100.0,103.0,104.0 were not getting data last night even though they have been neb-host up for past few days (and ipp076 has been getting data while in repair) -- it appears the ~ippitc account was not updated for ippdb01->ippdb09 when switch was made
+
+/data/ippc64.1/ippitc/psconfig/ipp-20170121.lin64/bin/neb-host
+uses
+/data/ippc64.1/ippitc/ippconfig/nebulous.config
+has set
+NEB_DBHOST  STR    ippdb01.ifa.hawaii.edu
+
+don't know the full implications of this, so am not making any changes to anything, but suggest the config (as well as for ~ipp it seems) be returned to the dns one nebulous.ipp.ifa.hawaii.edu one
+}}}
+I explored and wrote back:
+{{{
+ I dug into things a bit further and I'm not completely certain of the correct solution, but I have at least a short term fix.
+
+I believe ipp and ippitc only use that config value for neb-host.
+
+the ipp user, which runs nebdiskd, uses the config values in /etc/nebdiskd.rc on the machine where nebdiskd is running (ipp117) -- that is pointing at ippdb09.
+
+the apache nodes, which do all of the other interactions, are all pointing at ippdb09.
+
+the fact that both ipp and ippitc were pointing at ippdb01 meant that the changes they made to the 'mountedvol' tables (using neb-host) were not being seen by the apache nodes.  Thus, the apache nodes continues to respect the previous settings.
+
+Fortunately, the error did not affect the mysql replication process.
+
+I tried to change the two config files to point at nebulous.ipp.ifa.hawaii.edu, but this did not work.  I'm not certain why this did not work.  I am able to get neb-host to work using ippdb09, but nebulous.ipp does not, even though it resolves to ippdb09 correctly.
+
+(Also note that I can connect to mysql using the mysql shell with the machine name nebulous.ipp, so this problem is somewhere in the perl DBI layer or our use of it).
+
+I have set both ~ipp and ~ippitc to point at ippdb09 for now.  
+
+I have also confirmed that changes we make to ippdb09 with neb-host are propagated to ippdb01 via replication.
+}}}
 
 === Sunday : YYYY.MM.DD ===