----------------------------------------------------------------------------- Mauna Loa Solar Observatory Observer's Log ----------------------------------------------------------------------------- Tue Jun 24 16:39:58 GMT 1997 Year: 97 Doy: 175 Observer: koon WEATHER COMMENT: Tue Jun 24 16:40:02 GMT 1997 Scattered thin cirrus, wind=10 mph from the SSE, temp=47 F. Tue Jun 24 16:40:52 GMT 1997 CHIP Startup--Initializing new tape COMMENT: Tue Jun 24 17:47:30 GMT 1997 I have not been able to get the observing programs to startup due to some type of local network problem. Still trying to figure it out, unfortunately we can't take any data unless the local ethernet network is working correctly. COMMENT: Tue Jun 24 20:00:55 GMT 1997 I'm still trying to get CHIP and dPMon to work. At first I kept getting Optics:Busy type crashes, usually these mean there is a loose Ethernet connection and in the past very often I couldn't "ping" kii or nene when this type of crash took place. This time I could "ping" successfully, so I tried Kill/Run type cycles but could never get beyond the "Optics:Busy" crashing. Then I couldn't even get "Run" to work on Nene, after repeated tries I reset the MCC to see if that would help. Then the MCC crashed with the red "Fail" LEDs lit up. Cycling the MCC power wouldn't clear the Fails. Then I checked all the Ethernet connections because I was then unable to "ping" Kii and Nene. Connections were good. I rebooted Kii and Nene then cycled power to MCC. After that I could get all the way back up to the "Optics:Busy" crashes that I had initially. I've noticed a lot of network slowdowns and crashes lately on our side (in Hawaii) and there have also been al lot of CHIP and dPMon crashes at the same time, and today when I test using "ping" I'm usually getting a 50% packet loss rate, thats the worst I've ever seen for more than two consecutive 10 packet tests. I will continue to test the network and see if there is something else that can be done to fix the problem and/or isolate us from the fiber optic link if that is causing the problems. Tue Jun 24 20:31:00 GMT 1997 dPMon Start Patrol Tue Jun 24 20:31:04 GMT 1997 CHIP CHIP Start Patrol CHIP COMMENT: Tue Jun 24 20:37:58 GMT 1997 Taking data finally. See comment below. DPMON COMMENT: Tue Jun 24 20:38:47 GMT 1997 Taking data, see comment below. COMMENT: Tue Jun 24 20:39:05 GMT 1997 After isolating the local Ethernet from the Fiber Optic repeater I was able to get the observing programs to start up, so the problem appears to be a drastically slowed down network up here on the mountain but not due to us at MLSO. Perhaps the observing programs are timing out too soon on some of their communications and with a slow network that is causing the programs to crash. I tried this fix earlier without success but that was before I got the MCC Fail LEDs to clear (unlit) and before I rebooted Kii and Nene. I'll talk with the NOAA network administrator to help troubleshoot that slowdown problem since that affects our communications with the outside world. Tue Jun 24 21:02:03 GMT 1997 CHIP Bias Tue Jun 24 21:06:13 GMT 1997 dPMon Start Patrol Tue Jun 24 21:08:43 GMT 1997 CHIP CHIP Start Patrol COMMENT: Tue Jun 24 21:06:48 GMT 1997 I talked to NOAA, they said that they speeded up the network by reseting the Fiber Optic Hub and that I should try reconnecting to the F.O. link. So I did and had two Optics:Busy crashes, these might have happened before the switch, I'll try again to see if connecting to the F.O. link is causing those crashes. "Ping" packet losses were 50% also. Disconnected the FO link and did Kill/Run to fix crashes. Both programs are running without any problems right now. COMMENT: Tue Jun 24 21:13:47 GMT 1997 While everything is running correctly I will try reconnecting our local network to the FO repeater and see if there are any crashes. COMMENT: Tue Jun 24 21:15:40 GMT 1997 OK, we're back on the FO link. Both programs have crashed, CHIP with Optics:Busy, dPMon with Program:H-alpha.disc and Optics:Idle. Network appears slow but is up, I can "ping" puu in Boulder. I'll try to restart the observing programs with the FO link still in place. Tue Jun 24 21:31:06 GMT 1997 CHIP CHIP Start Patrol Tue Jun 24 21:31:23 GMT 1997 dPMon Start Patrol COMMENT: Tue Jun 24 21:31:22 GMT 1997 I couldn't get "Run" to work for dPMon or CHIP until I disconnected the FO link, now sure that the link causes our crashes. Talked to NOAA they will try disconnecting everyone from their FO Hub to see if the network works OK, then will try reconnecting everyone up here to the Hub again to see if any one connection is pulling everything down to a crawl. Until this is resolved we will not have any network connection to Boulder for image transfers or file transfers, etcetera. WEATHER COMMENT: Tue Jun 24 21:42:38 GMT 1997 Cirrus clouds are starting to pass over. Tue Jun 24 22:01:58 GMT 1997 dPMon Flat Tue Jun 24 22:02:03 GMT 1997 CHIP Bias Tue Jun 24 22:02:47 GMT 1997 CHIP End Bias Tue Jun 24 22:02:54 GMT 1997 dPMon End Flat Tue Jun 24 22:02:53 GMT 1997 CHIP Water Tue Jun 24 22:03:26 GMT 1997 CHIP End Water Tue Jun 24 22:20:10 GMT 1997 dPMon End Patrol Tue Jun 24 22:21:37 GMT 1997 CHIP CHIP End Patrol Tue Jun 24 22:58:54 GMT 1997 CHIP ending tape COMMENT: Tue Jun 24 22:58:45 GMT 1997 Activity report: QP: 42; 123; 235; 301-317. No coronal activity. TAPES: ****** MKIII: Still down for computer repairs DPMON: P00919 CHIP: C00340 LOWL: L00461 in drive #0