Page 1 of 1
Random Reboots
Posted: Tue Dec 13, 2016 9:09 pm
by RebusCom
I recently installed a new WS-12-250-AC, upgrading an original 8-port. It is running v1.4.6. It powers an AF5X on port 1, three rockets and a nanostation. The rest are unpowered. It's in a cool indoor location, about 40F, and the fan doesn't run most of the time. It operated fine for a few days, then started rebooting randomly, about an hour apart. This gradually increased in frequency to about every 20 minutes. Traffic would stop on the switch and it would become unresponsive. The fan would start cycling on and off and the switch would come back up after several minutes but the AF5X would reboot also. None of the other powered devices appeared to reboot. The rebooting of the AF5X was not due to watchdog on the radio or the switch (both off) but was due apparently to power interruption.
I tried powering the AF5X externally but that made no difference to the switch. It still randomly reboots. The switch is on a monitored UPS system, which shows steady output voltage.
Gone bad?
Re: Random Reboots
Posted: Tue Dec 13, 2016 9:24 pm
by sirhc
Yes, sounds like the switch power supply is defective.
If you RMA it we will replace the power supply.
Re: Random Reboots
Posted: Tue Dec 13, 2016 9:36 pm
by RebusCom
Thanks for the quick reply.
As a follow-up, I got the switch on the bench and it ran for about ten minutes and now it appears stuck in a perpetual reboot cycle which it can never complete. RMA time.
Re: Random Reboots
Posted: Sun Jan 29, 2017 5:45 pm
by RebusCom
sirhc wrote:Yes, sounds like the switch power supply is defective.
If you RMA it we will replace the power supply.
I reinstalled this switch (WS-12-250-AC) after the RMA power supply replacement. Every since installation the POE radios, with the exception of one, have been randomly rebooting. Typical behavior is a couple days will go by, then each will reboot within a couple hours of each other. Occasionally a particular radio will reboot a couple of times roughly half an hour to an hour apart intermixed with other reboots. Then a couple days typically go by without any reboots. The radios that are rebooting are Rocket M's. The one that isn't rebooting is the sole AF5x. There is nothing in the log related to ping watchdog -- just unexpected link change and the usual stuff that follows a radio reboot. The switch is on UPS and the UPS hasn't logged any power anomalies.
It's possible there's a coincidence but the random rebooting doesn't occur when a v1 WS-8-250-AC is used, driving the same POE devices. I'm thinking for troubleshooting purposes it would be good to continuously log the board voltages, though the reporting interval would probably have to be very short to be useful for that purpose. Any recommendations on the best way to do that?
Re: Random Reboots
Posted: Sun Jan 29, 2017 6:36 pm
by sirhc
No, I do not think this is switch related.
I would look at cabling, bad or loose connector.
Check for ping watchdog in radio
Re: Random Reboots
Posted: Sun Jan 29, 2017 7:52 pm
by RebusCom
It's not ping watchdog in the switch or radios. It occurs even when watchdog is disabled at both ends. Three Rockets started this behavior simultaneously, coincident with the replacement of the switch from the 8 port to the 12 port. The AF5x hasn't exhibited the behavior though. It began slowly, a few days between incidents and one or two radios about the same time, increasing in frequency to every couple of days now where all three will reboot close together but not at exactly the same time, and occasionally now one of the three will reboot a second time within about an hour. Then things settle down again for a couple days. Puzzling.
The Rockets are generating crash reports, with random errors such as page faults, etc., which theoretically could occur with voltage dips. So I was wondering if there was a way to monitor the voltages for stability as a troubleshooting step to see if there is a correlation or this is some freak coincidence of unstable radios, all of which have been in service for years. Just trying to rule things out. So far the switch has not rebooted itself like it was doing frequently before the RMA. That's a good thing.
The switch is in a cool 38-40F indoor environment. The fan is not running but the temps are okay.
Re: Random Reboots
Posted: Sun Jan 29, 2017 8:27 pm
by sirhc
If the switch 24V was dipping it would have a log entry saying 24V is out of range, is there any such entries?
I do not think this is the switch.
Is your tower ground rods bonded to electrical service ground rods? Excessive ground current can do weird things.
If you so sure it is the switch take it back to office and plug several radios in and let it run overnight.
Re: Random Reboots
Posted: Sun Jan 29, 2017 10:46 pm
by RebusCom
sirhc wrote:If the switch 24V was dipping it would have a log entry saying 24V is out of range, is there any such entries?
I do not think this is the switch.
Wasn't aware of the logging of out of range voltage. What is considered out-of-range?
I will check for log entries coincident with future reboots. BTW, the E-mailing of logs is a nice feature.
Re: Random Reboots
Posted: Sun Jan 29, 2017 11:37 pm
by sirhc
I think between 23V and 22V, sorry, at home and forget