Netonix Forums

We have this switch Randomly rebooting. When It reboots , logs show the following messages:





The check cable results during switch operational state are all Ok, while after each reboot, the results seen above.
FYI; ports 6,7,8,9 are serving as MIDSPAN for Rockets 5AC Prism.

The I2C timeout issue discussed in this post viewtopic.php?f=6&t=3621#p23402 could affect WS-12-250-DC as well but since the cable management of the WS-12-250-DC is done differently and since the I2C cable in the WS-12-250-DC is nowhere near the fan cable there is far less chance of the issue exhibiting itself but possible if the I2C cable is lose or possibly a bad SFP module.

What I want you to do is upgrade to v1.5.0rc2.
Then if the unit reboots look for the Detected cold (watchdog) boot in the log.

Report back what happens after upgrading to v1.5.0rc2, let me know if the problem is still there or has gone away.

The POE Smart cable check results are explained here: viewtopic.php?f=6&t=1178#p8809


It is also possible you have a bad radio? Recently a user reported a bad airMAX AC radio causing his switch to reboot due to drawing too much current, he replaced the radio and the problem went away.
viewtopic.php?f=17&t=2675&p=18795&hilit=+Rocket+bad+replaced#p18795

Also are you using Ethernet Surge protectors? If so take them OUT. Do not use Ethernet Surge Protectors with "passive" POE switches. This has been discussed on this forums and over on UBNT Forums.

viewtopic.php?f=6&t=3431&p=22490&hilit=+Rocket+bad+replaced#p22499

This user had water in a surge protector and or cable causing issues as it was causing a short and drawing too many watts causing a reboot when the power was drawn down.
viewtopic.php?f=6&t=3431&p=22490&hilit=+Rocket+bad+replaced#p22503

Hi @sirhc

I upgraded to 1.5.0rc2 as you asked. Will keep monitoring and see

We don't use surge protectors. Also, not sure if its a bad radio. We will only wait and see how switch will behave after upgrade

Thank you

We are looking further into an issue that affects WS-12-250-DC and WS-26-500-DC models.

There may be a issue we missed that we hope to have an answer to this soon, we pulled v1.5.0rc2 until we further investigate this.

Please stay tuned and we apologize for this but we are working hard to resolve this as soon as possible.

Hi,

We run dozens of 12-250DC in our network all in same configuration. We have had two reboot with no recordable reason now to date, one at 2am in the morning and one during prime time 9pm. One had 1.4.6 and one 1.4.7. Have not done it since.

FYI - not that it really helps but happy to provide more info if needed,

Sam

I just wondered whether there has been any progress on eliminating these 'random reboots' as we are severely suffering from it on our network at the moment. We have about 70 Netonix switches and they are all rebooting seemingly at random, causing their connected radios to be power cycled. Sometimes we can go 3-4 days with no problems, and other days we can see many tens of switches rebooting. It's turning into a disaster zone. On our remote syslog server I am seeing 'Detected cold (watchdog) boot'. Nothing is syslogged before the reboot occurs.

Last month we were running 1.4.9 on all switches apart from 2-3 units where we were trialling 1.5.0. After the first major 'reboot storm' (described in this thread) I upgraded all switches to 1.5.0 to try to prevent recurrence but the reboots seem to have got more frequent if anything, although that might be a co-incidence. We have around 7 Mikrotik switches which occasionally log packet loops ("etherX: bridge port received packet with own address as source address, probably loop") but I'm not sure whether this is an 'erroneous error' or if there is a genuine intermittent loop / storm and whether it's possibly causing the Netonix reboots.

Should I try enabling Loop Protection on all our Netonix Switches? What about the other Storm Control settings i.e. Broadcast / Multicast / Unicast / Pause Frame? Is there any downside or risk to enabling these settings?

Thanks
Glenn

I don't have any of my WS-12-250-DC "randomly" rebooting?

Are these units hooked directly to batteries or to a power supply/charger?
I have my units hooked directly to the batteries.

Are you running v1.5.0 or v1.5.1rc7?
I am running v1.5.0 or v1.5.1rc7.

Thanks sirhc - at the moment I'm suspecting that a network loop and/or bad plug/cable/radio is to blame.

In answer to your questions: some units are hooked directly to 12V batteries and then we have smart chargers also connected to the batteries to keep them trickle charged. However, the issue is affecting both battery and grid-powered switches.

We are running v1.5.0. I'm undecided whether to stick with 1.5.0 or downgrade back to 1.4.9 or upgrade to 1.5.1rc7.

Today I've been enabling loop protection on selected switches and it seems to be pointing me toward a certain part of the network based on the ports on which loops are being detected. So maybe I'm making some progress.

Thanks
Glenn

I also started to experience random reboots on WS-12-250-DC. 1.5.0

It happen right after I added two additional devices and maxed out this switch ... 11 ports are 24v and one port is 48HV. I started to think that maybe there is too much power draw on the 24v bus. How can we find the real cause of the reboot ?

kdsnetworks wrote:I also started to experience random reboots on WS-12-250-DC. 1.5.0

It happen right after I added two additional devices and maxed out this switch ... 11 ports are 24v and one port is 48HV. I started to think that maybe there is too much power draw on the 24v bus. How can we find the real cause of the reboot ?

I'm still investigating my own problems with this but my opinion is that Netonix switches can freeze up and then watchdog-reboot when they're saturated with broadcast traffic, possibly caused by a packet storm or network loop. Try enabling loop protection and see if your switch detects a loop (it will then disable the port for 3 minutes). See if it's always the same port on which the 'loop' (more accurately 'storm', in my opinion), is detected.

Maybe you could try downgrading to 1.4.9 to see if it helps? My own problems started/got worse when I upgraded to 1.5.0 but this may have been a co-incidence. Nonetheless I'm thinking about trying a downgrade myself but I have 70+ switches so it's not an easy decision.

Glenn

*** EDIT: In my case it seems to have been a faulty SFP module to blame (see next post).