WS-12-250-DC random reboot!!

DOWNLOAD THE LATEST FIRMWARE HERE
issamfayad
Member
 
Posts: 2
Joined: Fri Mar 03, 2017 4:36 am
Has thanked: 0 time
Been thanked: 0 time

WS-12-250-DC random reboot!!

Sun Mar 25, 2018 9:25 am

Image




Image



Image



We have this switch Randomly rebooting. When It reboots , logs show the following messages:

Image

Image

The check cable results during switch operational state are all Ok, while after each reboot, the results seen above.
FYI; ports 6,7,8,9 are serving as MIDSPAN for Rockets 5AC Prism.

User avatar
sirhc
Employee
Employee
 
Posts: 7416
Joined: Tue Apr 08, 2014 3:48 pm
Location: Lancaster, PA
Has thanked: 1608 times
Been thanked: 1325 times

Re: WS-12-250-DC random reboot!!

Sun Mar 25, 2018 10:58 am

The I2C timeout issue discussed in this post viewtopic.php?f=6&t=3621#p23402 could affect WS-12-250-DC as well but since the cable management of the WS-12-250-DC is done differently and since the I2C cable in the WS-12-250-DC is nowhere near the fan cable there is far less chance of the issue exhibiting itself but possible if the I2C cable is lose or possibly a bad SFP module.

What I want you to do is upgrade to v1.5.0rc2.
Then if the unit reboots look for the Detected cold (watchdog) boot in the log.

Report back what happens after upgrading to v1.5.0rc2, let me know if the problem is still there or has gone away.

The POE Smart cable check results are explained here: viewtopic.php?f=6&t=1178#p8809
Image

It is also possible you have a bad radio? Recently a user reported a bad airMAX AC radio causing his switch to reboot due to drawing too much current, he replaced the radio and the problem went away.
viewtopic.php?f=17&t=2675&p=18795&hilit=+Rocket+bad+replaced#p18795

Also are you using Ethernet Surge protectors? If so take them OUT. Do not use Ethernet Surge Protectors with "passive" POE switches. This has been discussed on this forums and over on UBNT Forums.

viewtopic.php?f=6&t=3431&p=22490&hilit=+Rocket+bad+replaced#p22499

This user had water in a surge protector and or cable causing issues as it was causing a short and drawing too many watts causing a reboot when the power was drawn down.
viewtopic.php?f=6&t=3431&p=22490&hilit=+Rocket+bad+replaced#p22503
Support is handled on the Forums not in Emails and PMs.
Before you ask a question use the Search function to see it has been answered before.
To do an Advanced Search click the magnifying glass in the Search Box.
To upload pictures click the Upload attachment link below the BLUE SUBMIT BUTTON.

issamfayad
Member
 
Posts: 2
Joined: Fri Mar 03, 2017 4:36 am
Has thanked: 0 time
Been thanked: 0 time

Re: WS-12-250-DC random reboot!!

Tue Mar 27, 2018 7:14 am

Hi @sirhc

I upgraded to 1.5.0rc2 as you asked. Will keep monitoring and see

We don't use surge protectors. Also, not sure if its a bad radio. We will only wait and see how switch will behave after upgrade

Thank you

User avatar
sirhc
Employee
Employee
 
Posts: 7416
Joined: Tue Apr 08, 2014 3:48 pm
Location: Lancaster, PA
Has thanked: 1608 times
Been thanked: 1325 times

Re: WS-12-250-DC random reboot!!

Tue Mar 27, 2018 9:37 am

We are looking further into an issue that affects WS-12-250-DC and WS-26-500-DC models.

There may be a issue we missed that we hope to have an answer to this soon, we pulled v1.5.0rc2 until we further investigate this.

Please stay tuned and we apologize for this but we are working hard to resolve this as soon as possible.
Support is handled on the Forums not in Emails and PMs.
Before you ask a question use the Search function to see it has been answered before.
To do an Advanced Search click the magnifying glass in the Search Box.
To upload pictures click the Upload attachment link below the BLUE SUBMIT BUTTON.

beambarossa
Member
 
Posts: 17
Joined: Wed Jun 21, 2017 12:11 am
Has thanked: 0 time
Been thanked: 0 time

Re: WS-12-250-DC random reboot!!

Tue Apr 10, 2018 5:32 am

Hi,

We run dozens of 12-250DC in our network all in same configuration. We have had two reboot with no recordable reason now to date, one at 2am in the morning and one during prime time 9pm. One had 1.4.6 and one 1.4.7. Have not done it since.

FYI - not that it really helps but happy to provide more info if needed,

Sam

User avatar
Banana Jack
Member
 
Posts: 62
Joined: Fri Sep 04, 2015 4:59 pm
Location: North Yorkshire, UK
Has thanked: 49 times
Been thanked: 13 times

Re: WS-12-250-DC random reboot!!

Tue Oct 09, 2018 4:13 pm

I just wondered whether there has been any progress on eliminating these 'random reboots' as we are severely suffering from it on our network at the moment. We have about 70 Netonix switches and they are all rebooting seemingly at random, causing their connected radios to be power cycled. Sometimes we can go 3-4 days with no problems, and other days we can see many tens of switches rebooting. It's turning into a disaster zone. On our remote syslog server I am seeing 'Detected cold (watchdog) boot'. Nothing is syslogged before the reboot occurs.

Last month we were running 1.4.9 on all switches apart from 2-3 units where we were trialling 1.5.0. After the first major 'reboot storm' (described in this thread) I upgraded all switches to 1.5.0 to try to prevent recurrence but the reboots seem to have got more frequent if anything, although that might be a co-incidence. We have around 7 Mikrotik switches which occasionally log packet loops ("etherX: bridge port received packet with own address as source address, probably loop") but I'm not sure whether this is an 'erroneous error' or if there is a genuine intermittent loop / storm and whether it's possibly causing the Netonix reboots.

Should I try enabling Loop Protection on all our Netonix Switches? What about the other Storm Control settings i.e. Broadcast / Multicast / Unicast / Pause Frame? Is there any downside or risk to enabling these settings?

Thanks
Glenn

User avatar
sirhc
Employee
Employee
 
Posts: 7416
Joined: Tue Apr 08, 2014 3:48 pm
Location: Lancaster, PA
Has thanked: 1608 times
Been thanked: 1325 times

Re: WS-12-250-DC random reboot!!

Wed Oct 10, 2018 7:36 pm

I don't have any of my WS-12-250-DC "randomly" rebooting?

Are these units hooked directly to batteries or to a power supply/charger?
I have my units hooked directly to the batteries.

Are you running v1.5.0 or v1.5.1rc7?
I am running v1.5.0 or v1.5.1rc7.
Support is handled on the Forums not in Emails and PMs.
Before you ask a question use the Search function to see it has been answered before.
To do an Advanced Search click the magnifying glass in the Search Box.
To upload pictures click the Upload attachment link below the BLUE SUBMIT BUTTON.

User avatar
Banana Jack
Member
 
Posts: 62
Joined: Fri Sep 04, 2015 4:59 pm
Location: North Yorkshire, UK
Has thanked: 49 times
Been thanked: 13 times

Re: WS-12-250-DC random reboot!!

Thu Oct 11, 2018 10:25 am

Thanks sirhc - at the moment I'm suspecting that a network loop and/or bad plug/cable/radio is to blame.

In answer to your questions: some units are hooked directly to 12V batteries and then we have smart chargers also connected to the batteries to keep them trickle charged. However, the issue is affecting both battery and grid-powered switches.

We are running v1.5.0. I'm undecided whether to stick with 1.5.0 or downgrade back to 1.4.9 or upgrade to 1.5.1rc7.

Today I've been enabling loop protection on selected switches and it seems to be pointing me toward a certain part of the network based on the ports on which loops are being detected. So maybe I'm making some progress.

Thanks
Glenn

User avatar
kdsnetworks
Member
 
Posts: 13
Joined: Fri Sep 25, 2015 10:58 pm
Location: Great Falls, MT
Has thanked: 0 time
Been thanked: 0 time

Re: WS-12-250-DC random reboot!!

Sat Oct 13, 2018 10:17 pm

I also started to experience random reboots on WS-12-250-DC. 1.5.0

It happen right after I added two additional devices and maxed out this switch ... 11 ports are 24v and one port is 48HV. I started to think that maybe there is too much power draw on the 24v bus. How can we find the real cause of the reboot ?

User avatar
Banana Jack
Member
 
Posts: 62
Joined: Fri Sep 04, 2015 4:59 pm
Location: North Yorkshire, UK
Has thanked: 49 times
Been thanked: 13 times

Re: WS-12-250-DC random reboot!!

Sun Oct 14, 2018 4:01 am

kdsnetworks wrote:I also started to experience random reboots on WS-12-250-DC. 1.5.0

It happen right after I added two additional devices and maxed out this switch ... 11 ports are 24v and one port is 48HV. I started to think that maybe there is too much power draw on the 24v bus. How can we find the real cause of the reboot ?


I'm still investigating my own problems with this but my opinion is that Netonix switches can freeze up and then watchdog-reboot when they're saturated with broadcast traffic, possibly caused by a packet storm or network loop. Try enabling loop protection and see if your switch detects a loop (it will then disable the port for 3 minutes). See if it's always the same port on which the 'loop' (more accurately 'storm', in my opinion), is detected.

Maybe you could try downgrading to 1.4.9 to see if it helps? My own problems started/got worse when I upgraded to 1.5.0 but this may have been a co-incidence. Nonetheless I'm thinking about trying a downgrade myself but I have 70+ switches so it's not an easy decision.

Glenn

*** EDIT: In my case it seems to have been a faulty SFP module to blame (see next post).
Last edited by Banana Jack on Sun Oct 14, 2018 8:11 am, edited 1 time in total.

Next
Return to Hardware and software issues

Who is online

Users browsing this forum: No registered users and 88 guests