WS-10-250-AC randomly rebooting and cutting poe power

[compguy892]

For the past week I have had a WS-10-250-AC running 1.4.7 supplying poe power to two mimosa b5c's completely fall off the face of the earth for a short bit then come back with logs stating it just powered up and normal poe tests/power on. It only seems to happen later in the day between 5-10 pm during higher load times, seems like its between every hour to 2 hours it will do it. Then randomly its just fine. Unit has been in production since 5/2017. Any ideas so I can start to narrow down the possible issue here? One other post was related to a possible bad 24V power circuit (only 24 volt devices were losing power)
and could be related to temp increase or decrease so maybe that's what is going on here?





Uptime
00:26:26


CPU Usage
13%


Memory Usage
44.80MB






Firmware Version
1.4.7


Board Rev
E



Power



Board 48V
49.8V


Board 24V
24.7V


Board 3V
3.33V



Thermal



Board Temp
40C


CPU Temp
66C


PHY Temp
69C






Fan 1 Speed
2430RPM




Logs:


Dec 31 19:00:06 netonix: 1.4.7 on WS-10-250-AC
Dec 31 19:00:09 system: Setting MAC address from flash configuration: EC:13:B2:C1:1C:CA
Dec 31 19:00:12 admin: adding lan (eth0.1001) to firewall zone lan
Dec 31 19:00:19 Port: link state changed to 'up' (1G) on port 6
Dec 31 19:00:19 Port: link state changed to 'up' (1G) on port 7
Dec 31 19:00:23 dropbear[753]: Running in background
Dec 31 19:00:25 switch[779]: Detected cold boot
Dec 31 19:00:26 Port: link state changed to 'down' on port 1
Dec 31 19:00:26 switch[779]: PoE enabled on port 1, PoE Smart is starting cable check
Dec 31 19:00:26 switch[779]: PoE enabled on port 2, PoE Smart is starting cable check
Dec 31 19:00:31 switch[779]: Port 2 cable check results: Open, Open, Open, Open
Dec 31 19:00:31 switch[779]: Port 2 passed PoE Smart cable check, turning on power
Dec 31 19:00:32 switch[779]: Port 1 cable check results: Ok, Ok, Ok, Ok
Dec 31 19:00:32 switch[779]: Port 1 passed PoE Smart cable check, turning on power
Dec 31 19:00:35 Port: link state changed to 'up' (1G) on port 1
Dec 31 19:00:38 Port: link state changed to 'up' (1G) on port 2
Dec 31 19:01:29 Port: link state changed to 'down' on port 1
Dec 31 19:01:31 Port: link state changed to 'down' on port 2
Dec 31 19:01:31 Port: link state changed to 'up' (1G) on port 1
Dec 31 19:01:33 Port: link state changed to 'up' (1G) on port 2
Dec 31 19:01:44 Port: link state changed to 'down' on port 1
Dec 31 19:01:47 Port: link state changed to 'down' on port 2
Dec 31 19:01:47 Port: link state changed to 'up' (1G) on port 1
Dec 31 19:01:49 Port: link state changed to 'up' (1G) on port 2

[mike99]

What else is connected to this switch ? Are those the only 2 rebooting ? A screen shot of ports page with voltage would help indicating device connected would help.

[Julian]

You have my permission to go ahead and open the unit to get the date code off of the power supply - I'm thinking this could be a holdout from that batch of power supplies we got last year with the stuffing issue at C51.

[bajaconnect]

Did you find the issue with this as i have a 8 port AC that just did the same thing. its only done it once but this is a main link for us and want to prevent it happening again. it also was at 5PM, sorry about the date on the log as its wrong but the time is correct.

Dec 31 17:00:06 netonix: 1.4.7 on WS-8-150-AC
Dec 31 17:00:10 system: Setting MAC address from flash configuration: EC:13:B2:D1:39:A8
Dec 31 17:00:12 system: starting ntpclient
Dec 31 17:00:13 LGeorge73: adding lan (eth0.5) to firewall zone lan
Dec 31 17:00:18 Port: link state changed to 'up' (1G) on port 1
Dec 31 17:00:18 Port: link state changed to 'up' (1G) on port 3
Dec 31 17:00:19 Port: link state changed to 'up' (1G) on port 5
Dec 31 17:00:19 Port: link state changed to 'up' (1G) on port 7
Dec 31 17:00:19 Port: link state changed to 'up' (1G) on port 8
Dec 31 17:00:20 Port: link state changed to 'up' (1G) on port 2
Dec 31 17:00:22 dropbear[748]: Running in background
Dec 31 17:00:24 switch[780]: Detected cold boot
Dec 31 17:00:25 Port: link state changed to 'down' on port 4
Dec 31 17:00:25 Port: link state changed to 'down' on port 5
Dec 31 17:00:25 Port: link state changed to 'down' on port 1
Dec 31 17:00:25 Port: link state changed to 'down' on port 2
Dec 31 17:00:25 Port: link state changed to 'down' on port 3
Dec 31 17:00:25 switch[780]: PoE enabled on port 1, PoE Smart is starting cable check
Dec 31 17:00:25 switch[780]: PoE enabled on port 2, PoE Smart is starting cable check
Dec 31 17:00:26 switch[780]: PoE enabled on port 3, PoE Smart is starting cable check
Dec 31 17:00:26 switch[780]: PoE enabled on port 4, PoE Smart is starting cable check
Dec 31 17:00:26 switch[780]: PoE enabled on port 5, PoE Smart is starting cable check

[quantumx]

Does the WS-12-250-AC have the same possible PSU problem? We had the exact same thing happen (Detected cold boot) today.

Device Details:

WS-12-250-AC
Board Rev: D
FW: 1.4.8rc10
MAC: ec:13:b3:61:0f:53
Average load ~100W

Clean power - other onsite WS-24-400A unaffected

Log:

Dec 31 19:00:06 netonix: 1.4.8rc10 on WS-12-250-AC

Dec 31 19:00:10 system: Setting MAC address from flash configuration: EC:13:B3:61:0F:5E

Dec 31 19:00:12 system: starting ntpclient

Dec 31 19:00:13 admin: adding lan (eth0.130) to firewall zone lan

Oct 4 14:30:55 dropbear[760]: Running in background

Oct 4 14:30:57 Port: link state changed to 'down' on port 6

Oct 4 14:30:58 switch[788]: Detected cold boot

Oct 4 14:30:59 Port: link state changed to 'down' on port 7

Oct 4 14:30:59 Port: link state changed to 'down' on port 8

Oct 4 14:30:59 Port: link state changed to 'down' on port 12

Oct 4 14:30:59 Port: link state changed to 'down' on port 1

Oct 4 14:31:00 switch[788]: PoE enabled on port 1, PoE Smart is starting cable check

Oct 4 14:31:00 switch[788]: PoE enabled on port 2, PoE Smart is starting cable check

Oct 4 14:31:00 switch[788]: PoE enabled on port 3, PoE Smart is starting cable check

Oct 4 14:31:00 switch[788]: PoE enabled on port 4, PoE Smart is starting cable check

Oct 4 14:31:00 switch[788]: PoE enabled on port 5, PoE Smart is starting cable check

Oct 4 14:31:00 switch[788]: PoE enabled on port 6, PoE Smart is starting cable check

Oct 4 14:31:04 switch[788]: Port 1 cable check results: Open, Open, Open, Open

Oct 4 14:31:04 switch[788]: Port 1 passed PoE Smart cable check, turning on power

Oct 4 14:31:04 switch[788]: Port 2 cable check results: Open, Open, Open, Open

Oct 4 14:31:04 switch[788]: Port 2 passed PoE Smart cable check, turning on power

Oct 4 14:31:04 switch[788]: Port 3 cable check results: Open, Open, Open, Open

Oct 4 14:31:04 switch[788]: Port 3 passed PoE Smart cable check, turning on power

Oct 4 14:31:05 switch[788]: Port 4 cable check results: Open, Open, Open, Open

Oct 4 14:31:05 switch[788]: Port 4 passed PoE Smart cable check, turning on power

Oct 4 14:31:05 switch[788]: Port 5 cable check results: Open, Open, Open, Open

Oct 4 14:31:05 switch[788]: Port 5 passed PoE Smart cable check, turning on power

Oct 4 14:31:06 switch[788]: Port 6 cable check results: Ok, Ok, Short, Short

Oct 4 14:31:06 switch[788]: Port 6 passed PoE Smart cable check, turning on power

Oct 4 14:31:07 Port: link state changed to 'up' (10M-H) on port 4

Oct 4 14:31:07 Port: link state changed to 'up' (100M-F) on port 7

Oct 4 14:31:07 Port: link state changed to 'up' (100M-F) on port 8

Oct 4 14:31:09 Port: link state changed to 'up' (10M-F) on port 6

Oct 4 14:31:10 Port: link state changed to 'up' (1G) on port 12

Oct 4 14:31:10 Port: link state changed to 'up' (1G) on port 1

Oct 4 14:31:26 Port: link state changed to 'down' on port 12

Oct 4 14:31:26 Port: link state changed to 'down' on port 1

Oct 4 14:31:29 Port: link state changed to 'up' (1G) on port 12

Oct 4 14:31:29 Port: link state changed to 'up' (1G) on port 1

Oct 4 14:31:36 Port: link state changed to 'down' on port 6

Oct 4 14:31:37 Port: link state changed to 'up' (100M-F) on port 6

Oct 4 14:31:40 Port: link state changed to 'down' on port 6

Oct 4 14:31:41 Port: link state changed to 'up' (100M-F) on port 6

Oct 4 14:31:45 Port: link state changed to 'down' on port 4

Oct 4 14:31:48 Port: link state changed to 'up' (10M-H) on port 4

Oct 4 14:32:20 Port: link state changed to 'up' (1G) on port 2

Oct 4 14:32:28 Port: link state changed to 'down' on port 4

Oct 4 14:32:30 Port: link state changed to 'up' (1G) on port 3

Oct 4 14:32:30 Port: link state changed to 'up' (1G) on port 5

Oct 4 14:32:30 Port: link state changed to 'up' (1G) on port 4

Oct 4 14:32:36 Port: link state changed to 'down' on port 7

Oct 4 14:32:38 Port: link state changed to 'up' (100M-F) on port 7

Oct 4 14:32:43 Port: link state changed to 'down' on port 2

Oct 4 14:32:45 Port: link state changed to 'down' on port 8

Oct 4 14:32:45 Port: link state changed to 'up' (1G) on port 2

Oct 4 14:32:47 Port: link state changed to 'up' (100M-F) on port 8

Oct 4 14:32:50 Port: link state changed to 'down' on port 3

Oct 4 14:32:50 Port: link state changed to 'down' on port 5

Oct 4 14:32:51 Port: link state changed to 'down' on port 4

Oct 4 14:32:51 Port: link state changed to 'up' (1G) on port 3

Oct 4 14:32:52 Port: link state changed to 'up' (1G) on port 5

Oct 4 14:32:53 Port: link state changed to 'up' (1G) on port 4

[Kingsley]

What about the WS-26-500-DC? I have a site that had a WS-12-250-DC for 6+ months run perfectly fine. Switch was upgraded recently for more ports to the 26 port switch on 1.4.7 and now every few days the switch reboots. Does exactly as explained above. The site has external battery monitoring and a massive battery bank at 12v so it is NOT a power issue.

On the other hand I have another WS-26-500-DC on a different site that is running solid and running a whole lot more equipment than the switch that is rebooting. This first switch was bought around the start of the year how ever this second switch that is rebooting is only a couple months old.

Any ideas? Can I roll the firmware back to something else?

[sirhc]

What about the WS-26-500-DC? I have a site that had a WS-12-250-DC for 6+ months run perfectly fine. Switch was upgraded recently for more ports to the 26 port switch on 1.4.7 and now every few days the switch reboots. Does exactly as explained above. The site has external battery monitoring and a massive battery bank at 12v so it is NOT a power issue.

On the other hand I have another WS-26-500-DC on a different site that is running solid and running a whole lot more equipment than the switch that is rebooting. This first switch was bought around the start of the year how ever this second switch that is rebooting is only a couple months old.

Any ideas? Can I roll the firmware back to something else?

You should be running v1.4.8rc11 or the latest code as there are IMPORTANT fixes for DC switches - read release notes.

You're saying your powering the WS-26 with 12V DC?

PLEASE upgrade your DC switches to v1.4.8rcX, in fact unless you want to run known bugs I would upgrade all switches to the latest version.

"RC" does not mean release candidate to us.

The day we released v1.4.7 as soon as someone reported a bug a fix was made and v1.4.8rc1 was born.
v1.4.8rcX has all the known bugs reported and fixed since v1.4.7 was released.

After upgrading your switches please detail the power source:
How many batteries
How are they interconnected (series or parallel)
What is charging them
Does the switch connect DIRECTLY to the battery bank or is there something in between?

Post up screen grabs of the following TABs:
Status
Ports
Power
Device/Status