8port DC issue

DOWNLOAD THE LATEST FIRMWARE HERE
User avatar
LRL
Experienced Member
 
Posts: 238
Joined: Sun Nov 23, 2014 4:00 am
Location: Rock Springs, WY
Has thanked: 18 times
Been thanked: 49 times

8port DC issue

Mon Sep 07, 2015 8:47 pm

I've got two 8 port DC switches deployed and two 12 port DC switches deployed and until now I haven't had any issues.

Over the weekend I received three e-mails from one of our 8 port switches saying "Disabling PoE power on port 5 due to Power Supply Thermal out of specification (controller is 1610612736 C) - from JT Switch (10.247.28.5)". All three e-mail were received within 2 minutes.

right after I received this e-mail I logged into the switch and here are the temp readings from the status page:


Board Temp 41C /106F

CPU Temp55C /131F

PHY Temp57C /135F

Power Supply Board Temp35C /95F

Power Supply Heatsink #1 Temp28C /82F

Power Supply Heatsink #2 Temp28C /82F

Power Supply Controller Temp50C /122F


Our environmental monitor showed the box to be at 79f and our average power usage is 21w. Thoughts on this?
Last edited by LRL on Mon Sep 07, 2015 9:01 pm, edited 1 time in total.
-LRL

"My reading of history convinces me that most bad government results from too much government." - Thomas Jefferson

User avatar
LRL
Experienced Member
 
Posts: 238
Joined: Sun Nov 23, 2014 4:00 am
Location: Rock Springs, WY
Has thanked: 18 times
Been thanked: 49 times

Re: 8port DC issue

Mon Sep 07, 2015 8:59 pm

I forgot, port 5 is a camera and has PoE priority set to 1 so it would naturally be the first port to shutdown. The device was indeed powered off and recovered pretty quick. Below is the log.

Code: Select all
 Sep 5 16:07:37 switch[782]: Disabling PoE power on port 5 due to Power Supply Thermal out of specification (controller is 1610612736 C)
Sep 5 16:07:38 sSMTP[1563]: Sent mail for *** outbytes=590
Sep 5 16:09:16 switch[782]: Disabling PoE power on port 5 due to Power Supply Thermal out of specification (controller is 1073741824 C)
Sep 5 16:09:18 sSMTP[1749]: Sent mail for *** outbytes=590
Sep 5 16:09:54 switch[782]: Disabling PoE power on port 5 due to Power Supply Thermal out of specification (controller is -1073741824 C)
Sep 5 16:09:55 sSMTP[1831]: Sent mail for*** outbytes=592 
-LRL

"My reading of history convinces me that most bad government results from too much government." - Thomas Jefferson

User avatar
sirhc
Employee
Employee
 
Posts: 7415
Joined: Tue Apr 08, 2014 3:48 pm
Location: Lancaster, PA
Has thanked: 1608 times
Been thanked: 1325 times

Re: 8port DC issue

Mon Sep 07, 2015 9:47 pm

lrl wrote: The device was indeed powered off and recovered pretty quick.


Does this mean it recovered on it's own?

Sep 5 16:07:37 switch[782]: Disabling PoE power on port 5 due to Power Supply Thermal out of specification (controller is 1610612736 C)
Sep 5 16:07:38 sSMTP[1563]: Sent mail for *** outbytes=590
Sep 5 16:09:16 switch[782]: Disabling PoE power on port 5 due to Power Supply Thermal out of specification (controller is 1073741824 C)
Sep 5 16:09:18 sSMTP[1749]: Sent mail for *** outbytes=590
Sep 5 16:09:54 switch[782]: Disabling PoE power on port 5 due to Power Supply Thermal out of specification (controller is -1073741824 C)
Sep 5 16:09:55 sSMTP[1831]: Sent mail for*** outbytes=592


Obviously those are bogus readings in the log because at the 2 first temperatures nuclear fusion would occur at extremely low gravimetric pressure, and at the 3rd temperature all molecular motion would stop, the question is why?

What is the DC input voltage from the batteries?
Support is handled on the Forums not in Emails and PMs.
Before you ask a question use the Search function to see it has been answered before.
To do an Advanced Search click the magnifying glass in the Search Box.
To upload pictures click the Upload attachment link below the BLUE SUBMIT BUTTON.

User avatar
LRL
Experienced Member
 
Posts: 238
Joined: Sun Nov 23, 2014 4:00 am
Location: Rock Springs, WY
Has thanked: 18 times
Been thanked: 49 times

Re: 8port DC issue

Mon Sep 07, 2015 9:55 pm

It did indeed recover on it's own. Shortly after these I had a rocket go down and ping watchdog reset it.

Operating temps are up to 55c or 131f, correct? I know I seen a post where you outlined what happen at what temps but I couldn't find it again.
-LRL

"My reading of history convinces me that most bad government results from too much government." - Thomas Jefferson

User avatar
sirhc
Employee
Employee
 
Posts: 7415
Joined: Tue Apr 08, 2014 3:48 pm
Location: Lancaster, PA
Has thanked: 1608 times
Been thanked: 1325 times

Re: 8port DC issue

Mon Sep 07, 2015 10:02 pm

LRL wrote:It did indeed recover on it's own. Shortly after these I had a rocket go down and ping watchdog reset it.

Operating temps are up to 55c or 131f, correct? I know I seen a post where you outlined what happen at what temps but I couldn't find it again.


The Rocket going down and recovered by ping watch dog probably is not related to this at all unless you see something in the log that ties them together?

The temperatures are all fine except those bogus values reported in the log which range from negative BILLION to positive BILLION.
Support is handled on the Forums not in Emails and PMs.
Before you ask a question use the Search function to see it has been answered before.
To do an Advanced Search click the magnifying glass in the Search Box.
To upload pictures click the Upload attachment link below the BLUE SUBMIT BUTTON.

User avatar
LRL
Experienced Member
 
Posts: 238
Joined: Sun Nov 23, 2014 4:00 am
Location: Rock Springs, WY
Has thanked: 18 times
Been thanked: 49 times

Re: 8port DC issue

Mon Sep 07, 2015 10:13 pm

Nothing really, just shortly after those.

Code: Select all
 Sep 5 16:13:36 pinger: Failure pinging 10.246.204.226, powering off port 6 for 5 seconds
Sep 5 16:13:37 sSMTP[2397]: Sent mail for *** outbytes=515


The switches uptime wasn't affected nor was the backhaul. :???:

We had 0 errors on all the interfaces the day prior and after the even we errors on all interfaces.

JT status.jpg
JT status.jpg (174.99 KiB) Viewed 6129 times

JT Power.jpg
JT Power.jpg (144.29 KiB) Viewed 6129 times
-LRL

"My reading of history convinces me that most bad government results from too much government." - Thomas Jefferson

User avatar
sirhc
Employee
Employee
 
Posts: 7415
Joined: Tue Apr 08, 2014 3:48 pm
Location: Lancaster, PA
Has thanked: 1608 times
Been thanked: 1325 times

Re: 8port DC issue

Tue Sep 08, 2015 8:03 am

Hi LRL,

We are looking into this.

We believe those bogus temp readings are a product of an I2C bus error.

We have Eric working on code to compensate for erroneous readings like this and will release a firmware patch soon.
Support is handled on the Forums not in Emails and PMs.
Before you ask a question use the Search function to see it has been answered before.
To do an Advanced Search click the magnifying glass in the Search Box.
To upload pictures click the Upload attachment link below the BLUE SUBMIT BUTTON.

User avatar
sirhc
Employee
Employee
 
Posts: 7415
Joined: Tue Apr 08, 2014 3:48 pm
Location: Lancaster, PA
Has thanked: 1608 times
Been thanked: 1325 times

Re: 8port DC issue

Tue Sep 08, 2015 4:27 pm

Sep 5 16:07:37 switch[782]: Disabling PoE power on port 5 due to Power Supply Thermal out of specification (controller is 1610612736 C)
Sep 5 16:07:38 sSMTP[1563]: Sent mail for *** outbytes=590
Sep 5 16:09:16 switch[782]: Disabling PoE power on port 5 due to Power Supply Thermal out of specification (controller is 1073741824 C)
Sep 5 16:09:18 sSMTP[1749]: Sent mail for *** outbytes=590
Sep 5 16:09:54 switch[782]: Disabling PoE power on port 5 due to Power Supply Thermal out of specification (controller is -1073741824 C)
Sep 5 16:09:55 sSMTP[1831]: Sent mail for*** outbytes=592


HEY - LRL

Was this the complete log or was this edited?

What I mean is there is 5 entries showing that the same port was disabled 3 times which there should have been entries in between showing the POE was restored before it could disable it again or it should have moved on to the next port.

Eric said he found where he was formating the the number wrong in the log but as I said something seems wrong unless this was an edited log just showing the POE disables?

Also your other readings which were taken right after the event show the Controller temp at a more reasonable range of 50C, so the reading must have been bogus as there is no way the Controller would heat up above 110C to trigger this event then cool back down to 50C so quickly? I am assuming by what you said these events all occurred within a 10 minute span to where it corrected itself and restored POE power?
Support is handled on the Forums not in Emails and PMs.
Before you ask a question use the Search function to see it has been answered before.
To do an Advanced Search click the magnifying glass in the Search Box.
To upload pictures click the Upload attachment link below the BLUE SUBMIT BUTTON.

User avatar
LRL
Experienced Member
 
Posts: 238
Joined: Sun Nov 23, 2014 4:00 am
Location: Rock Springs, WY
Has thanked: 18 times
Been thanked: 49 times

Re: 8port DC issue

Thu Sep 10, 2015 5:24 pm

The log was for "switch" only and was unedited aside from the e-mail address. No clears or anything left out.

Here is the log "all" only e-mail editied

Code: Select all
 Jan 3 06:42:57 system: starting ntpclient
Sep 5 02:06:49 pinger: starting up with 2 ping watchdogs and 0 scheduled port bounces
Sep 5 16:06:48 kernel: link state changed to 'down' on port 6
Sep 5 16:06:50 kernel: link state changed to 'up' on port 6
Sep 5 16:07:37 switch[782]: Disabling PoE power on port 5 due to Power Supply Thermal out of specification (controller is 1610612736 C)
Sep 5 16:07:38 sSMTP[1563]: Sent mail for *** (221 2.0.0 **** closing connection) uid=0 username=* outbytes=590
Sep 5 16:07:38 kernel: link state changed to 'down' on port 5
Sep 5 16:07:42 kernel: link state changed to 'up' on port 5
Sep 5 16:07:45 kernel: link state changed to 'down' on port 5
Sep 5 16:07:48 kernel: link state changed to 'up' on port 5
Sep 5 16:07:58 kernel: link state changed to 'down' on port 5
Sep 5 16:08:00 kernel: link state changed to 'up' on port 5
Sep 5 16:09:16 switch[782]: Disabling PoE power on port 5 due to Power Supply Thermal out of specification (controller is 1073741824 C)
Sep 5 16:09:18 sSMTP[1749]: Sent mail for *** (221 2.0.0 **** closing connection) uid=0 username=* outbytes=590
Sep 5 16:09:18 kernel: link state changed to 'down' on port 5
Sep 5 16:09:21 kernel: link state changed to 'up' on port 5
Sep 5 16:09:24 kernel: link state changed to 'down' on port 5
Sep 5 16:09:27 kernel: link state changed to 'up' on port 5
Sep 5 16:09:37 kernel: link state changed to 'down' on port 5
Sep 5 16:09:39 kernel: link state changed to 'up' on port 5
Sep 5 16:09:54 switch[782]: Disabling PoE power on port 5 due to Power Supply Thermal out of specification (controller is -1073741824 C)
Sep 5 16:09:55 sSMTP[1831]: Sent mail for *** (221 2.0.0 **** closing connection) uid=0 username=* outbytes=592
Sep 5 16:09:55 kernel: link state changed to 'down' on port 5
Sep 5 16:09:59 kernel: link state changed to 'up' on port 5
Sep 5 16:10:01 kernel: link state changed to 'down' on port 5
Sep 5 16:10:04 kernel: link state changed to 'up' on port 5
Sep 5 16:10:14 kernel: link state changed to 'down' on port 5
Sep 5 16:10:17 kernel: link state changed to 'up' on port 5
Sep 5 16:13:36 pinger: Failure pinging 10.246.204.226, powering off port 6 for 5 seconds
Sep  5 16:13:37 sSMTP[2397]: Sent mail for *** (221 2.0.0 **** closing connection) uid=0 username=* outbytes=515
Sep  5 16:13:38 kernel: link state changed to 'down' on port 6
Sep  5 16:13:44 kernel: link state changed to 'up' on port 6
Sep  5 16:14:07 kernel: link state changed to 'down' on port 6
Sep  5 16:14:09 kernel: link state changed to 'up' on port 6
Sep  5 16:14:11 kernel: link state changed to 'down' on port 6
Sep  5 16:14:13 kernel: link state changed to 'up' on port 6
-LRL

"My reading of history convinces me that most bad government results from too much government." - Thomas Jefferson

Return to Hardware and software issues

Who is online

Users browsing this forum: Google [Bot] and 47 guests