Netonix Forums

Hi,

Today I noticed on our graphs that there was a brief drop in traffic and ping loss. As I investigated I noticed it was caused by the netonix and the log looks like this:


Jan 1 00:00:07 netonix: 1.3.9 on WS-8-250-AC
Jan 1 00:00:11 system: Setting MAC address from flash configuration: EC:13:B3:41:09:68
Jan 1 00:00:11 kernel: link state changed to 'up' on port 2
Jan 1 00:00:11 kernel: link state changed to 'up' on port 4
Jan 1 00:00:12 kernel: link state changed to 'up' on port 5
Jan 1 00:00:12 kernel: link state changed to 'up' on port 6
Jan 1 00:00:12 kernel: link state changed to 'up' on port 7
Jan 1 00:00:12 kernel: link state changed to 'up' on port 8
Jan 1 00:00:14 admin: adding lan (eth0) to firewall zone lan
Dec 31 19:00:20 admin: removing lan (eth0) from firewall zone lan
Dec 31 19:00:23 admin: adding lan (eth0) to firewall zone lan
Dec 31 19:00:25 kernel: link state changed to 'down' on port 4
Dec 31 19:00:25 kernel: link state changed to 'down' on port 5
Dec 31 19:00:25 kernel: link state changed to 'down' on port 6
Dec 31 19:00:25 kernel: link state changed to 'down' on port 7
Dec 31 19:00:25 kernel: link state changed to 'down' on port 8
Dec 31 19:00:27 kernel: link state changed to 'up' on port 2
Dec 31 19:00:27 kernel: link state changed to 'up' on port 4
Dec 31 19:00:27 kernel: link state changed to 'up' on port 6
Dec 31 19:00:27 kernel: link state changed to 'up' on port 7
Dec 31 19:00:27 kernel: link state changed to 'up' on port 5
Dec 31 19:00:27 kernel: link state changed to 'up' on port 8
Dec 31 19:00:28 admin: adding lan (eth0) to firewall zone lan
Dec 31 19:00:30 system: starting ntpclient
Jan 30 13:31:02 dropbear[1351]: Running in background
Jan 30 13:31:08 switch[1374]: Detected warm boot
Jan 30 13:31:11 switch[1365]: Port 5: PoE is enabled and port has link, power should not be 0
Jan 30 13:31:11 switch[1365]: Port 6: PoE is enabled and port has link, power should not be 0
Jan 30 13:31:11 switch[1365]: Port 7: PoE is enabled and port has link, power should not be 0
Feb 4 00:22:07 kernel: link state changed to 'down' on port 6
Feb 4 00:22:10 kernel: link state changed to 'up' on port 6
Feb 4 00:22:16 switch[1365]: Port 6: PoE is enabled and port has link, power should not be 0
Feb 4 00:22:30 kernel: link state changed to 'down' on port 6
Feb 4 00:22:32 kernel: link state changed to 'up' on port 6
Feb 4 00:22:38 switch[1365]: Port 6: PoE is enabled and port has link, power should not be 0
Feb 8 09:25:55 kernel: link state changed to 'down' on port 4
Feb 8 09:25:58 kernel: link state changed to 'up' on port 4
Feb 8 09:26:18 kernel: link state changed to 'down' on port 4
Feb 8 09:26:21 kernel: link state changed to 'up' on port 4
Feb 12 06:58:16 kernel: link state changed to 'down' on port 5
Feb 12 06:58:19 kernel: link state changed to 'up' on port 5
Feb 12 06:58:25 switch[1365]: Port 5: PoE is enabled and port has link, power should not be 0
Feb 12 06:58:39 kernel: link state changed to 'down' on port 5
Feb 12 06:58:42 kernel: link state changed to 'up' on port 5
Feb 12 06:58:47 switch[1365]: Port 5: PoE is enabled and port has link, power should not be 0
Feb 28 17:02:13 kernel: link state changed to 'down' on port 4
Feb 28 17:02:15 kernel: link state changed to 'up' on port 4
Feb 28 17:02:36 kernel: link state changed to 'down' on port 4
Feb 28 17:02:38 kernel: link state changed to 'up' on port 4
Mar 1 07:43:03 kernel: link state changed to 'down' on port 4
Mar 1 07:43:05 kernel: link state changed to 'up' on port 4
Mar 1 07:43:27 kernel: link state changed to 'down' on port 4
Mar 1 07:43:29 kernel: link state changed to 'up' on port 4
Mar 1 07:59:25 kernel: link state changed to 'down' on port 4
Mar 1 07:59:27 kernel: link state changed to 'up' on port 4
Mar 1 07:59:48 kernel: link state changed to 'down' on port 4
Mar 1 07:59:50 kernel: link state changed to 'up' on port 4
Mar 2 23:00:48 kernel: link state changed to 'down' on port 6
Mar 2 23:00:50 kernel: link state changed to 'up' on port 6
Mar 2 23:00:56 switch[1365]: Port 6: PoE is enabled and port has link, power should not be 0
Mar 2 23:01:10 kernel: link state changed to 'down' on port 6
Mar 2 23:01:13 kernel: link state changed to 'up' on port 6
Mar 2 23:01:18 switch[1365]: Port 6: PoE is enabled and port has link, power should not be 0
Mar 6 08:57:18 kernel: link state changed to 'down' on port 4
Mar 6 08:57:21 kernel: link state changed to 'up' on port 4
Mar 6 08:57:41 kernel: link state changed to 'down' on port 4
Mar 6 08:57:43 kernel: link state changed to 'up' on port 4

The GUI still shows that on port 5 6 and 7 that they are powered on with 0 Watts and have 1G Link. Port 2 and 4 show 20 Watt though they are just a powerbeam and nanobeam and normally much lower wattage. The uptime on the switch is 40 Days and has two business clients on it that we can't afford go down.

Any recommendations?

Let me see if I follow you?

You have ports that are powering devices but the status page shows 0 watts?

Normally this is caused by the current sensors being damaged. Damaged current sensors are most often damaged from 1 of 3 things listed below and when damaged can cause erroneous readings:

1) Ground potential shift which is from the tower ground not being bonded to the service ground in which case the Ethernet cable is the bond and when a ground potential shift occurs it exposes the current sensors to more than 80V which damages them.

2) ESD discharge. There was an ESD discharge at your antenna and there is no dedicated ground path to take that has a shorter less resistive path to take to Earth Ground than the Ethernet cable exposing the current sensor to more than 80V.

3) Static discharge. There was a static charge accumulated on the antenna/radio and there is no dedicated ground path to take that has a shorter less resistive path to take to Earth Ground than the Ethernet cable exposing the current sensor to more than 80V. If you're experiencing frequent winds at your tower location and the humidity is right it will constantly add a static charge to your Ethernet cable which will cause Ethernet errors and over time will cause further damage to the port and or switch until the port or switch fails.

2 great posts on grounding are here:
viewtopic.php?f=30&t=188
viewtopic.php?f=30&t=1429

If am am totally wrong with what your describing please describe the issue again.

Hi Sirhc,

You are correct with my description, If the current sensors are blown are they easily repairable (my gut says no but i have to ask)

The fact that it appears the current sensors are damaged and thus it is possible the radio Ethernet port is also damaged?

I would take the switch out of service and bench test it.

Follow the testing method outlined in post # 2 of the RMA thread: viewtopic.php?f=6&t=1259#p9359

I would also test the radios.

No, they are not easily repaired, sorry.

I would be more concerned as to why the current sensors are damaged.

I would read those 2 grounding posts I linked because if you have poor grounding replacing the switch and or radios will only mean it will happen again.

We are currently testing some new modifications to the switches to better protect those current sensors but we fear this is only masking a problem that will eventually cause more damage because if the current sensors are taking damage with the current design it means they are being exposed to greater than 80V and if we mask that problem it would be like treating cancer with pain meds, sure the patient feels better "for awhile" but they will die without treating/curing the disease.

I just realized this wasn't the cause of the ping/traffic loss, The powerbeams had breifly disconnected from eachother. But luckily led me to see this, Just so I can give you a idea of our set up we are on a roof top (not the highest point of the building) with direct power running to the 8 port netonix. Our antennas are fairly sheltered from most parts of the building, Our longest outdoor cable is maybe 10-15 meters at most and the antennas are located on a tripod on the roof. All of the other sensors such as the power on device status page all seem to be functioning.

Try power cycling the switch and if that does not fix your current reported issue please post up a screen grab of the Status and Device/Status Tab.

I would make sure you have a ground wire run from a ground bus bar in your box to a small ground bus at each antenna mast, #6 Green should be enough for that short of run, then from that ground bar at the mast run a #6 to each antenna mount hardware. In your box I would make sure you run a 12AWG ground wire from the switch chassis ground screw to the small ground bus in the box.

Then I would add an extra 20+ feet of Ethernet cable as a service loop at each radio to make sure the Ethernet path is a longer run and higher resistance than the intended ground wire path.

Next I would make sure that the service ground for this building is in good working condition and corrosion free. Maybe add an additional ground rod 10 feet away if there is only 1 and bond them together.

Also are you using ToughCABLE and either ToughCABLE ESD RJ45 ends or RF Armor or Netonix ESD ends?