v1.4.0 Bug Reports and Comments

[AlexC]

Could you change it so the switch does the reboot for you in scheduled port bounce that would then prevent those alerts if Eric did his code right and ignores port speed changes on scheduled port bounces.

I've just stopped those particular devices from rebooting on their own for now as they don't really need it anymore. It was something I added due to a flaky earlier version of their firmware.

[sww]

Upgraded first one last night from 1.3.9 and upon reboot it locked up. My tech said he found it with all LEDs on solid. Odd thing was all the backhauls were still powered up by it, but each was showing no ethernet link. I had him replace it with another one on site and I have this one sitting in on my desk. I'll try to log into it later and see if I can get into it. Do you want any information from it before I do anything? This particular switch had a pre-existing problem where the board was not reading any of the voltages or temperatures so it may be a fluke, but I havn't tried to update any others yet. This is a WS-8-250-DC.

[sirhc]

Upgraded first one last night from 1.3.9 and upon reboot it locked up. My tech said he found it with all LEDs on solid. Odd thing was all the backhauls were still powered up by it, but each was showing no ethernet link. I had him replace it with another one on site and I have this one sitting in on my desk. I'll try to log into it later and see if I can get into it. Do you want any information from it before I do anything? This particular switch had a pre-existing problem where the board was not reading any of the voltages or temperatures so it may be a fluke, but I havn't tried to update any others yet. This is a WS-8-250-DC.

If the switch was not reading voltages and temperatures then the unit was either damaged or located near an FM tower or similar RF signal that was interfering with the I2C communications.

When you set it up on your bench I would test it out. If the temp and voltages are not reading at your office then the unit is damaged most likely from a surge in which case you need to compare your grounding technique to those shown in these posts:

viewtopic.php?f=30&t=188
viewtopic.php?f=30&t=1429

Also as this is a DC switch I sure hope you had the Earth Ground lug hooked up to the Earth Ground as that is the only way the switch can get access to Earth Ground properly other wise it is getting access to Earth Ground from the tower through the Ethernet cables which is bad and will result in damage to the switch.

After you read the grounding posts above you need to ask yourself these questions:
1) Does the tower have multiple ground rods?

2) Is the tower ground bonded to the electric service ground (if AC power is available at this site)
ALL GROUND RODS ON A SITE NEED TO BE BONDED TOGETHER OR YOU HAVE A GROUND POTENTIAL DIFFERENCE WHICH IS BAD, GROUND IS NOT GROUND, EVERY GROUND ROD HAS A DIFFERENT IMPEDANCE TO GROUND OR GROUND POTENTIAL DIFFERENCE.

3) Do I have an insulated dedicated ground wire and bus from the grounding system to each radio/mount

4) Is the Ethernet cable length to the switch at least 10% longer from the radio to the switch to the ground rods so that static discharges will see the intended ground path as shorter and less resistance than going through the Ethernet cable and switch to get to ground?

If you answer no to any of the above questions you will incur damage again and again.

If the I2C bus is damaged or being interfered with then the upgrade process may get messed up, but I would bet more on a damaged unit causing an upgrade error over interference?

You can test the switch on your bench as outlined on Post #2 of the RMA instructions thread:
viewtopic.php?f=6&t=1259

[sww]

Fired it up in my office and it appears fine, the update to 1.4.0 also took. Now the voltage readings are working as well. I'll keep it on a bench for a while and report back if I find any other issues. I'll also keep an eye on the replacement module at the tower site, but so far both appear ok.

[sirhc]

Fired it up in my office and it appears fine, the update to 1.4.0 also took. Now the voltage readings are working as well. I'll keep it on a bench for a while and report back if I find any other issues. I'll also keep an eye on the replacement module at the tower site, but so far both appear ok.

Sometimes (rare) a firmware upgrade goes in such a way that a hard power cycle is needed, I have seen this with not only our stuff but most any other piece of equipment I have used over the years. ehhh it does happen?

[Banana Jack]

http://forum.netonix.com/viewtopic.php?f=17&t=1459&p=12102&hilit=Loop+Protection#p12102
I do not know how you do your network but if you allow customers Layer 2 access to you network if a customer creates a loop inside their network then Loop Protection will see this and take action depending on how they have their router set up as the special packet enters then exits their location and gets pack to the sending switch.

Thanks sirhc, we generally don't allow customers Layer 2 access although some customer host repeaters so in theory they could access a spare port on their switch.

We've spent some time trying to find a network loop today but without success. One thing we have found is that running a UBNT Discovery causes immediate loss of access to the Netonix switches. This is why the problem was occurring hourly - because we had an hourly scan configured in AirControl.

I think I'm going to have to fully downgrade all the Netonix switches so that I can get Loop Detection to be disabled again. I'd be interested to hear if you find whether or not the setting is recognised correctly in 1.4.0 after you get chance to check.

I downloaded Manager - it looks like it's going to be very handy - thank you!

Glenn

[sporkman]

Copying this over from my other thread...

Have a WS-6-Mini that flipped out today while running 1.4.0rc25. Saw uplink port drop a few times, then the switch rebooted. It's now running at 100% CPU with most of the usage from a process called "RX Thread".

There's very little traffic on this thing. The most complex thing happening here is allowing tagged vlan 101 in one port and out another.

Port stats:

Code: Select all

bwayadmin@483_Bway_Netonix_Switch:/www# switch -d
Port 1: link 100M-F, poe Off, power 0.0, tx_pkts 105388, tx_octets 36792207, tx_errors 0, rx_pk
ts 79664, rx_octets 23351507, rx_errors 0
Port 2: link Down, poe Off, power 0.0, tx_pkts 0, tx_octets 0, tx_errors 0, rx_pkts 0, rx_octet
s 0, rx_errors 0
Port 3: link 1G, poe 24V, power 0.0, tx_pkts 68638, tx_octets 20769806, tx_errors 0, rx_pkts 91
912, rx_octets 25981625, rx_errors 0
Port 4: link 100M-F, poe 24V, power 0.0, tx_pkts 7548, tx_octets 1768414, tx_errors 0, rx_pkts
4844, rx_octets 827677, rx_errors 0
Port 5: link Down, poe Off, power 0.0, tx_pkts 0, tx_octets 0, tx_errors 0, rx_pkts 0, rx_octet
s 0, rx_errors 0
Port 6: link Down, poe Off, power 0.0, tx_pkts 0, tx_octets 0, tx_errors 0, rx_pkts 0, rx_octet
s 0, rx_errors 0


status screen

top

[sirhc]

@ Banana Jack

Sadly Glenn you found a BUG v1.4.0 indeed does not disable Loop Protection when you disable it in the UI.

I will post v1.4.1rc1 later today with a FIX

[Banana Jack]

@ Banana Jack

Sadly Glenn you found a BUG v1.4.0 indeed does not disable Loop Protection when you disable it in the UI.

I will post v1.4.1rc1 later today with a FIX

Thanks sirhc, yes that agrees with my findings too. I got up early today and downgraded our 17 switches from 1.4.0 to 1.3.9 and the network became stable again so I am looking forward to trying 1.4.1 in due course. I might try it on a bench switch first this time ;)

As part of our troubleshooting experiments yesterday we enabled STP on all switches. The network loop detections ceased, but instead we got repeated discarding/learning/forwarding messages in the log, which again effectively produced short network dropouts, so I think we do have some kind of genuine network problem and the Network Loop Detection was highlighting it. Interestingly, the problem is reproducible by running the UBNT Discovery tool which causes the dropouts every time. I have no idea why.

Anyway thanks for all your help,

Glenn

[Hightech]

This Loop protection bug, can this been courcing that our STP on serveral towers is very unstabel and trigger STP fail over on heaverly loaded radio links?

We are fighting with this since upgrading from 1.3.9 to 1.4.0

Log from one switch:


May 25 17:11:32 STP: set port 3 to discarding
May 25 17:11:34 STP: set port 3 to learning
May 25 17:11:34 STP: set port 3 to forwarding
May 25 17:12:18 STP: set port 3 to discarding
May 25 17:12:19 STP: set port 3 to learning
May 25 17:12:19 STP: set port 3 to forwarding
May 25 17:19:52 STP: set port 3 to discarding
May 25 18:19:07 STP: set port 3 to learning
May 25 18:19:07 STP: set port 3 to forwarding
May 25 18:19:32 STP: set port 3 to discarding
May 25 20:48:28 STP: set port 3 to learning
May 25 20:48:28 STP: set port 3 to forwarding
May 25 20:49:18 STP: set port 3 to discarding
May 25 20:54:50 STP: set port 3 to learning
May 25 20:54:50 STP: set port 3 to forwarding
May 25 21:19:33 STP: set port 3 to discarding
May 25 21:29:49 STP: set port 3 to learning
May 25 21:29:49 STP: set port 3 to forwarding
May 25 21:30:39 STP: set port 3 to discarding
May 25 21:40:58 STP: set port 3 to learning
May 25 21:40:58 STP: set port 3 to forwarding
May 25 22:01:01 STP: set port 3 to discarding
May 26 11:43:11 STP: set port 3 to learning
May 26 11:43:11 STP: set port 3 to forwarding
May 26 11:43:33 STP: set port 3 to discarding
May 26 17:56:29 STP: set port 3 to learning
May 26 17:56:29 STP: set port 3 to forwarding
May 26 17:56:53 STP: set port 3 to discarding
May 26 18:35:51 STP: set port 3 to learning
May 26 18:35:51 STP: set port 3 to forwarding
May 26 18:36:14 STP: set port 3 to discarding
May 27 18:34:13 STP: set port 1 to discarding
May 27 18:34:13 STP: set port 3 to learning
May 27 18:34:13 STP: set port 3 to forwarding
May 27 18:34:13 STP: set port 3 to discarding
May 27 18:34:13 STP: set port 1 to learning
May 27 18:34:13 STP: set port 1 to forwarding
May 27 18:34:21 STP: set port 1 to discarding
May 27 18:34:21 STP: set port 3 to learning
May 27 18:34:21 STP: set port 3 to forwarding
May 27 18:34:22 STP: set port 3 to discarding
May 27 18:34:22 STP: set port 1 to learning
May 27 18:34:22 STP: set port 1 to forwarding
May 27 18:35:29 STP: set port 1 to discarding
May 27 18:35:29 STP: set port 3 to learning
May 27 18:35:29 STP: set port 3 to forwarding
May 27 18:35:29 STP: set port 3 to discarding
May 27 18:35:29 STP: set port 1 to learning
May 27 18:35:29 STP: set port 1 to forwarding

Br.
Carsten