LACP goes down briefly when radio loses power?

[sbyrd]

For now I have disabled all Spanning Tree on the switch as it seems to be causing the LACP flapping. However, I do not know why or what would cause it to flap in the first place.

I will see if this eliminates the flapping.

Code: Select all

 
Jan 14 20:30:26 UI: Configuration changed by admin
Jan 14 20:30:26 UI: STP_LAG_Enable: changed from 'Enabled' to 'Disabled'
Jan 14 20:30:26 UI: Port 7 STP: changed from 'Enabled' to 'Disabled'
Jan 14 20:30:26 UI: Port 8 STP: changed from 'Enabled' to 'Disabled'
Jan 14 20:30:42 UI: Configuration changed by admin
Jan 14 20:30:42 UI: STP_Enable: changed from 'Enabled' to 'Disabled'


[sbyrd]

I am really at a loss as to what is going on so I am posting this to each Mfgs support forum to see if anyone has any insight.



My network is very simple (at least I think so) and I am fully routed with static routes and I use a Netonix switch with a Mikrotik router in a Router-on-a-stick type setup (http://wiki.mikrotik.com/wiki/SwOS/Router-On-A-Stick


Anytime one of the Airfiber Slaves (remote end) goes down and breaks the link to the Master (local switch end) for any reason (power loss, soft reboot, setting change) it causes all traffic for every backhaul to go down, drops my connection to the Netonix switch, and even appears to temporarily stall the tower router (CCR1009) as well.



One other oddity is if I make a setting change on the Master side that breaks the wireless link (changing TX power) nothing locks up or stalls. It is only if the Slave side goes down while the Master is online that I have this issue.


I am at a loss as to whose device is to blame (Airfiber, Netonix, Mikoritk) as none of the devices show any errors or log entries before or after the temporary outage. Correction the Netonix sometimes show TX drops on the backhaul facing ethernet ports immediatley after the switch is reachable again. It does not record any TX drops at any other time, but only during this 'event'.

This is 100% reproducible. All I have to do is logon to one of the Airfiber slaves and for example change the TX power from 20 to 19. As soon as I hit apply all pings to all devices at Master side tower stop and a majority of the time all traffic to the tower router stops and I even lose my Winbox session.

If I have the Active-LACP enabled on the tower router and the switch when a Slave breaks it wireless link besides having all traffic at the tower router stall I then get an email from the Netonix switch saying the LACP state changed.


This is very odd and as I said earlier I do not know if the Airfiber, the switch, or the router is the blame.



The 3 AF links are as follows.



AF5X running 4.0b1

AF5X running 3.2

AF3X running 3.4b1

[sirhc]

Seth, please upgrade switches to v1.4.7rc4, your firmware is REALLY OLD.

Before you upgrade make sure none of your passwords are longer than 8 characters and no special characters as we had a password bug fixed a long times ago.

I think you are possibly seeing Pause Frame storms from your AFX radios which is a known UBNT bug.

v1.4.7rc4 has protection against this.

Read the release notes and get up to date.

Also upgrade AF radios to latest firmware UBNT claims they fixed their Pause Frame bug.

I have no idea how old your MT firmware is but I know ours is OLD.

I upgraded to 1.4.6 and still have this issue.

OK I see this now. You did upgrade, I hope all switches are upgraded????

[sirhc]

Please make a diagram of your towers, links, routers and such.

[sbyrd]

I will say I would NOT use LAGs with wireless links.
http://forum.netonix.com/viewtopic.php? ... ACP#p16189

Please make a diagram of your towers, links, routers and such.

I will warn you now that I suck at diagrams, but I will do my best.

Also I am not using the LAG with the wireless links, but with the Uplink from the switch to the router. Very similar to what you do.
viewtopic.php?f=17&t=237#p836

Ports 1 & 2 are aggregated using Static Aggregation to our Cisco 2951 router, remember that Cisco does not support LACP in a lot of routers.

[sbyrd]

Here is the best I could come up with using the tools I have right now. If it is not sufficient let me know and I will try to do better.

[sbyrd]

I am basically using this configuration on the Mikrotik except substitute the Miktotik Switch running SwOS for a Netonix and the uplink is a LAG. http://wiki.mikrotik.com/wiki/SwOS/Router-On-A-Stick


However, I see the same issue even if I remove the LAG and make the uplink from the Netonix to the Mtik router a standard unbonded Ethernet link.

[sirhc]

Picture/Diagram is great, thanks.

Now please update and screen captures of Config Tabs in thread above to match what current setup is (if anything has changed).

It is late so I may doze off soon but PM me your cell number and times you will be available tomorrow.

Also please load latest Teamviewer on your PC.

Also please update AFX firmware.

By the way I would power your AFX radios with 48V or 48VH as long as they are not the original few that require 24V

Read this post:
viewtopic.php?f=8&t=2370&p=16681&hilit=+AFX#p16681

[sbyrd]

Picture/Diagram is great, thanks.

Now please update and screen captures of Config Tabs in thread above to match what current setup is (if anything has changed).

It is late so I may doze off soon but PM me your cell number and times you will be available tomorrow.

Also please load latest Teamviewer on your PC.

Also please update AFX firmware.

By the way I would power your AFX radios with 48V or 48VH as long as they are not the original few that require 24V

Read this post:
http://forum.netonix.com/viewtopic.php? ... AFX#p16681

Only config changes on the switch in an attempt to resolve the issue.
STP has been disabled on the switch. This eliminated the LACP going up/down randomly but did not fix the data stall issue.

Turned on Loop/Pause Protection and turned off Discovery

New Device Config 2.PNG (8.96 KiB) Viewed 10059 times

It is late here as well and is actually storming/raining here in the Midwest. I make it a habbit to not do infrastructure FW upgrades to outdoor devices in this type of weather so I will not have all the AFX radios on the latest FW by tomorrow. However, I have never seen any signs of Pause Frame storms that others have seen with my AF radios.

I will PM you my contact information, but if you want to wait until I get the FWs upgraded before calling me I understand.

[sirhc]

Can you try the following:

Try disabling STP on LAGs and see if it makes a change.

Try Enabling RSTP on ports bit leave STP on LAGs OFF and see if it happens.

Last verify that if you do not use a LACP LAG the problem goes away.

Another option would be to try a Static LAG instead of LACP.

Let me know what you find out.

I figure this stuff is better to try at night anyway.