Flow Control..... take 2, 3, 4?

[sbyrd]

You should be enabling flow control on everything, everywhere. But definitely on the links going to the APs and your backhauls.

You will also want to go into the airFiber and enable flow control there, the airMax gear does not have a "FC On" option, but you should still enable it on the router ports if your router supports it.

You mean everywhere that a bridged (L2) device sits between router interfaces? I think FC could be detrimental or not needed between routers that are directly connected.

Also, I have Mikrotik routers at each AP tower, backhaul tower, and in my core and with version 6.11 and higher they do support FC.

I have run tests and never in my network really seen any performance issues with FC off. I have never seen the low download high upload issue. Then again my tests were just done by running an Iperf from the head office to the lan port of the APs at the towers.

[mhoppes]

It probably isn't needed, but, FC doesn't hurt to have enabled... if it isn't needed it just sits there doing nothing.

[mhoppes]

You should be enabling flow control on everything, everywhere. But definitely on the links going to the APs and your backhauls.

You will also want to go into the airFiber and enable flow control there, the airMax gear does not have a "FC On" option, but you should still enable it on the router ports if your router supports it.

You mean everywhere that a bridged (L2) device sits between router interfaces? I think FC could be detrimental or not needed between routers that are directly connected.

Also, I have Mikrotik routers at each AP tower, backhaul tower, and in my core and with version 6.11 and higher they do support FC.

I have run tests and never in my network really seen any performance issues with FC off. I have never seen the low download high upload issue. Then again my tests were just done by running an Iperf from the head office to the lan port of the APs at the towers.

iPerf UDP or TCP? UDP won't show it, TCP will. I've seen it on many networks and FC improves speeds... Chris has seen this as well. An L2 network sits everywhere you have an AP connected to a router and devices hanging off that router.

[sbyrd]

You should be enabling flow control on everything, everywhere. But definitely on the links going to the APs and your backhauls.

You will also want to go into the airFiber and enable flow control there, the airMax gear does not have a "FC On" option, but you should still enable it on the router ports if your router supports it.

You mean everywhere that a bridged (L2) device sits between router interfaces? I think FC could be detrimental or not needed between routers that are directly connected.

Also, I have Mikrotik routers at each AP tower, backhaul tower, and in my core and with version 6.11 and higher they do support FC.

I have run tests and never in my network really seen any performance issues with FC off. I have never seen the low download high upload issue. Then again my tests were just done by running an Iperf from the head office to the lan port of the APs at the towers.

iPerf UDP or TCP? UDP won't show it, TCP will. I've seen it on many networks and FC improves speeds... Chris has seen this as well. An L2 network sits everywhere you have an AP connected to a router and devices hanging off that router.

10 Stream TCP tests. Even though I have not seen any performance improvements I have been turning it on on devices that support it recently. I just wanted to make sure there was not a downside to just going whole hog and turning it on everywhere.

[sbyrd]

It probably isn't needed, but, FC doesn't hurt to have enabled... if it isn't needed it just sits there doing nothing.

So basically if I turn it on and see TX or RX pause frames show up on the router interface it was needed. If I have FC on and no pause frames are counted than it was not needed but does not hurt to have on?

Also if it was not clear my network is fully routed. to the CPE. The only bridging is done on the APs or at PoE switches.

[mhoppes]

That is a correct understanding.

[sirhc]

Also if it was not clear my network is fully routed. to the CPE. The only bridging is done on the APs or at PoE switches.

You want Flow Control from the AP to the Router (make sure your router supports Flow Control on the port(s) feeding the switch) regardless of what is in between so If your router is at the tower like I do (which is the best way) then you want Flow Control on all the AP ports and on the uplink ports. If your router handles it then you could also create a LAG between the switch and the router this way the Pause Frames (and they will occur) will be split across how however many ports are bonded together in your LAG. Now Cisco does NOT support LACP on their Routers (at least my model) so I use STATIC LAGs but the EdgeMAX Routers do support LACP but sadly I "hear" they do NOT support Flow Control which is BAD.

I did not implement EdgeMAX routers because they did not support BFD but I almost did because I love Ubiquiti so much and wanted to support them but in the end we decided to wait and see first if they implemented BFD, boy am I glad I did not switch because without Flow Control my net would SUCK!

I hope they eventually switch from the OLD Vyatta to the new branch of VyOS which does support BFD and a lot more newer stuff then I will consider the switch again.

You can see how I do it HERE.

Currently I have my backhauls configured as mid-spans in my switch but I eventually plan to go back and just create 2 STATIC LAGs between my switch and my router so I have 1 LAG for backhauls and one LAG for local AP's this way Pause Frames generated from my local AP's do not affect my backhauls just carrying packets onto the next tower in my OSPF with BFD ring.

People might say why do you care if your airFIBERS are in your switch as just a mid-span, well let me tell you:
1) I can power cycle them if I want. - YOU NEVER KNOW

2) I can have the switch automatically power cycle them with ping watch dog.

3) I can log into the switch at a tower and get instant traffic graphs up to 1 hour in the past which recently came in handy when we were tracking down an OSPF issue because we could see that traffic first was going in one direction on the graphs then some changes the traffic stopped, a few more changes and I see traffic in both directions as it should be. Now yes you can go get that info from the routers and or NMS but the UI in the switch is more convenient and prettier.

4) Less garbage in my cabinets - NO POE BRICKS AND LESS CABLES AND LESS AC CORDS

5) Easy DETAILED Ethernet Stats on the backhauls

I can keep going but I LOVE having everything in ONE device.

And our switches are TOUGH, 48V dead short one of our switch ports and you "probably" lose the "port" but dead short a ToughSwitch 48V port and you probably lose the entire switch, at least that has been my experience. Maybe your experience is different? "If" you do happen to dead short our switches and you do fry the port and you want the port fixed PLEASE just be honest because the fee is only $50 to repair it. We are small and every dollar counts but if the switch fails on it's own we do have your back!

[mhoppes]

And the hits just keep on coming....

Just discovered the Cisco ME3400E switches we have been deploying will RECEIVE FC but will not send FC.... who designs these things?!??!

[wayneorack]

Just discovered the Cisco ME3400E switches we have been deploying...

Now why would you do that? You should know better. What do they have (besides dual power supply) that the WS does not?

[mhoppes]

The name "Cisco", which some people like... I'm about to recommend ripping three of these out in favor of Netonix and see what happens. Enabled FC on the airFibers and I'm seeing nothing incrementing on the switches.

Now before I jump too far... Chris, is this normal -- that being Flow Control is received by the switch and sent by the end device? Or should the switch be sending a flow control when its buffers are almost full? I would think the last option, but before I throw the baby out, I want to be sure the bath water is actually dirty.