VLANs ... "finnicky" (1.2.2 and prior)

DOWNLOAD THE LATEST FIRMWARE HERE
User avatar
josh
Associate
Associate
 
Posts: 109
Joined: Thu Apr 17, 2014 9:29 pm
Location: USA
Has thanked: 69 times
Been thanked: 31 times

Re: VLANs ... "finnicky" (1.2.2 and prior)

Mon Jun 22, 2015 6:39 pm

sirhc wrote:I can not seem to replicate this with the information provided.

Display the config Tabs and provide steps to replicate the behavior because we will fix it if we can replicate it.


Maybe it has to do with a certain REV of switches? IDK.

Like I said above, it is simple as pie for us to reproduce this.

[1] pull switch out of box
[2] plug in two test devices with static ips on them, set one to tagged "vlan 100" on it's interface
[3] plug devices into switch ports
[4] create access port to untagged device
[5] create tagged port to tagged device
[6] observe no traffic
[7] add random ip to vlan, traffic starts flowing instantly
[8] remove unneeded ip on interface, traffic still flows

3 different people in our office have observed the same behavior when setting up switches

note: it doesn't matter if it's access -> tagged or tagged -> tagged, we still see this

User avatar
adairw
Associate
Associate
 
Posts: 465
Joined: Wed Nov 05, 2014 11:47 pm
Location: Amarillo, TX
Has thanked: 98 times
Been thanked: 132 times

Re: VLANs ... "finnicky" (1.2.2 and prior)

Mon Jun 22, 2015 7:04 pm

Maybe I'm not following your instructions correctly. But I don't have a NIB switch but the 24 port sitting on my desk, I defaulted, logged in, setup vlan 100 tagged on a port with a rocket m5 plugged in to it. also built vlan 2 and untagged the rm5 along with another port on the switch. rocket was not reach able. switched cable to port that it was untagged in, and it started pinging right away.

User avatar
sirhc
Employee
Employee
 
Posts: 7415
Joined: Tue Apr 08, 2014 3:48 pm
Location: Lancaster, PA
Has thanked: 1608 times
Been thanked: 1325 times

Re: VLANs ... "finnicky" (1.2.2 and prior)

Mon Jun 22, 2015 7:56 pm

Personally I Untag traffic on the switch port as shown in my example below, but everybody runs their network differently and there is no "correct" way. I think you as passing the VLAN TAG OUT to the radio? In my example below my router is connected to ports 22,23 and my APs are on 6-20. I Untag on each switch port an AP is in making the switch port sort of a logical port in my router with the VLAN function. Port 22 and 23 are a STATIC LAG that way Pause Frames generated by the AP ports are spread across 2 interfaces to the router. The backhauls are just configured as Mid-Spans (VLANs 97.98,99) that way I can reboot them, monitor their power consumption and see how much traffic is on them at a glance with 1 hour of usage history.

vlan1.jpg
vlan1.jpg (231.31 KiB) Viewed 7989 times


Josh could you please be more specific in your Lab, it is simple for you to follow as your doing it your normal way but I am not following you exactly and I need to do exactly what your doing.

Could you please display your VLAN and Ports Tab screen shot.

Specify which device is plugged into what port and if the device has what VLAN configured on it's interface.

Also since you are passing the Tagged frame out the switch interface make sure your MTU is at least 1528 on all your switch ports, this could be your issue as the port MTU must be larger than what ever MTU your running as the packet will now be passing the VLAN ID in the header.

PLEASE TRY SETTING YOUR MTU LARGER AND SEE WHAT HAPPENS

We run a standard 1500 MTU on our network but all of my switches are set to 1526 or 1528 I can not remember which but 1528 will not hurt.

That way your MTU is sufficiently large enough, 1518 is NOT enough for what your doing, please try 1528 and let me know what happens.
Support is handled on the Forums not in Emails and PMs.
Before you ask a question use the Search function to see it has been answered before.
To do an Advanced Search click the magnifying glass in the Search Box.
To upload pictures click the Upload attachment link below the BLUE SUBMIT BUTTON.

User avatar
josh
Associate
Associate
 
Posts: 109
Joined: Thu Apr 17, 2014 9:29 pm
Location: USA
Has thanked: 69 times
Been thanked: 31 times

Re: VLANs ... "finnicky" (1.2.2 and prior)

Wed Jun 24, 2015 3:39 pm

First, THIS HAS NOTHING TO DO WITH MTU. I don't know why you would think enabled/disabling the IP on an interface has ANYTHING to do with MTU.

Chris, we just had a MASSIVE power outage that effected the entire peninsula all the way up to Anchorage.

When devices came back online, one of our sites DIDN'T. On a hunch, I enabled the vlan IP on the wispswitch going to that site and the site IMMEDIATELY came back online.

WTF? Screenshots attached.
Image
Image

Image

User avatar
josh
Associate
Associate
 
Posts: 109
Joined: Thu Apr 17, 2014 9:29 pm
Location: USA
Has thanked: 69 times
Been thanked: 31 times

Re: VLANs ... "finnicky" (1.2.2 and prior)

Wed Jun 24, 2015 3:42 pm

VLAN 1000 is (obviously) a tagged interface that goes over an AF24 link to a TSPro on the other side.

This is the 3rd time that traffic to that vlan from this site has stopped. Every single time, enabling the IP address on the vlan interface has caused traffic to instantly start flowing again. Once it starts flowing, you can disable the IP on the interface and traffic continues.

I have *never* seen anything like this before.

User avatar
sirhc
Employee
Employee
 
Posts: 7415
Joined: Tue Apr 08, 2014 3:48 pm
Location: Lancaster, PA
Has thanked: 1608 times
Been thanked: 1325 times

Re: VLANs ... "finnicky" (1.2.2 and prior)

Wed Jun 24, 2015 4:06 pm

Josh wrote:
First, THIS HAS NOTHING TO DO WITH MTU. I don't know why you would think enabled/disabling the IP on an interface has ANYTHING to do with MTU.



The reason I was thinking MTU is because in a previous thread a user was having troubles passing Tagged frames out of a port because if you are passing Tagged frames OUT a port the packet size will be larger. Read this thread as to why I am asking about MTU. viewtopic.php?f=6&t=698&p=5204&hilit=qinq#p5138

In the first post you make reference to having radios plugged into the switch that have VLANs configured on them

Josh wrote: Like I said above, it is simple as pie for us to reproduce this.
[1] pull switch out of box
[2] plug in two test devices with static ips on them, set one to tagged "vlan 100" on it's interface
[3] plug devices into switch ports
[4] create access port to untagged device
[5] create tagged port to tagged device
[6] observe no traffic
[7] add random ip to vlan, traffic starts flowing instantly
[8] remove unneeded ip on interface, traffic still flows


Thank you for posting the TABs

WOW, this is not a normal VLAN configuration, at least that "I" am used to seeing or implementing?

Once again that does not mean it is wrong just different.

josh wrote:VLAN 1000 is (obviously) a tagged interface that goes over an AF24 link to a TSPro on the other side.

No it is not obvious to me, I am sure it is for you as it is your network.

josh wrote:This is the 3rd time that traffic to that vlan from this site has stopped. Every single time, enabling the IP address on the vlan interface has caused traffic to instantly start flowing again. Once it starts flowing, you can disable the IP on the interface and traffic continues.

Josh if you know enabling an IP on the VLAN prevents this issue from happening might I suggest leaving the IP address active on the VLAN until we figure out what is causing you this grief.

I am having a hard time following exactly what you're doing.

I sure could use a network diagram to help wrap my head around how you're expecting your network to function.

On the diagram can you detail out what equipment is in each port and if there are VLAN's configured beyond the switch.

This is simple for you because this is the way you do your network but it is not what I am used to so it makes perfect sense to you but I will need a little more detail.
Support is handled on the Forums not in Emails and PMs.
Before you ask a question use the Search function to see it has been answered before.
To do an Advanced Search click the magnifying glass in the Search Box.
To upload pictures click the Upload attachment link below the BLUE SUBMIT BUTTON.

User avatar
josh
Associate
Associate
 
Posts: 109
Joined: Thu Apr 17, 2014 9:29 pm
Location: USA
Has thanked: 69 times
Been thanked: 31 times

Re: VLANs ... "finnicky" (1.2.2 and prior)

Wed Jun 24, 2015 4:21 pm

Chris,

This segment is a large bridged network where each site is split off by VLAN.

There is nothing complicated about this setup at all, and it worked FLAWLESSLY whenever we had our TSPROs in place at this site. We replaced the TSPROs because of reliability issues and packet loss long term. All of our other wispswitches are working well with their configs for the most part, and we only see this issue upon initial deployment and, longer-term, at this site repeatedly.

Port Descriptions
1 AF24 data/mgmt* to head end K0x site
2 AF24 data/mgmt to HDx site
3 AP
4 AP
5 AP
6 AP
7 local host cpe (airrouter)
8 Private PtP link
9-14 unused
* (most of our AF24 sites have separate management cables, but this is an older site and they haven't been ran... yet)

VLAN 172 is the local vlan for the site. It is untagged across the board (AF backhauls there are untagged management), but the upstream/downstream devices are only looking for tagged traffic, or converting the untagged traffic to tagged on the inbound.

VLAN 466 carries private multilink traffic to a customer with several sites, ports 2 (backhaul to HDx site), 4/5 (APs)

AND THE IMPORTANT/RELEVANT ONE
VLAN 1000 is the vlan for the HDx site. It exists on the two AF24 ports only (Just noticed it was enabled on unused ports 11 and 12, so disabled those.)

VLAN 1000 is the one that drops, sometimes randomly, but often when there has been a power outage. Looking at the historical data on the AF24 link, it's signal is impeccable and doesn't vary more than 3dB on it's short link. Yay Alaska, where we don't get heavy rains for crap.

User avatar
sirhc
Employee
Employee
 
Posts: 7415
Joined: Tue Apr 08, 2014 3:48 pm
Location: Lancaster, PA
Has thanked: 1608 times
Been thanked: 1325 times

Re: VLANs ... "finnicky" (1.2.2 and prior)

Wed Jun 24, 2015 6:34 pm

I am working on your setup Josh but I want to stress that to do what you are doing you "need" to have an MTU of at least 1522 preferably 1528 on all your ports. You still have not answered me as to the MTU settings.

This switch is an older RevB board which means it shipped with a much older firmware that we originally had a standard MTU of 1518 which is not enough for what you're doing. If you never factory defaulted the switch after upgrading to v1.2.2 but just upgraded them the MTU would remain at the original 1518 standard.

The primary differences between Revs was the Polyfuses we used but there are "very" few Rev B boards out there, less than 200 total. There are some other minor changes between RevB and RevC but from RevC up just heavier PolyFuses. The board Rev would have nothing to do with this issue. Just thought I would throw that out there as people keep asking me what the differences are. So to re-cap the only differences between RevA and the current RevD boards only deal with PolyFuses used and some changes to the MOSFET control circuits nothing to do with how the switch does switchy kind of stuff.

If your switch still has an MTU of 1518 PLEASE set them all up to 1522 or preferably 1528 which is what is our new standard MTU size and YES if the MTU is still 1518 it can mess with the way you are passing VLANs as it increases the packet sizes. Not all packets would care as a lot of packets as smaller than the MTU to begin with.

Now why enabling an IP on the interface seems to fix your issue we are totally baffled however if your MTU is indeed 1518 assigning an IP address to the VLAN "could' cause somethings to happen differently.

But in the mean time "if" what you say is true that assigning an IP to the VLAN prevents it from happening why not leave the IP on there until we fix this.

Since nobody else is reporting VLAN issues this means there is "something" different with your config than other people are using which I want to stress does not mean this is not our fault but rather it makes it more difficult to figure out.

Every customer is important and we want to try and solve this with you Josh but it may take some time and we will have to pass information back and forth. I have asked Eric to review this post and will ask him to LAB what you outlined above tomorrow. I am also going to reach out to Rory (my old CTO) as his VLAN experience is far better than mine and maybe he can weigh in on what might be happening.

As I have said before we do not write the code that runs the switch core no more then Ubiquiti writes the code that runs their ToughSwitch. The ToughSwitch was originally based on the BCM53118 Switch core with an AR7240 SOC driving the UI. The UI is used to configure the core, the switch core itself and ALL the switch abilities are handled by the Core's internal code. With that being said Vitesse who makes our switch core which is from the VSC-742X family of switch Cores is a mature product that has been on the market for several years and is used in many switches under different Names likes Cisco, Level 1, and so on so I highly doubt there is anything wrong with the core. Now there could be something in the way we are configuring the core which is what we have to find out.

One possible difference is in the way we configure the VLAN Tag INGRESS filtering and maybe when we assign an IP to a VLAN it messes with the filters but I do not think this is it either.

Also "if" your MTU is set at 1518 and you assign an IP to the VLAN is Linux somehow increasing the MTU setting but then you remove the IP and the MTU setting sticks or the INGRESS filters for that matter but when you reboot the switch it reverts back to the saved MTU or standard INGRESS Filtering we are using?
We are here to help Josh that is all I can say, I think we make a damn good switch and we try very hard to solve everyone's problems as fast as we can and I know it can be frustrating on your end when something does not work the way you want it to.

Josh.jpg
Josh.jpg (36.24 KiB) Viewed 7967 times
Support is handled on the Forums not in Emails and PMs.
Before you ask a question use the Search function to see it has been answered before.
To do an Advanced Search click the magnifying glass in the Search Box.
To upload pictures click the Upload attachment link below the BLUE SUBMIT BUTTON.

User avatar
josh
Associate
Associate
 
Posts: 109
Joined: Thu Apr 17, 2014 9:29 pm
Location: USA
Has thanked: 69 times
Been thanked: 31 times

Re: VLANs ... "finnicky" (1.2.2 and prior)

Wed Jun 24, 2015 6:55 pm

Chris, I want you to understand something.

I'm not mad. I'm frustrated that we keep running into this bug, but it's not the end of the world and I'm positive it will get fixed once the proper conditions can be reproduced or directly observed by somebody with interior knowledge of the software design.

I'm simply here to report the bug :)

You should see my emails to UBNT this week. Having to pull lines out of various RFCs to document things done incorrectly... :|

User avatar
adairw
Associate
Associate
 
Posts: 465
Joined: Wed Nov 05, 2014 11:47 pm
Location: Amarillo, TX
Has thanked: 98 times
Been thanked: 132 times

Re: VLANs ... "finnicky" (1.2.2 and prior)

Wed Jun 24, 2015 11:42 pm

you-mad-bro.jpg
you-mad-bro.jpg (102.63 KiB) Viewed 7958 times

Previous
Return to Hardware and software issues

Who is online

Users browsing this forum: No registered users and 70 guests