Problem with WS Generating Traffic and dying

[Mac_SPITwSPOTS]

Yes, I was hardwired into port 23 with my laptop. Port 17 connects into a Ubiquiti EdgeRouter Pro.

[tma]

As a FC storm coming in on one of the other ports would be visible in the graphs, I find it convincing that the switch does that 8/15 traffic on its own.

When it happens again, it would be interesting to call up "Port Detail" for port 17 and find out what kind of traffic this could be. FC frames have their own Tx counter and loop protection probes (which is shown enabled in one of the earlier screenshots) would probably increase the Tx Q7 queue counter.

Fortunately, all discovery protocols are turned off and can probably be eliminated as a possible cause. But it's unclear whether you have RSTP enabled or not (in general and on port 17).

[Mac_SPITwSPOTS]

Had 2 more instances of this issue happen yesterday/early today. 02/05 around 5:50pm and 02/06 around 12:50AM. Each instance happened to a different physical switch, both 400A models. Does not appear to be hardware related as far as revisions go as they were different revisions, one E, one D, one of the switches being one we replaced roughly 1 week ago due to this same issue. I was unable to log into the switch that had been recently replaced, labeled as sw1.BTx and ended up having to power cycle it to gain access. I did not have a serial cable + necessary adapters to log into the switch via that method however I will be looking at what it takes to have that set up on my macbook for future use.

[tma]

Yeah, being unable to access the switch on the normal management path has made it impossible for me to send details as demanded by Netonix. To get at least some indication of which port is the source, I told my guys to unplug one port at a time while I had a ping going so I could determine exactly when the switch became accessible again.

[Mac_SPITwSPOTS]

List of switches displaying symptoms in chronological order with a brief summary of what we are seeing for each switch.

sw1.HH WS-12-400-AC: This was the initial switch that displayed these symptoms as far as we can tell, layout is pretty simple with 3 VLANS, switch powers an AF24 and plugs into a Ubiquiti ER-X-SFP that the switch also powers. Symptoms were the 7/8Mpbs with 15Kpps transmitting on a single port with no apparent receiving traffic to match it

sw2.k0x WS-24-400A: For the switch that displayed these symptoms around 5:50pm yesterday, I was able to log into it. It has multiple VLAN's that handle traffic from different physical sites, and was passing around 400Mbps of traffic with no problems in regards to these other sites. This switch also handles all of the internet traffic for the site it is located at, and the port that links to the public interface on our local router was displaying the same symptoms of 7-8Mbps/15Kpps, none of the customer devices that connect to this site had internet until we rebooted this switch as well as the router it was connected to.

sw1.BTx WS-24-400A: This switch starting displaying problems around 12:50AM earlier this morning. On site I was unable to connect to the switch through conventional means of ethernet cable on a port that we had previously accessed it on so not a lot of info able to be gathered. After a reboot it came back up as normal and I was able to log into it. This switch had previously displayed the same symptoms multiple times over a 24hr period roughly a week ago and was replaced with a new switch, the new switch is the one that displayed these same symptoms around 12:50AM earlier this morning. I would be confident saying this rules out hardware failure.

[Mac_SPITwSPOTS]

@sirhc We are currently investigating if this issue is being caused by a pause frame storm that may not be respecting the pause frame option to disable flow control on any switch port that is receiving excessive pause frames. We are testing having flow control disabled on the switch to see if this has any effect.

If we have a QoS limiter specified on a port, and that port is receiving more data than that limiter, would this generate pause frames?

[sirhc]

If we have a QoS limiter specified on a port, and that port is receiving more data than that limiter, would this generate pause frames?

In the latest version of firmware we disable Flow Control on any port that you enable bandwidth control on as this can cause a LOT of Pause Frames

[Mac_SPITwSPOTS]

If we have a QoS limiter specified on a port, and that port is receiving more data than that limiter, would this generate pause frames?

In the latest version of firmware we disable Flow Control on any port that you enable bandwidth control on as this can cause a LOT of Pause Frames

Does hitting the QoS limit generate a pause frame?

[sirhc]

If you have a port limit set and more data goes to the port then allowed to pass then yes it begins to consume port buffers and when the port buffers are full the switch will start sending out Tx Pause Frames out the port receiving the packets heading to port that is limited and now buffers full.