Dropping ports on new WS, what is wrong with my setup?

[Dave]

Nick, thanks a lot for your hard work on this. We have some ideas & are discussing internally how to attack issue.

[nickwhite]

Nick, thanks a lot for your hard work on this. We have some ideas & are discussing internally how to attack issue.

Thanks Dave. The next time it reboots, I'm hoping to catch it before a power cycle and run top and some other things from the command line to see what's going on. I think the CPU is actually spiking during this and I'd like to catch what is causing it for you.

[Dave]

Nick

ok, discussed issue with our software developer. We think we can come up with a test build in the next 1-2 days that will hopefully help with this issue.

dave

[adairw]

Dave, what are the thoughts about what this issue is?!

[Dave]

Sorry guys, when I said test build, I meant a fix.

[sirhc]

Adair,

What we are hypothesizing may be the cause is the way we shape / limit communications from the core to the CPU.

Currently we have a single pipe that all traffic destined to the CPU / Linux (NOT SWITCHING PACKETS DESTINED ELSEWHERE).

This pipe is sized to a "reasonable" level based on nothing going wrong or too out of the ordinary.

Legit traffic to CPU would be as follows:
SNMP
Discovery
Telemetry used for stats
BPDU packets
Users accessing the UI / CLI
Normal ARP and Broadcast traffic within the management VLAN sub-net

However this could be a problem if for instance you have a problem child on your network possibly doing something like ICMP denial, ARP / Broadcast storms, and such which could flood this pipe constricting or delaying legitimate traffic such as BPDU packets that are needed to control switch core functions like RSTP / LAG and such.

This flood of illegitimate traffic could then affect the CPU's ability to properly control the core and cause the CPU to spike further hindering the CPUs ability to control the core.

This could also explain why you have many switches in your network on differnt routed segments that function great but on this segment of your network there is a problem and an inordinate amount of bad traffic is being forwarded to the CPU controlling the core bogging it down.

What Eric is going to do is change this arrangement from a single pipe size to multiple pipes to carry the different types of packets and limit each one separately.

If this hypothesis is correct it will not FIX your "problem" simply prevent it from affecting the switch in such a negative way (locking it up). But you will still have a problem child out there somewhere and you will still want to find it.

[Eric Stern]

Nick,

Try turning off Flow Control. Do all ports to start. If that has an effect then you can try selectively disabling it on specific ports to see which one helped.

[sirhc]

Eric is investigating a possible bad interaction between Flow Control and RSTP.
Will keep you posted.

[adairw]

I'm going to turn flow control off also.

[sirhc]

I'm going to turn flow control off also.

Normally you want Flow control ON so if turning it OFF fixes your issue for now make sure you turn it back on when we release the new v1.4.0rcX