Each of the blue/green, red/yellow and grey/black pairs are in a LAG and the LAGs form a loop over all switches which obviously needs RSTP to prevent packet storms. We use multiple VLANs to partition the switches into (usually) 6 broadcast domains. Each LAG is a VLAN trunk for all of them.
In the lab setup tested now, S1/S2 are DLINK DGS-1210-24 switches with latest firmware and S3/S4 are Netonix WS-24-400A with firmware 1.3.7. Yes, I know 1.3.8 is available now, but it was not when I started this setup and there isn't anything in the release notes that would indicate a fix for the problems I will report. Also, please excuse that the drawing shows connections to ports 25-28 which don't exist on the WS-24-400A of course but just imagine these cables are connected to ports 21-24 instead.
So we got the RSTP ring
S1.23===S3.23 S3.21===S4.21 S4.23===S2:23 S2.21===S1.21
S1.24===S3.24 S3.22===S4.22 S4.24===S2:24 S2.22===S2.22
where the 2 lines are LAG'ed with LACP/A on all ports.
S1 is the STP root bridge with priority 8192.
If S1 fails, S2 is supposed to become the root bridge with priority 16384.
S3 and S4 have priority 32768. "STP on LAGs" is enabled.
Problem #1:
With everything in RSTP left at its defaults, the loop would be prevented by RSTP like this:
S1.23===S3.23 S3.21===S4.21 S4.23<--S2:23 S2.21===S1.21
S1.24===S3.24 S3.22===S4.22 S4.24<--S2:24 S2.22===S2.22
That's not too bad, but (for reasons of where the traffic is expected) I want it to be like this
S1.23===S3.23 S3.21-->S4.21 S4.23===S2:23 S2.21===S1.21
S1.24===S3.24 S3.22-->S4.22 S4.24===S2:24 S2.22===S2.22
The appropriate thing to enforce this would be to add path cost on S4.21 and S4.22.
As the screenshot on the S4 Netonix shows, S4 selects the path to S3 as the root path, even if I add significant path cost to prevent right that. The DLINKs would have 19900 cost for GbE-links. So 50000 should be enough, and I even tried with 200000 and probably more. The only way to get traffic out on S4.23 and S4.24 is by unplugging the LAG cable pair between S3 and S4.
But be careful as you are trying that, as this is how I discovered problem #2.
Problem #2:
In this setup, when I unplug the cable at S3:21 (and only this one) like this
S1.23===S3.23 S3.21 /// S4.21 S4.23===S2:23 S2.21===S1.21
S1.24===S3.24 S3.22===S4.22 S4.24===S2:24 S2.22===S2.22
a storm will start if there is at least one other port connected (not shown here) where packets are injected. The storm can be terminated by reconnecting S3 and S4 fully, i.e. with both LAG legs, or by unplugging the remaining leg.
Obviously, something is wrong with your LAG/RSTP implementation (and actually, as reported in a previous post but still unproven, I believe it is the LAG part). Do I hear you say it is the DLINK's fault? So take the DLINKs out of the picture, doing this (S3 and S4 are the Netonixes)
S4.23===S3.23 S3.21===S4.21
S4.24===S3.24 S3.22===S4.22
which works okay as expected, but as soon as one leg of a LAG is unplugged like this
S4.23===S3.23 S3.21 /// S4.21
S4.24===S3.24 S3.22===S4.22
the storm starts.
You might argue that this may depend on how I used the VLANs on these LAGs - but as a matter of fact it works well as long as both legs of all LAGs are connected and loosing one leg should never cause a storm to develop. You can believe me that the LAG ports have all T's on all VLANs and trunking 1-4096 is enabled.
Actually, I do have a problem #3, but I fear this would get us in the static LAG vs. LACP LAG debate again. So I will put this, if at all, into a separate thread.
