Packet loss during a config commit

User avatar
SYS7EM-Daniel
Member
 
Posts: 10
Joined: Fri Aug 19, 2016 6:11 pm
Location: Austin, TX
Has thanked: 0 time
Been thanked: 0 time

Re: Packet loss during a config commit

Fri Dec 16, 2016 1:55 pm

We have on the schedule to factory default the switch and configure it from scratch this weekend, but I wanted to give an update to the forum today. On the same switch a customer was maxing out their download speeds on a port with QoS enabled to set them at a 10Mbps x 10Mbps package. Once they maxed out the download speeds on their switch port all customers and backhauls connected to the switch saw latency shoot up to around ~380+ ms.

Is this normal behavior?

unspecified.png

User avatar
sirhc
Employee
Employee
 
Posts: 7422
Joined: Tue Apr 08, 2014 3:48 pm
Location: Lancaster, PA
Has thanked: 1609 times
Been thanked: 1326 times

Re: Packet loss during a config commit

Fri Dec 16, 2016 2:02 pm

SYS7EM-Daniel wrote:
My image didn't display in my original post so here's a dropbox link for reference to show my findings.


To upload an image use the "Upload attachment" link under the BLUE submit button

Please post up screen grabs of all your config tabs.
Support is handled on the Forums not in Emails and PMs.
Before you ask a question use the Search function to see it has been answered before.
To do an Advanced Search click the magnifying glass in the Search Box.
To upload pictures click the Upload attachment link below the BLUE SUBMIT BUTTON.

User avatar
SYS7EM-Daniel
Member
 
Posts: 10
Joined: Fri Aug 19, 2016 6:11 pm
Location: Austin, TX
Has thanked: 0 time
Been thanked: 0 time

Re: Packet loss during a config commit

Fri Dec 16, 2016 2:30 pm

Please see the attached screen shots of our configuration.

The customer that was maxing out the connection on port 20.

Screen Shot 2016-12-16 at 12.11.48 PM.png
QoS tab

Screen Shot 2016-12-16 at 12.11.43 PM.png
STP tab

Screen Shot 2016-12-16 at 12.11.37 PM.png
LAG tab

Screen Shot 2016-12-16 at 12.11.24 PM.png
Ports tab

Screen Shot 2016-12-16 at 12.11.15 PM.png
Status tab
Last edited by SYS7EM-Daniel on Fri Dec 16, 2016 2:40 pm, edited 1 time in total.

User avatar
sirhc
Employee
Employee
 
Posts: 7422
Joined: Tue Apr 08, 2014 3:48 pm
Location: Lancaster, PA
Has thanked: 1609 times
Been thanked: 1326 times

Re: Packet loss during a config commit

Fri Dec 16, 2016 2:34 pm

Well it would be nice to see the port descriptions to give us an idea of what each port is doing.

Another words which port or ports feed the switch and which ports go to what
Support is handled on the Forums not in Emails and PMs.
Before you ask a question use the Search function to see it has been answered before.
To do an Advanced Search click the magnifying glass in the Search Box.
To upload pictures click the Upload attachment link below the BLUE SUBMIT BUTTON.

User avatar
sirhc
Employee
Employee
 
Posts: 7422
Joined: Tue Apr 08, 2014 3:48 pm
Location: Lancaster, PA
Has thanked: 1609 times
Been thanked: 1326 times

Re: Packet loss during a config commit

Fri Dec 16, 2016 2:39 pm

So you are not using QOS per say, just port speed limits?

Once again would be nice to know what each port goes to.
Support is handled on the Forums not in Emails and PMs.
Before you ask a question use the Search function to see it has been answered before.
To do an Advanced Search click the magnifying glass in the Search Box.
To upload pictures click the Upload attachment link below the BLUE SUBMIT BUTTON.

User avatar
SYS7EM-Daniel
Member
 
Posts: 10
Joined: Fri Aug 19, 2016 6:11 pm
Location: Austin, TX
Has thanked: 0 time
Been thanked: 0 time

Re: Packet loss during a config commit

Fri Dec 16, 2016 3:21 pm

I have attached a screenshot with descriptions on the status tab. Our customer names would not be much help so I labeled each port as what is on that switch port.
Attachments
Screen Shot 2016-12-16 at 12.11.15 PM.png
Status tab with descriptions

User avatar
sirhc
Employee
Employee
 
Posts: 7422
Joined: Tue Apr 08, 2014 3:48 pm
Location: Lancaster, PA
Has thanked: 1609 times
Been thanked: 1326 times

Re: Packet loss during a config commit

Fri Dec 16, 2016 3:27 pm

Do you have a VLANs defined?
Support is handled on the Forums not in Emails and PMs.
Before you ask a question use the Search function to see it has been answered before.
To do an Advanced Search click the magnifying glass in the Search Box.
To upload pictures click the Upload attachment link below the BLUE SUBMIT BUTTON.

User avatar
sirhc
Employee
Employee
 
Posts: 7422
Joined: Tue Apr 08, 2014 3:48 pm
Location: Lancaster, PA
Has thanked: 1609 times
Been thanked: 1326 times

Re: Packet loss during a config commit

Fri Dec 16, 2016 3:31 pm

I think your problem here is Flow Control.

On any port you are defining bandwidth control disable Flow Control on that port.

My guess is that if you look at the port stats on port 1 (the uplink port) you will see a TON of Tx Pause Frames.

WHat is happening here is a shit ton of packets come into the switch destined to the customer port with a speed limit and the switch tries to buffer up the packets as you are limiting the speed and it fill the switch buffers forcing the switch to send Tx Pause Frames out port 1 to slow them down which would affect everyone on that switch.

In this case you are best to disable Flow Control on any customer port where you are limiting the speed so the switch will simply drop the packets.
Support is handled on the Forums not in Emails and PMs.
Before you ask a question use the Search function to see it has been answered before.
To do an Advanced Search click the magnifying glass in the Search Box.
To upload pictures click the Upload attachment link below the BLUE SUBMIT BUTTON.

User avatar
SYS7EM-Daniel
Member
 
Posts: 10
Joined: Fri Aug 19, 2016 6:11 pm
Location: Austin, TX
Has thanked: 0 time
Been thanked: 0 time

Re: Packet loss during a config commit

Fri Dec 16, 2016 4:08 pm

We do have VLANs defined. We are using one management VLAN and two customer access VLANs.

I do see lots of TX pause frames on the trunk port. I have disabled flow control on ports that we are traffic shaping and I'm monitoring the switch and the attached backhauls. I'll let you know what I see.

Previous
Return to General Discussion

Who is online

Users browsing this forum: No registered users and 79 guests