Flow Control..... take 2, 3, 4?

[sirhc]

When you send packets via TCP say from NY to Japan the packets are handled from Router to Router very much the same way they would be handled from Computer to Computer in the same building on a Layer 2 Switch.

So when a router somewhere in the middle of the route is talking to the next router in the route and there was a transmission error such as a CRC there can absolutely be a re transmission of that packet between those 2 routers.

Now if a packet gets dropped for what ever reason such as a buffer over flowed or retrys are given up on somewhere and the packet was lost and never showed up at the destination it is the decision of the FINAL end point to decide if there will be a re-transmission request sent to the ORIGINAL source. But this gets into window size and protocol implemented as UDP destinations will not ask for the originator to re-transmit any lost packets nor will it happen anywhere in between along the route.

https://en.wikipedia.org/wiki/Retransmi ... a_networks)

Retransmission
Retransmission is a very simple concept. Whenever one party sends something to the other party, it retains a copy of the data it sent until the recipient has acknowledged that it received it. In a variety of circumstances the sender automatically re transmits the data using the retained copy. Reasons for re-sending include:
if no such acknowledgment is forthcoming within a reasonable time (most common cause of duplicate packets), the time-out the sender discovers, often through some out of band means, that the transmission was unsuccessful if the receiver knows that expected data has not arrived, and so notifies the sender if the receiver knows that the data has arrived, but in a damaged condition, and indicates that to the sender

One of the things TCP does at the FINAL distention is reassemble and order packets throwing away duplicate packets. How do you suppose there was duplicate packets? The most common way duplicate packets arrive at the destination is a re-transmission occurred when it really did not need to occur which by the way is a symptom of buffer bloat. A packet was held in the buffer too long and the sender took it as a time out and decided to re-transmit all on his own but in reality the packet was not lost just held in a buffer too long.

Even with protocols such as UDP there is still handshaking and other communications going on from router to router, UDP uses FEC or Forward Error Correction where it will sort of guess at what the corrupted data should have been by using an error-correcting code (ECC). The American mathematician Richard Hamming pioneered this field in the 1940s and invented the first error-correcting code in 1950.

[tma]

So when a router somewhere in the middle of the route is talking to the next router in the route and there was a transmission error such as a CRC there can absolutely be a re transmission of that packet between those 2 routers.

Such retransmission between routers COULD occur but please tell me at which layer it occurs and which protocol (level) would do that. Ethernet uses retransmission only if it detects a collision as part of CSMA/>CD< but that is no longer an issue since we all moved from hubs to switches. Other than that, the sender won't be able to determine the fate of the packet it just sent, so there would have to be some ACK (or NACK) signalling back from the receiver in case of a bit flip or similar - never heard of such a thing. ACKs or NACKs would be bad anyway (as they are in 802.11) because they bring distance and the speed of light into play (as they do in 802.11, unless you go Airmax or nstream).

So while the concept of retransmission certainly exists and could be used, it is not used on Layer 2 and 3, and on layer 4 only in some protocols such as TCP (but then end-to-end), or even not up to layer 7 if shoot-and-forget is the only viable option (as in VoIP RTP). Retransmission occurs end-to-end as part of TCP - and devices on the path can manipulate that by early dropping packets - but there is no lower level of communication between routers (and special to them) that avoids overflowing each others buffers other than (again) FC, or that causes a "private" retransmission once an overflow or CRC happened. Such features were added in 802.11 for wireless transmissions ... not sure whether that was a good design decision even for pure Wifi in-house nets.

[tma]

Even with protocols such as UDP there is still handshaking and other communications going on from router to router, UDP uses FEC or Forward Error Correction where it will sort of guess at what the corrupted data should have been by using an error-correcting code (ECC).

Sorry, I hate to say no, partly because it's you, Chris, but also because this thread is (otherwise) so helpful about FC. But:

When routers talk to each other in order to forward a packet, they do that on Ethernet (or something else at THAT layer). Routers will happily pass on a damaged UDP packet as long as it is okay at the IP frame layer, because the only thing they need to look at is the destination IP address - well, unless you tell them to do connection tracking and firewalling, which is not the primary purpose of routers. The IP layer has its own checksum (which only covers the IP header, btw).

Also, like Ethernet and IP, UDP does not use FEC. The UDP header has a checksum but that lets the end-point receiver decide only whether the UDP packet is okay or not. Actually, because a simple checksum algorithm is used, certain errors will go by undetected. That is far from being able to fix anything after receiving a damaged packet.

[sirhc]

Well Eric and I was discussing this on skype just a little bit ago and he leans towards your description of how it is handled between routers along the route so I will defer until I have time to do more reading on the subject but I think he agreed with me that Flow Control needs to make it back to a router?

Having Flow Control just on the switch port facing the AP does very little if any, you need to be able to pass the pause frame back to the router which means you need Flow Control turned on for the port facing the AP and also on the port facing the router and the Router needs to be able to deal with Flow Control. So with my switches which I have at every tower connected to a Cisco 2951, I would replace with Edge Routers if they supported BFD and Flow Control which I am told is coming? I see the Tx Pause Frames on the interface facing the router and NO Pause Frames on the interface facing the AP. If I turn Flow Control off on the router port performance suffers greatly during peak hours.

Now this gets into TCP congestion mechanisms such as sliding window sizes and such....need to do more reading obviously.

Now I am told Ubiquiti will be implementing Flow Control on a firmware release "soon", I hope that is not the same definition used to describe UNII-I and UNII-II release time frame? The entire UBNT router line hardware supports Flow Control so it is just a software issue.

And Thomas, always feel free to disagree with me, I am wrong sometimes but people have to prove me wrong before I realize I am wrong.

[tma]

I agree on everything you said about FC and why it is necessary with lossy half-duplex wireless links. The router should/must support FC.

Yes, the UBNT EdgeMax guys have BFD in 1.8.0-alpha. Not sure about FC support, though. All I can say is that if the *EdgeMax* guys said they will do it, they will do it pretty soon (most of the time). Just look at their release note postings - it's a pleasure to work with them to improve EdgeOS. But the same is true for this forum :-)

[ste]

Well Eric and I was discussing this on skype just a little bit ago and he leans towards your description of how it is handled between routers along the route so I will defer until I have time to do more reading on the subject but I think he agreed with me that Flow Control needs to make it back to a router?

Having Flow Control just on the switch port facing the AP does very little if any, you need to be able to pass the pause frame back to the router which means you need Flow Control turned on for the port facing the AP and also on the port facing the router and the Router needs to be able to deal with Flow Control. So with my switches which I have at every tower connected to a Cisco 2951, I would replace with Edge Routers if they supported BFD and Flow Control which I am told is coming? I see the Tx Pause Frames on the interface facing the router and NO Pause Frames on the interface facing the AP. If I turn Flow Control off on the router port performance suffers greatly during peak hours.

Now this gets into TCP congestion mechanisms such as sliding window sizes and such....need to do more reading obviously.

Now I am told Ubiquiti will be implementing Flow Control on a firmware release "soon", I hope that is not the same definition used to describe UNII-I and UNII-II release time frame? The entire UBNT router line hardware supports Flow Control so it is just a software issue.

And Thomas, always feel free to disagree with me, I am wrong sometimes but people have to prove me wrong before I realize I am wrong.

Having flow control just between the switch and the AP adds up the ap buffering and the switch buffering. This helps reducing problems with small bursts. That is what FC is made for. FC should not be used to buffer longer periods of high traffic as this would add jitter and latency. At some point it is better to drop packets than to hold them too long. Keep in mind that FC pauses all traffic on an interface not just the burst traffic.

[sirhc]

You want Flow Control to the Router.

I have tested this many times at my WISP. Turning Flow control only on the interface facing the AP did not do very much but once I enabled on the interface facing the router things were much better.

Also only enabling Flow Control only on the interface facing the router and not the interface facing the AP did not help it was when I had Flow Control from the AP to the Router did my network function best.

[jjonsson]

Can't find anything on http://community.ubnt.com/ stating that FC on EdgeRouter is being worked on ?!?

[sirhc]

Can't find anything on http://community.ubnt.com/ stating that FC on EdgeRouter is being worked on ?!?

This is information that Stig and Ancheng have been disseminating direct to some users such as Matt and Mike.

[mike99]

I wonder if you would had the same result with a low end router like Mikrotik or ER (if the ever add FC) ?