Odd throughput issue with fiber SFP

[mhoppes]

So I'm fairly new to fiber, but got my first fiber link between two Netonix switches up and running and it's doing great.

Just today I tried to bring a link up between two buildings and I'm having an odd issue I can't explain.

Ping times between the two switches are .247ms, and I can pass very very small amounts of traffic to the Internet at the other side. However, if I really try to go to any website or do much of anything the connection on this side just grinds to a hault. Ping times, however, never waver from .247ms.

Setup is:
Netonix1----((fiber))--->EdgeRouter---->Netonix2----((fiber))---Netonix3

At Netonix 1, EdgerRouter, and Netonix 2 speeds are fine and everything works great. However when connected to Netonix3 things are terrible.

I have no way to test connected directly to the fiber.
Switches are running 1.2.4

Any thoughts?

[sirhc]

YES - UPGRADE YOUR FIRMWARE - YOUR FIRMWARE IS MONTHS OLD

I WOULD USE v1.3.3rc5 as it is about to go final anyway and it has some fixes for SFP like the ability to select speed/duplex on the ports tab which you need to do with some SFP modules to work right. However the speed duplex setting on both switches NEEDS to be the same.

Before we spend a lot of time trying to figure out something that may have been fixed in a firmware release lets take that off the table.

Beyond that you should try swapping out SFP modules one may be bad or your fiber terminations may be bad so you should use say 2 media converters that you have tested to work on the bench to test the fiber cable.

[mhoppes]

I upgraded to 1.3.2 on both switches and now I can actually access the switch on the remote side, but Internet browsing is still really sluggish.

Question. Should I have two arrows on the SFP port? I currently only have one pointing upwards that blinks, not one pointing downwards... yet traffic is flowing and working albeit slowly.

[mhoppes]

I'm downloading 1.3.3rc5 now. Something odd I noticed is it seems to "ramp up" the throughput. I'm downloading this over the fiber line and it initially said it would take 2 hours, then slowly was at 1 hour 30 minutes, then 1 hour, now we're down to 6 minutes -- granted this normally takes less than 2 minutes to download the firmware.

[mhoppes]

Upgrade to 1.3.3rc5 has not solved the issue.

[mhoppes]

A little more investigation and I'm seeing CRC errors on the TX side of the fiber from the remote end.

Is there a way to diagnose this more? See light levels/etc on the SFP modules?

Screen Shot 2015-09-17 at 3.33.49 PM.png (31.46 KiB) Viewed 15289 times

[mike99]

Seem like the MTU error but normaly, it show Rx Drops. Can you try MTU to 1528 ? Old firmware add MTU at 1500 and it could be problematic. Also, have you try both mode of Link ? 1G and auto ?

[mhoppes]

No. I was about to try auto vs 1G and ran out of time today. The other idea I just came up with is we might be using MM gbics on SM fiber.

I have no idea how that would manifest itself.

[mhoppes]

An MTU issue wouldn't cause CRC errors would it?

[mike99]

I don't think so but I add the same kind of beavior to be able to ping but not to browse remote device webui with too much low MTU and it should be higher than 1500 on SFP anyway.