Switches Not Passing Packets
Posted: Tue Aug 04, 2020 1:19 am
There has been at least seven different people reporting issue's with packets not being passed that I've seen so far on the forum's.
Based on the reports, it appears that several switches sent out with the preloaded firmware 1.5.5 may have been afflicted.
What would help us now would be if everyone posted their experience's on this thread so that we can consolidate all the various factor's that are presently spread across the forum's.
As stated above the working theory is that switches preloaded with 1.5.5 are the affected unit's. If there are any counter examples, please post them here.
If you are affected by this you should be able to solve it with one of the following:
If anyone has been sent a switch preloaded with 1.5.5 and DOES NOT have this issue, it would also be helpful to find out who that is.
It will give us a better scope of the situation and potentially what caused it. So if you are reading this and you haven't seen this problem with a switch sent to you that was preloaded with 1.5.5 learning more about your setup or even just when it was sent might help.
What our current theory is on how this happened:
Around the time that me and Eric went to Philly to work directly with Chris to see if we could hammer out some serious problems that we had (the most notorious being the vtss_appl crashes - which was fixed, or at least GREATLY reduced) we were also working on a new software system to help technicians complete serializing switches faster. This also ties into the new RMA system that will hopefully make that entire process smoother for everyone. Many change's to the firmware were necessary during that time and development continues to this day. Potentially something may have happened there that could have caused this. That is what I will be investigating to prevent this from occurring again when 1.5.6 eventually comes out and replaces 1.5.5 as the preloaded firmware.
To that end, any information about your experience seeing this or not seeing this with switches with the preloaded 1.5.5 firmware will be helpful.
Thank you everyone for your reports, I have been watching and running test's in my lab to try and isolate what's going on. For now, please follow steps laid out by others above to prevent it from impacting your own operations as much as possible.
Based on the reports, it appears that several switches sent out with the preloaded firmware 1.5.5 may have been afflicted.
What would help us now would be if everyone posted their experience's on this thread so that we can consolidate all the various factor's that are presently spread across the forum's.
As stated above the working theory is that switches preloaded with 1.5.5 are the affected unit's. If there are any counter examples, please post them here.
If you are affected by this you should be able to solve it with one of the following:
- downgrading to 1.5.3 then back to 1.5.5 has been proven effective which to my knowledge was first reported by yahel here
- upgrading to the latest rc firmware 1.5.6rc1 should also fix it, as reported by t.beesley@centracom.com here
- upgrade to v1.5.5, even though it already is v1.5.5 this would show if the upgrade process itself clears the issue (possible config file issue)
EDIT: jadkins@skynetky.com and mhoppes have both confirmed this method works as well - potentially, defaulting the config on a new switch with 1.5.5 preloaded may solve the issue (no confirmation on this yet, please post here if this works or doesn't work for you)
EDIT: jadkins@skynetky.com and Supernova have confirmed that unfortunately, this does NOT work
If anyone has been sent a switch preloaded with 1.5.5 and DOES NOT have this issue, it would also be helpful to find out who that is.
It will give us a better scope of the situation and potentially what caused it. So if you are reading this and you haven't seen this problem with a switch sent to you that was preloaded with 1.5.5 learning more about your setup or even just when it was sent might help.
What our current theory is on how this happened:
Around the time that me and Eric went to Philly to work directly with Chris to see if we could hammer out some serious problems that we had (the most notorious being the vtss_appl crashes - which was fixed, or at least GREATLY reduced) we were also working on a new software system to help technicians complete serializing switches faster. This also ties into the new RMA system that will hopefully make that entire process smoother for everyone. Many change's to the firmware were necessary during that time and development continues to this day. Potentially something may have happened there that could have caused this. That is what I will be investigating to prevent this from occurring again when 1.5.6 eventually comes out and replaces 1.5.5 as the preloaded firmware.
To that end, any information about your experience seeing this or not seeing this with switches with the preloaded 1.5.5 firmware will be helpful.
Thank you everyone for your reports, I have been watching and running test's in my lab to try and isolate what's going on. For now, please follow steps laid out by others above to prevent it from impacting your own operations as much as possible.