Segfault after config changes v1.52

[cswanson]

One of our WS-8-150-AC switches in our WISP network crashed after making configuration changes. It was running the v1.52 firmware. The problem didn't present right away, much like the last one that crashed in this manner--it was several hours later that it stopped responding. Upon physically recovering the device from it's installed location, booting with a null modem cable showed a series of segfaults in the kernel. As with the last time, resetting to factory default remedied the issue and the switch is functioning again and on our test bench being stress-tested.

I'm a little nervous to put this switch back into production because it sucks to have a switch crash unpredictably when it's installed on top of a mountain but we'll see how the testing goes. I know in other posts on this forum, people were seeing this with a previous firmware version but I thought it had been remedied so I thought I should share that we are still seeing this.

I'm going to upgrade other switches on the network to 1.54 after some testing.

[sirhc]

A segmentation fault like described is from a corrupted config file.

A power on factory default will clear the corruption.

What causes this?

I am going to guess that there could have been a communication issues between the computer and the switch when the configuration was sent from the workstation to the switch such as an over loaded wireless link or poor link or possibly packets were dropped due to some condition.

I am not sure if this communications is checksum 'd, this would be a question Eric or Stephen would need to answer. If not I will suggest we put that in the next release.

I have also seen this occur when users write scripts to affect the switch configuration on their own.

I would also assume that on rare occasions it could occur if the flash chip suffers a bad block just like a hard drive does. A power on default will re-format that part of the flash blocking out bad blocks or recovering them cleanly.

[cswanson]

That's great information. We were fighting some layer 2 issues at the time on that portion of the network so this explanation makes a whole lot of sense. I would love to find out if there's a checksum of some sort in place and if not, yes, that would be excellent to see in a future release.

Thanks for this info!