v1.5.6 Bug Reports and Comments

[scracha]

Is the "switch latitude, longitude" a bug or a feature that's been added in latest FW?


Oct 6 09:05:00 switch[1773]: Watchdog 'AF5x Res Tranzit 24VH' failure checking 172.16.1.113, watchdog triggered on port 2 (AF5x Res Tranzit 24VH), action is Enable Power switch latitude: , longitude:

[Stephen]

It's a feature, requested by quadra awhile back.

[david.sovereen@mercury.net]

I have upgraded 20+ switched to 1.5.6 without issue, but have one switch that after being upgraded overnight, gives this over and over and over and SNMP does not work:


Nov 3 08:02:41 UI: Log cleared by admin (::ffff:64.7.161.3)
Nov 3 08:02:51 monitor: restarting snmpd
Nov 3 08:03:05 monitor: restarting snmpd
Nov 3 08:03:18 monitor: restarting snmpd

I changed all settings (thinking it would re-write any underlying configuration that might be slightly wrong), disabled SNMP and re-enabled SNMP, so no avail.

SNMP is not working, so I really need to get this figured out and fixed ASAP.

Thanks,


Dave

[Stephen]

Hey David, couple things to try:

Do the update again.

Second, default switch and apply the configuration again,
Backup your current configuration (from Tools->Backup), then default the switch to try and completely clear out whatever is happening and then re-apply your configuration.

Let us know how that goes.

[Ludvik]

I have upgrade one piece of ws-12-250-DC ... stuck! I do not know why ... same type, same hw revision, same version (1.5.5) and same (restored) configuration in lab - and upgrade was fine.

Before upgrade i reboot, because free RAM was very low.

[Stephen]

Hey Ludvick, could you send me the conf file you used in a PM?

[cbl]

We also had an issue with SNMP on 1.5.6. I backed up the config, reset to factory, still didnt work. Upgraded to 1.5.6 again, and it started responding to snmp requests.

FWIW, before I did the re-upgrade, I saw this in /var/log/snmpd.log. Not sure if it helps:

Code: Select all

 admin@Netonix_Switch:/tmp/log# cat snmpd.log
Cannot find module (IP-MIB): At line 0 in (none)
Cannot find module (IF-MIB): At line 0 in (none)
Cannot find module (TCP-MIB): At line 0 in (none)
Cannot find module (UDP-MIB): At line 0 in (none)
Cannot find module (HOST-RESOURCES-MIB): At line 0 in (none)
Cannot find module (SNMPv2-MIB): At line 0 in (none)
Cannot find module (SNMPv2-SMI): At line 0 in (none)
Cannot find module (NOTIFICATION-LOG-MIB): At line 0 in (none)
Cannot find module (SNMP-VIEW-BASED-ACM-MIB): At line 0 in (none)
Cannot find module (SNMP-COMMUNITY-MIB): At line 0 in (none)
Cannot find module (SNMP-FRAMEWORK-MIB): At line 0 in (none)
Cannot find module (SNMP-MPD-MIB): At line 0 in (none)
Cannot find module (SNMP-USER-BASED-SM-MIB): At line 0 in (none)
Cannot find module (TUNNEL-MIB): At line 0 in (none)
Cannot find module (UCD-DLMOD-MIB): At line 0 in (none)
Cannot find module (IPV6-ICMP-MIB): At line 0 in (none)
Cannot find module (IPV6-MIB): At line 0 in (none)
Cannot find module (IPV6-TCP-MIB): At line 0 in (none)
Cannot find module (IPV6-UDP-MIB): At line 0 in (none)

[corey.b]

I had to just re-apply the 1.5.6 FW and mine started responding to SNMP.

[david.sovereen@mercury.net]

We upgraded all 50-ish Netonix switches in our network from a variety of versions, ranging from 1.4.9 to 1.5.4 to 1.5.6 at the beginning of November and it failed miserably. We use STP and LACP extensively. We ran into massive problems switches spontaneously crashing/rebooting and sometimes locking up, requiring technicians to go onsite and manually power-cycle them. Because logs are lost when crashes and power-cycles happen, I don't have much to share. After reading various threads and replies here (like those from mayheart), we downgraded to 1.5.2 and things stabilized.

The one one log I do have is from a Fiber Ethernet provider, whose equipment logged each time it saw the Gigabit Ethernet interface go down. This is what it looked like:

All CPE logs from November
Nov 9 19:08:14 MN5_2613_13TH eventd: ALARM ID: 1.0004.01 (clear), SEVERITY: major (nsa), MSG: Link down on Client port
Nov 9 19:07:51 MN5_2613_13TH eventd: ALARM ID: 1.0004.01 (present), SEVERITY: major (nsa), MSG: Link down on Client port
Nov 9 19:07:41 MN5_2613_13TH eventd: ALARM ID: 1.0004.01 (clear), SEVERITY: major (nsa), MSG: Link down on Client port
Nov 9 19:07:18 MN5_2613_13TH eventd: ALARM ID: 1.0004.01 (present), SEVERITY: major (nsa), MSG: Link down on Client port
Nov 8 12:31:01 MN5_2613_13TH eventd: ALARM ID: 1.0004.01 (clear), SEVERITY: major (nsa), MSG: Link down on Client port
Nov 8 12:30:31 MN5_2613_13TH eventd: ALARM ID: 1.0004.01 (present), SEVERITY: major (nsa), MSG: Link down on Client port
Nov 8 09:25:09 MN5_2613_13TH eventd: ALARM ID: 1.0004.01 (clear), SEVERITY: major (nsa), MSG: Link down on Client port
Nov 8 09:24:45 MN5_2613_13TH eventd: ALARM ID: 1.0004.01 (present), SEVERITY: major (nsa), MSG: Link down on Client port
Nov 8 09:24:24 MN5_2613_13TH eventd: ALARM ID: 1.0004.01 (clear), SEVERITY: major (nsa), MSG: Link down on Client port
Nov 8 09:24:14 MN5_2613_13TH eventd: ALARM ID: 1.0004.01 (present), SEVERITY: major (nsa), MSG: Link down on Client port
Nov 8 08:28:26 MN5_2613_13TH eventd: ALARM ID: 1.0004.01 (clear), SEVERITY: major (nsa), MSG: Link down on Client port
Nov 8 08:28:02 MN5_2613_13TH eventd: ALARM ID: 1.0004.01 (present), SEVERITY: major (nsa), MSG: Link down on Client port
Nov 8 08:27:42 MN5_2613_13TH eventd: ALARM ID: 1.0004.01 (clear), SEVERITY: major (nsa), MSG: Link down on Client port
Nov 8 08:27:31 MN5_2613_13TH eventd: ALARM ID: 1.0004.01 (present), SEVERITY: major (nsa), MSG: Link down on Client port
Nov 8 08:26:53 MN5_2613_13TH eventd: ALARM ID: 1.0004.01 (clear), SEVERITY: major (nsa), MSG: Link down on Client port
Nov 8 08:25:42 MN5_2613_13TH eventd: ALARM ID: 1.0004.01 (present), SEVERITY: major (nsa), MSG: Link down on Client port
Nov 8 04:56:09 MN5_2613_13TH eventd: ALARM ID: 1.0004.01 (clear), SEVERITY: major (nsa), MSG: Link down on Client port
Nov 8 04:55:45 MN5_2613_13TH eventd: ALARM ID: 1.0004.01 (present), SEVERITY: major (nsa), MSG: Link down on Client port
Nov 8 04:55:24 MN5_2613_13TH eventd: ALARM ID: 1.0004.01 (clear), SEVERITY: major (nsa), MSG: Link down on Client port
Nov 8 04:55:13 MN5_2613_13TH eventd: ALARM ID: 1.0004.01 (present), SEVERITY: major (nsa), MSG: Link down on Client port
Nov 7 21:46:03 MN5_2613_13TH eventd: ALARM ID: 1.0004.01 (clear), SEVERITY: major (nsa), MSG: Link down on Client port
Nov 7 21:45:40 MN5_2613_13TH eventd: ALARM ID: 1.0004.01 (present), SEVERITY: major (nsa), MSG: Link down on Client port
Nov 7 21:45:18 MN5_2613_13TH eventd: ALARM ID: 1.0004.01 (clear), SEVERITY: major (nsa), MSG: Link down on Client port
Nov 7 21:45:08 MN5_2613_13TH eventd: ALARM ID: 1.0004.01 (present), SEVERITY: major (nsa), MSG: Link down on Client port
Nov 7 21:43:59 MN5_2613_13TH eventd: ALARM ID: 1.0004.01 (clear), SEVERITY: major (nsa), MSG: Link down on Client port
Nov 7 21:43:36 MN5_2613_13TH eventd: ALARM ID: 1.0004.01 (present), SEVERITY: major (nsa), MSG: Link down on Client port
Nov 7 21:43:15 MN5_2613_13TH eventd: ALARM ID: 1.0004.01 (clear), SEVERITY: major (nsa), MSG: Link down on Client port
Nov 7 21:43:04 MN5_2613_13TH eventd: ALARM ID: 1.0004.01 (present), SEVERITY: major (nsa), MSG: Link down on Client port
Nov 5 18:06:38 MN5_2613_13TH eventd: ALARM ID: 1.0004.01 (clear), SEVERITY: major (nsa), MSG: Link down on Client port
Nov 5 18:03:05 MN5_2613_13TH eventd: ALARM ID: 1.0004.01 (present), SEVERITY: major (nsa), MSG: Link down on Client port
Nov 5 18:02:48 MN5_2613_13TH eventd: ALARM ID: 1.0004.01 (clear), SEVERITY: major (nsa), MSG: Link down on Client port
Nov 5 18:02:38 MN5_2613_13TH eventd: ALARM ID: 1.0004.01 (present), SEVERITY: major (nsa), MSG: Link down on Client port
Nov 5 18:02:33 MN5_2613_13TH eventd: ALARM ID: 1.0004.01 (clear), SEVERITY: major (nsa), MSG: Link down on Client port
Nov 5 18:02:22 MN5_2613_13TH eventd: ALARM ID: 1.0004.01 (present), SEVERITY: major (nsa), MSG: Link down on Client port
Nov 5 18:01:56 MN5_2613_13TH eventd: ALARM ID: 1.0004.01 (clear), SEVERITY: major (nsa), MSG: Link down on Client port
Nov 5 18:01:46 MN5_2613_13TH eventd: ALARM ID: 1.0004.01 (present), SEVERITY: major (nsa), MSG: Link down on Client port
Nov 5 18:01:41 MN5_2613_13TH eventd: ALARM ID: 1.0004.01 (clear), SEVERITY: major (nsa), MSG: Link down on Client port
Nov 5 18:01:17 MN5_2613_13TH eventd: ALARM ID: 1.0004.01 (present), SEVERITY: major (nsa), MSG: Link down on Client port
Nov 5 18:00:51 MN5_2613_13TH eventd: ALARM ID: 1.0004.01 (clear), SEVERITY: major (nsa), MSG: Link down on Client port
Nov 5 18:00:40 MN5_2613_13TH eventd: ALARM ID: 1.0004.01 (present), SEVERITY: major (nsa), MSG: Link down on Client port
Nov 5 18:00:02 MN5_2613_13TH eventd: ALARM ID: 1.0004.01 (clear), SEVERITY: major (nsa), MSG: Link down on Client port
Nov 5 17:59:37 MN5_2613_13TH eventd: ALARM ID: 1.0004.01 (present), SEVERITY: major (nsa), MSG: Link down on Client port
Nov 5 17:59:10 MN5_2613_13TH eventd: ALARM ID: 1.0004.01 (clear), SEVERITY: major (nsa), MSG: Link down on Client port
Nov 5 17:59:00 MN5_2613_13TH eventd: ALARM ID: 1.0004.01 (present), SEVERITY: major (nsa), MSG: Link down on Client port
Nov 5 17:58:22 MN5_2613_13TH eventd: ALARM ID: 1.0004.01 (clear), SEVERITY: major (nsa), MSG: Link down on Client port
Nov 5 17:57:58 MN5_2613_13TH eventd: ALARM ID: 1.0004.01 (present), SEVERITY: major (nsa), MSG: Link down on Client port
Nov 5 17:57:38 MN5_2613_13TH eventd: ALARM ID: 1.0004.01 (clear), SEVERITY: major (nsa), MSG: Link down on Client port
Nov 5 17:57:27 MN5_2613_13TH eventd: ALARM ID: 1.0004.01 (present), SEVERITY: major (nsa), MSG: Link down on Client port
Nov 5 17:57:03 MN5_2613_13TH eventd: ALARM ID: 1.0004.01 (clear), SEVERITY: major (nsa), MSG: Link down on Client port
Nov 5 17:56:53 MN5_2613_13TH eventd: ALARM ID: 1.0004.01 (present), SEVERITY: major (nsa), MSG: Link down on Client port
Nov 5 17:56:48 MN5_2613_13TH eventd: ALARM ID: 1.0004.01 (clear), SEVERITY: major (nsa), MSG: Link down on Client port
Nov 5 17:56:25 MN5_2613_13TH eventd: ALARM ID: 1.0004.01 (present), SEVERITY: major (nsa), MSG: Link down on Client port
Nov 5 17:56:04 MN5_2613_13TH eventd: ALARM ID: 1.0004.01 (clear), SEVERITY: major (nsa), MSG: Link down on Client port
Nov 5 17:55:54 MN5_2613_13TH eventd: ALARM ID: 1.0004.01 (present), SEVERITY: major (nsa), MSG: Link down on Client port
Nov 5 17:55:17 MN5_2613_13TH eventd: ALARM ID: 1.0004.01 (clear), SEVERITY: major (nsa), MSG: Link down on Client port
Nov 5 17:54:33 MN5_2613_13TH eventd: ALARM ID: 1.0004.01 (present), SEVERITY: major (nsa), MSG: Link down on Client port
Nov 5 16:44:09 MN5_2613_13TH eventd: ALARM ID: 1.0004.01 (clear), SEVERITY: major (nsa), MSG: Link down on Client port
Nov 5 16:43:45 MN5_2613_13TH eventd: ALARM ID: 1.0004.01 (present), SEVERITY: major (nsa), MSG: Link down on Client port
Nov 5 16:43:24 MN5_2613_13TH eventd: ALARM ID: 1.0004.01 (clear), SEVERITY: major (nsa), MSG: Link down on Client port
Nov 5 16:43:13 MN5_2613_13TH eventd: ALARM ID: 1.0004.01 (present), SEVERITY: major (nsa), MSG: Link down on Client port
Nov 5 16:42:22 MN5_2613_13TH eventd: ALARM ID: 1.0004.01 (clear), SEVERITY: major (nsa), MSG: Link down on Client port
Nov 5 16:41:58 MN5_2613_13TH eventd: ALARM ID: 1.0004.01 (present), SEVERITY: major (nsa), MSG: Link down on Client port
Nov 5 16:41:38 MN5_2613_13TH eventd: ALARM ID: 1.0004.01 (clear), SEVERITY: major (nsa), MSG: Link down on Client port
Nov 5 16:41:27 MN5_2613_13TH eventd: ALARM ID: 1.0004.01 (present), SEVERITY: major (nsa), MSG: Link down on Client port
Nov 5 16:16:25 MN5_2613_13TH eventd: ALARM ID: 1.0004.01 (clear), SEVERITY: major (nsa), MSG: Link down on Client port
Nov 5 16:16:01 MN5_2613_13TH eventd: ALARM ID: 1.0004.01 (present), SEVERITY: major (nsa), MSG: Link down on Client port
Nov 5 16:15:40 MN5_2613_13TH eventd: ALARM ID: 1.0004.01 (clear), SEVERITY: major (nsa), MSG: Link down on Client port
Nov 5 16:15:30 MN5_2613_13TH eventd: ALARM ID: 1.0004.01 (present), SEVERITY: major (nsa), MSG: Link down on Client port
Nov 4 08:31:24 MN5_2613_13TH eventd: ALARM ID: 1.0004.01 (clear), SEVERITY: major (nsa), MSG: Link down on Client port
Nov 4 08:27:21 MN5_2613_13TH eventd: ALARM ID: 1.0004.01 (present), SEVERITY: major (nsa), MSG: Link down on Client port
Nov 4 06:25:06 MN5_2613_13TH eventd: ALARM ID: 1.0004.01 (clear), SEVERITY: major (nsa), MSG: Link down on Client port
Nov 4 06:24:42 MN5_2613_13TH eventd: ALARM ID: 1.0004.01 (present), SEVERITY: major (nsa), MSG: Link down on Client port
Nov 4 06:24:22 MN5_2613_13TH eventd: ALARM ID: 1.0004.01 (clear), SEVERITY: major (nsa), MSG: Link down on Client port
Nov 4 06:24:11 MN5_2613_13TH eventd: ALARM ID: 1.0004.01 (present), SEVERITY: major (nsa), MSG: Link down on Client port
Nov 2 16:56:14 MN5_2613_13TH eventd: ALARM ID: 1.0004.01 (clear), SEVERITY: major (nsa), MSG: Link down on Client port
Nov 2 16:55:50 MN5_2613_13TH eventd: ALARM ID: 1.0004.01 (present), SEVERITY: major (nsa), MSG: Link down on Client port
Nov 2 16:55:28 MN5_2613_13TH eventd: ALARM ID: 1.0004.01 (clear), SEVERITY: major (nsa), MSG: Link down on Client port
Nov 2 16:55:18 MN5_2613_13TH eventd: ALARM ID: 1.0004.01 (present), SEVERITY: major (nsa), MSG: Link down on Client port
Nov 2 16:49:40 MN5_2613_13TH eventd: ALARM ID: 1.0004.01 (clear), SEVERITY: major (nsa), MSG: Link down on Client port
Nov 2 16:49:12 MN5_2613_13TH eventd: ALARM ID: 1.0004.01 (present), SEVERITY: major (nsa), MSG: Link down on Client port
Nov 2 16:48:50 MN5_2613_13TH eventd: ALARM ID: 1.0004.01 (clear), SEVERITY: major (nsa), MSG: Link down on Client port
Nov 2 16:48:39 MN5_2613_13TH eventd: ALARM ID: 1.0004.01 (present), SEVERITY: major (nsa), MSG: Link down on Client port
Nov 2 16:48:31 MN5_2613_13TH eventd: ALARM ID: 1.0004.01 (clear), SEVERITY: major (nsa), MSG: Link down on Client port
Nov 2 16:48:20 MN5_2613_13TH eventd: ALARM ID: 1.0004.01 (present), SEVERITY: major (nsa), MSG: Link down on Client port
Nov 1 06:06:05 MN5_2613_13TH eventd: ALARM ID: 1.0004.01 (clear), SEVERITY: major (nsa), MSG: Link down on Client port
Nov 1 06:05:35 MN5_2613_13TH eventd: ALARM ID: 1.0004.01 (present), SEVERITY: major (nsa), MSG: Link down on Client port

November 9 is obviously when we downgraded back to 1.5.2. The switch has been solid since.

It wasn't clear to us at the time that the Netonix switch upgrade was the problem, as we were performing lots of both hardware and software upgrades on lots of devices in the network at the time. But after downgrading to 1.5.2 and the problem disappearing, it was crystal clear.

What can we do to try and find out the cause and help get the fix for this, without causing outages ticking off our customers at the same time (which happened a lot at the beginning of November)?

Thanks,
Dave

[Stephen]

Dealing with this number of devices and this little of detail I'm afraid there isn't much I can do to even guess as to what happened. I know that's not your fault though, I have been experimenting for awhile now with an option to make a limited size log permanent after reboot so that when things like this occur the reports can be more helpful.

What suggestions from mayheart did you find helpful?