SNMP log file

[Ludvik]

Hi.

I noticed that the /var/log/snmpd.log file is 11 MB big on my switch.

Code: Select all

admin@curve1:/www# ls -l /var/log
-rw-r--r-- 1 admin root 0 Jan 1 1970 lastlog
drwxr-xr-x 2 admin root 0 Mar 25 01:20 lighttpd
-rw-r--r-- 1 admin root 245 Apr 29 18:06 messages
drwxr-xr-x 2 admin root 0 Mar 25 01:20 radius
-rw-r--r-- 1 admin root 11028919 Apr 30 13:41 snmpd.log
-rw-r--r-- 1 admin root 0 Jan 1 1970 wtmp


Contains a large number of records:

Code: Select all

No MemShared line in /proc/meminfo

It is always created after querying OID .1.3.6.1.4.1.2021.4.6.0

What happens when the free space is completely occupied?

Code: Select all

Filesystem Size Used Available Use% Mounted on
rootfs 5.8M 5.8M 0 100% /
/dev/root 5.8M 5.8M 0 100% /rom
/dev/mtdblock6 4.3M 1.2M 3.0M 28% /jffs
mini_fo:/jffs 5.8M 5.8M 0 100% /


[Stephen]

Would you mind running the command:

Code: Select all

cat /var/log/snmpd.log


And sharing the output?

I'm wondering if this might be another piece to the puzzle on the other issue we're dealing with that seems to be related to discovery.

[Ludvik]

317632 lines ...

Code: Select all

Cannot find module (IP-MIB): At line 0 in (none)
Cannot find module (IF-MIB): At line 0 in (none)
Cannot find module (TCP-MIB): At line 0 in (none)
Cannot find module (UDP-MIB): At line 0 in (none)
Cannot find module (HOST-RESOURCES-MIB): At line 0 in (none)
Cannot find module (SNMPv2-MIB): At line 0 in (none)
Cannot find module (SNMPv2-SMI): At line 0 in (none)
Cannot find module (NOTIFICATION-LOG-MIB): At line 0 in (none)
Cannot find module (SNMP-VIEW-BASED-ACM-MIB): At line 0 in (none)
Cannot find module (SNMP-COMMUNITY-MIB): At line 0 in (none)
Cannot find module (SNMP-FRAMEWORK-MIB): At line 0 in (none)
Cannot find module (SNMP-MPD-MIB): At line 0 in (none)
Cannot find module (SNMP-USER-BASED-SM-MIB): At line 0 in (none)
Cannot find module (TUNNEL-MIB): At line 0 in (none)
Cannot find module (UCD-DLMOD-MIB): At line 0 in (none)
Cannot find module (IPV6-ICMP-MIB): At line 0 in (none)
Cannot find module (IPV6-MIB): At line 0 in (none)
Cannot find module (IPV6-TCP-MIB): At line 0 in (none)
Cannot find module (IPV6-UDP-MIB): At line 0 in (none)
Error opening specified endpoint "udp:161"
Server ExitingNo MemShared line in /proc/meminfo
No MemShared line in /proc/meminfo


... last line repeated 317610x (as I said).

Other piece of switch: 883756 lines.
---

I do not have this monitoring on all of netonixes. These pieces have snmp log almost empty. This is *probably* not our memory problem.

[Stephen]

Sorry I know you did say that, I just wanted to see the difference between what I'm seeing on mine and this one.

The only lines that are different are these:

Code: Select all

Error opening specified endpoint "udp:161"
Server ExitingNo MemShared line in /proc/meminfo
No MemShared line in /proc/meminfo


On mine, the very next line should be:

Code: Select all

NET-SNMP version 5.1.2


It appears that port 161 is already being used by something which is blocking snmpd from starting, possibly causing it to continuously restart. Or, it is locking it into a error state and whenever it is queried it shows that error message.

See if you can get the output for the following:

Code: Select all

ps | grep snmp


On mine, the result should be

Code: Select all

969 admin     5292 S    /usr/sbin/snmpd -Lf /dev/null -p /var/run/snmpd.pid


Also, to be safe, there should be no severe consequence if you remove that log file, it will just re-create it when it needs it:

Code: Select all

rm /var/log/snmpd.log


[Ludvik]

snmpd is running. I have graphs ... you see it in other thread. IP 10.106.30.74 is visible in graph header. 1.5.2rc6

Code: Select all

 1083 admin 5488 S /usr/sbin/snmpd -Lf /dev/null -p /var/run/snmpd.pid

This is another switch, also monitored (1.5.0), 884063 lines:

Code: Select all

Cannot find module (IP-MIB): At line 0 in (none)
Cannot find module (IF-MIB): At line 0 in (none)
Cannot find module (TCP-MIB): At line 0 in (none)
Cannot find module (UDP-MIB): At line 0 in (none)
Cannot find module (HOST-RESOURCES-MIB): At line 0 in (none)
Cannot find module (SNMPv2-MIB): At line 0 in (none)
Cannot find module (SNMPv2-SMI): At line 0 in (none)
Cannot find module (NOTIFICATION-LOG-MIB): At line 0 in (none)
Cannot find module (SNMP-VIEW-BASED-ACM-MIB): At line 0 in (none)
Cannot find module (SNMP-COMMUNITY-MIB): At line 0 in (none)
Cannot find module (SNMP-FRAMEWORK-MIB): At line 0 in (none)
Cannot find module (SNMP-MPD-MIB): At line 0 in (none)
Cannot find module (SNMP-USER-BASED-SM-MIB): At line 0 in (none)
Cannot find module (TUNNEL-MIB): At line 0 in (none)
Cannot find module (UCD-DLMOD-MIB): At line 0 in (none)
NET-SNMP version 5.1.2
No MemShared line in /proc/meminfo
...


Deleting file is only temporary solution (but file is not recreated). You need solve error and implementing logrotate.

[Ludvik]

It was not re-created after deleting the log file. I tried "kill -HUP" and hoped to reload the configuration and create a log file.
But the process was killed ... and after a few seconds started: Apr 30 23:49:30 monitor: restarting snmpd

However, I was very surprised at the status of RAM occupation. Instead of 106MB, now only 74!

I myself wonder what will eventually be a mistake. Later I try it on another switch.

[Ludvik]

Test on 10.106.30.74

The CPU usage is strange. I didn't do anything, just checked the log file and got the WEB-GUI open. I would like the htop program to be part of the firmware.

At this state i remove log file and reload snmpd.

Result:

---
My management system also simply check STP root port and bridge. OID .1.3.6.1.2.1.17.2.5.0. Before reload snmpd it has value 65535 as root port and own MAC as root bridge. After reload it is port 0 and 00:00:00:00:00:00 as root bridge. I have STP disabled. It is strange.

[Stephen]

Well part of this I suspected actually - I am really beginning to suspect that the issue we've been attributing to discovery may actually have to do with snmpd.

The error:

Code: Select all

    No MemShared line in /proc/meminfo


Is probably happening because when your snmp client is querying the device for OID .1.3.6.1.4.1.2021.4.6.0
Which, is the amount of free memory available, net-snmp might be trying to make a call to a MemShared variable in meminfo which does not exist on the version of the kernel we have to use. I took a look around and apparently this is not an unknown possibility with net-snmp.

I think we may very well have found the source of the memory leak we've been hunting.