SNMP Collector logfile SQL error: database disk image is malformed
Situation:
The SNMP collector SQLite database file can get corrupted with a power failure. This will result in the logging of a huge amount of errors in the messages log-file. Which will eventually raise alarms of a full log directory.
Symptomes:
- systemmanagertrap (305,918) on snmp-col-02-dev-a1.iplease.local. message: On sysmgr@snmp-col-02-dev-a1.iplease.local for service sysmgr, log storage used reached 100.5%
Solution:
execute:
sqlite3 /opt/recorder/snmpcol/snmpcol.db "PRAGMA integrity_check;"
should normally return "ok"
When returns errors:
- execute: "service recorder-sysmgr stop"
- execute: "rm /opt/recorder/snmpcol/snmpcol.db"
- clean up log directory:
- execute: "ls -l /var/log/"
- find biggest uncompressed logfile
- mv /var/log/messages-[BIGGEST] /tmp/
- cd /tmp
- gzip messages-[BIGGEST]
- mv messages-[BIGGEST].gz /var/log/
- execute: "service recorder-sysmgr start"