if there is a god: August 2004 Archives

« July 2004 | Main | September 2004 »

August 27, 2004

OpenNMS .III.

Monitoring Squid via ONMS

Khoo Kah Jin
v1.3, August 2004

Abstract:
This walkthrough will guide the reader to configure Squid’s utilization in ONMS.

1. We’ll begin by determining whether or not squid was precompiled with the –-enable-snmp flag.

# squid -v
Squid Cache: Version 2.5.STABLE3
configure options: --host=i386-redhat-linux --build=i386-redhat-linux --target=
i386-redhat-linux-gnu --program-prefix= --prefix=/usr --exec-prefix=/usr --bindi
r=/usr/bin --sbindir=/usr/sbin --sysconfdir=/etc --datadir=/usr/share --included
ir=/usr/include --libdir=/usr/lib --libexecdir=/usr/libexec --localstatedir=/var
--sharedstatedir=/usr/com --mandir=/usr/share/man --infodir=/usr/share/info --e
xec_prefix=/usr --bindir=/usr/sbin --libexecdir=/usr/lib/squid --localstatedir=/
var --sysconfdir=/etc/squid --enable-poll --enable-snmp --enable-removal-policie
s=heap,lru --enable-storeio=aufs,coss,diskd,null,ufs --enable-ssl --with-openssl
=/usr/kerberos --enable-delay-pools --enable-linux-netfilter --with-pthreads --e
nable-basic-auth-helpers=LDAP,NCSA,PAM,SMB,SASL,MSNT,winbind --enable-ntlm-auth-
helpers=SMB,winbind,fakeauth --enable-external-acl-helpers=ip_user,ldap_group,un
ix_group,wbinfo_group,winbind_group --enable-auth=basic,ntlm --enable-useragent-
log --enable-referer-log

Once this has been confirmed, proceed by editing the necessary options in /etc/squid/squid.conf

2. We’ll need to specify a community string:

acl snmp182 snmp_community public

The port that squid will listen for snmp queries:

snmp_port 3401

Don’t forget to allow snmp enquiries from the acl you created

snmp_access allow snmp182 all

Specify a range to allow incoming messages

snmp_incoming_address 0.0.0.0

And finally the address to return those messages

snmp_outgoing_address 255.255.255.255

Keep in mind of the following paragraph, taken from squid’s config file:

“NOTE, snmp_incoming_address and snmp_outgoing_address can not have
same value since they both use port 3401. “

Restart squid. Just to make sure it works, perform a snmpwalk:

# snmpwalk -m /tmp/squid.mib -v 1 -c public localhost:3401 1.3.6.1.4.1.3495.1

You should be able to see some results from here.

3. Now that we are able to query squid’s mib, it’s now time to include their respective oids in datacollection-config.xml Start running this script:

/opt/OpenNMS/contrib/mibparser/dist/parseMib.sh /etc/squid/mib.txt

Copy the results into datacollection-config.xml as a group, like so:

Add the following oids, since not all of it are compatible with ONMS:

Remember to include the group in the respective system definition. (Note: The system definition might already have been defined, so locate one first before defining your own)

4. At this point, we have not yet defined as to how ONMS is supposed to query Squid at port 3401. Thankfully, net-snmp has a solution for this. What we have to do is to get net-snmp to proxy requests for Squid. Edit snmpd.conf and add the following line:

proxy -m /tmp/squid.mib -c public -v 1 localhost:3401 .1.3.6.1.4.1.3495.1

This line will enable a query to forward its request to squid, whenever the OID of .1.3.6.1.4.1.3495.1 is defined. Perform a snmpwalk, this time without specifying port 3401:

# snmpwalk -m /tmp/squid.mib -v 1 -c public localhost 1.3.6.1.4.1.3495.1

5. If you have gotten this far, it might be a good idea to check whether or not rrds have been created in the nodeid’s directory. If it indeed has, you’re all set to define those graphs. If it’s not there, read on.

6. First up, check your logs. Pay special attention to collectd.log because that’s where all your clues are. It might be a good idea to |grep the IP address of the monitored server.

2004-08-19 17:29:27,213 DEBUG [CollectdScheduler-5 Pool-fiber1] SnmpCollector: Failed to retrieve interface count from remote host 134.211.22.72
2004-08-19 17:29:27,463 DEBUG [CollectdScheduler-5 Pool-fiber1] CollectableService: run: change in collection status, generating event.
2004-08-19 17:29:27,464 DEBUG [CollectdScheduler-5 Pool-fiber1] CollectableService: sendEvent: Sent event uei.opennms.org/nodes/dataCollectionFailed for 37/134.211.22.72/SNMP

This particular log is very generic, because it doesn’t really tell you what went wrong. Next, check on your snmpd service to see if it’s still running. If it’s not, you’d probably get this message from snmpd.log:

Aug 19 09:46:16 localhost snmpd[16055]: response to proxy request illegal. We're screwed.

Looks like it died. This is where things get interesting. For now, edit your datacollection-config.xml and change the ifType from ‘ignore’ to ‘all’:

Restart opennms and snmpd. You will now have your oids in line for collection:

2004-08-18 17:15:35,771 DEBUG [main] SnmpCollector: buildDataSourceList: ds_name: cacheSysVMsize ds_oid: .1.3.6.1.4.1.3495.1.1.1.0 ds_max: U ds_min: U

But the rrds were not generated!

2004-08-18 17:16:35,782 DEBUG [CollectdScheduler-5 Pool-fiber4] SnmpCollector: updateRRDs: Skipping update, no data retrieved for node/ifindex: 14/3 datasource: cacheSysVMsize

Upon closer inspection:

2004-08-18 17:16:35,328 DEBUG [SnmpPortal--1] SnmpIfCollector: getNextSnmpV2Pdu: adding oid to pdu: .1.3.6.1.4.1.3495.1.1.1
2004-08-18 17:16:35,328 DEBUG [SnmpPortal--1] SnmpIfCollector: SnmpCollector.snmpReceivedPdu(): Sending next GETBULK packet.
2004-08-18 17:16:35,329 DEBUG [SnmpPortal--1] SnmpIfCollector: snmpReceivedPdu: got an SNMP pdu, num vars=1
2004-08-18 17:16:35,330 DEBUG [SnmpPortal--1] SnmpIfCollector: snmpReceivedPdu: interface SNMP response arrived. Handling GETBULK response.

Notice how SnmpIfCollector is issuing a GETBULK via snmpv2. This is where our problem is. We have to keep in mind that squid’s snmp module only supports snmpv1. So in this case, we have to reconfigure snmp-config.xml, and add the following line:

This will enforce a collection via snmpv1, and therefore solving the issue. Now it’s time to see if it worked:

2004-08-19 18:45:19,301 DEBUG [CollectdScheduler-5 Pool-fiber0] SnmpCollector: createRRD: rrd path and file name to create: /var/opennms/rrd/snmp/64/cacheSysVMsize.rrd
2004-08-19 18:45:19,301 DEBUG [CollectdScheduler-5 Pool-fiber0] SnmpCollector: updateRRDs: Issuing RRD update command: update /var/opennms/rrd/snmp/64/cacheSysVMsize.rrd N:84

And there you have it. If needed, you may want to rename your ifType back to ‘ignore’.

7. All you need to do now is to edit snmp-graph.properties and define those graphs. For data sources that exceed the 19 character limit, be sure to specify the truncated data source, like below:

report.cacheCurrentSwapSiz.name=Disk Space used by Squid
report.cacheCurrentSwapSiz.columns=cacheCurrentSwapSiz
report.cacheCurrentSwapSiz.type=node
report.cacheCurrentSwapSiz.command=--title="Disk Space used by Squid" \
DEF:cacheCurrentSwapSiz={rrd1}:cacheCurrentSwapSiz:AVERAGE \
LINE2:cacheCurrentSwapSiz#0000ff:"MB" \
GPRINT:cacheCurrentSwapSiz:AVERAGE:" Avg \\: %8.2lf %s" \
GPRINT:cacheCurrentSwapSiz:MIN:"Min \\: %8.2lf %s" \
GPRINT:cacheCurrentSwapSiz:MAX:"Max \\: %8.2lf %s\\n"

Failure to do so will result in missing graphs from the SNMP performance report.

Data sources that are graphable on onms are:

• cacheHttpInKb
• cacheHttpOutKb
• cacheServerOutKb
• cacheServerInKb
• cacheSysVMsize
• cacheUptime
• cacheMemUsage
• cacheCpuUsage
• cacheNumObjCount
• cacheClients
• cacheCurrentSwapSize
• cacheDnsReplies
• cacheDnsRequests
• cacheHttpHits
• cacheHttpErrors
• cacheFqdnEntries
• cacheFqdnRequests
• cacheFqdnHits
• cacheSysStorage
• cacheSysPageFaults
• cacheProtoClientHttpRequests

Posted by kahjin at 03:37 PM | Permalink | Comments (6)

August 03, 2004

The Cubicle

I came in this morning, and found nice 'lil calculator and a Lipton mug on my desk... hmm that can't be good :P

I'd never thought this would happen, but I fell for a prank my colleagues pulled on me over lunch, which gave me a freakkin' heart relapse, those bastiches! You guys should really consider a career option in... local theatre for a start :P Regardless, I still feel like an idiot :D

Posted by kahjin at 03:06 PM | Permalink | Comments (1)

if there is a god

be aware

August 27, 2004

OpenNMS .III.

August 03, 2004

The Cubicle