OpenNMS ships with the thresholds for some events already defined. For example, there is a memory threshold defined as:

<group name="netsnmp-memory-nonlinux" rrdRepository="/opt/opennms/share/rrd/snmp/">
        <expression type="low" expression="memAvailReal / memTotalReal * 100.0" ds-type="node" ds-label="" value="5.0" rearm="10.0" trigger="2"/>
</group>

ie. if free memory drops below 5% then an event will be created. The alert will be cancelled automatically if free memory subsequently rises above 10%

I wanted to configure some specific nodes with a different threshold, eg. generate an event when free memory drops below 2.5%.

Here's what I did.

Add new Surveillance Category

Add nodes to new Surveillance Category

Add a new group to thresholds.xml:

    <group name="netsnmp-memory-linux-2.5" rrdRepository="/opt/opennms/share/rrd/snmp/">
        <expression type="low" ds-type="node" value="2.5" rearm="5.0"
            trigger="2" filterOperator="or" expression="(memAvailReal + memCached) / memTotalReal * 100.0"/>
    </group>

Update threshd-configuration.xml to modify the existing netsnmp-memory-linux package and add a new netsnmp-memory-linux-2.5 package:

    <package name="netsnmp-memory-linux">
        <filter>IPADDR != '0.0.0.0' &amp; nodeSysOID == '.1.3.6.1.4.1.8072.3.2.10' &amp; ! ( catincNetSNMP-Mem-2_5 )</filter>
        <include-range begin="1.1.1.1" end="254.254.254.254"/>
        <include-range begin="::1" end="ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff" />
        <service name="SNMP" interval="300000" user-defined="false" status="on">
            <parameter key="thresholding-group" value="netsnmp-memory-linux"/>
        </service>
    </package>

    <package name="netsnmp-memory-linux-2.5">
        <filter>IPADDR != '0.0.0.0' &amp; nodeSysOID == '.1.3.6.1.4.1.8072.3.2.10' &amp; catincNetSNMP-Mem-2_5</filter>
        <include-range begin="1.1.1.1" end="254.254.254.254"/>
        <include-range begin="::1" end="ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff" />
        <service name="SNMP" interval="300000" user-defined="false" status="on">
            <parameter key="thresholding-group" value="netsnmp-memory-linux-2.5"/>
        </service>
    </package>

The tricky bit is the "catincNetSNMP-Mem-2_5". This is a function "catinc" which matches all nodes in the specified category ie. "NetSNMP-Mem-2_5" in this example. The first use of it is in the netsnmp-memory-linux category to exclude nodes in the NetSNMP-Mem-2_5 category. The second use is to include nodes in the NetSNMP-Mem-2_5 category.