Zabbix: LLD low-level discovery returning multiple values

Zabbix low-level discovery (LLD) provides a way to create an array of related items, triggers, or graphs without needing to know the exact number of entities up front.

The easiest way to populate the keys of a discovery item is to add a “UserParameter” in zabbix_agentd.conf, and then the Zabbix agent will  invokes a script which returns the set of keys.

But the keys are only the first part of a real solution, because what you really want to send back are the values associated with those keys.  For example, if you are monitoring a database, you don’t want to just send the list of tables available, you may want to send back each table name and then its row count and size on disk.

Unfortunately Zabbix does not support sending back multiple values [1,2,3,4].  There are various workarounds such as using one UserParameter for the discovery key and another with a UserParameter=key[*] to fetch each row of data, or using vfs.file.regexp to parse values that have been written to a file.

But I think the cleanest solution, and one that requires the minimal number of spawned processes on the agent host is to invoke zabbix_sender from inside the script to send back all the values you want to populate.

Continue reading “Zabbix: LLD low-level discovery returning multiple values”

SaltStack: Combine multiple pillar files under a single key

saltstack_logo-thumbnailAn issue that keeps coming up on the mailing lists as well as Stackoverflow[1,2] is how to merge multiple pillar files for use with a single state.  The problem is that pillars using the same key overwrite each other, and there is no easy way to express the desire to merge instead.

There are various workarounds, but all of these expect the human operator to know about these disparate sources and manually mend them together with a unifying sls file (using includes or anchors/references).

The state and pillar files in this article can be downloaded from my github page.

Continue reading “SaltStack: Combine multiple pillar files under a single key”