ELK: Connecting to ElasticSearch with a Go client

ElasticSearch very often serves as a repository for monitoring, logging, and business data.  As such, integrations with external system are a requirement.

The Go programming language with its convenient deployment binary and rich set of packages can easily serve as a bridge between these systems and the ElasticSearch server.

We will use the olivere/elastic package for this purpose, it is well maintained and has support for both ElasticSearch 5.x and 2.x depending on your import statement.  In this article, we will be hitting an ElasticSearch 2.x backend.

Continue reading “ELK: Connecting to ElasticSearch with a Go client”

ELK: Using Ruby in Logstash filters

elastic-logstash-fwLogstash has a rich set of filters, and you can even write your own, but often this is not necessary since there is a out-of-the-box filter that allows you to embed Ruby code directly in the configuration file.

Using logstash-filter-ruby, you can use all the power of Ruby string manipulation to parse an exotic regular expression, an incomplete date format, write to a file, or even make a web service call.

Continue reading “ELK: Using Ruby in Logstash filters”

Zabbix: Accessing Zabbix using the py-zabbix Python module

The open-source Zabbix monitoring solution has a REST API that provides the ability for deep integrations with your existing monitoring, logging, and alerting systems.

This fosters development of community-driven modules like the py-zabbix Python module, which is an easy way to automate Zabbix as well as send/retrieve metrics.

Continue reading “Zabbix: Accessing Zabbix using the py-zabbix Python module”

Jenkins: Setting up a continuous integration server on Ubuntu

Jenkins is the open-source automation server that is critical in building a continuous integration and delivery pipeline.  It is extensible and has a wealth of plugins that  integrate with numerous enterprise systems.

Here are the detailed steps for installing a Jenkins server on Ubuntu.

Continue reading “Jenkins: Setting up a continuous integration server on Ubuntu”

Grafana: Connecting to a Zabbix datasource

Zabbix is an open-source monitoring solution that provides metrics collection, dynamic indexes, alerting, dashboards, and an API for external integration.  But graphing is arguably one Zabbix’s weak points; it still builds static images while other enterprise and consumer applications have set end users’ expectations for graph visualization and interactivity very high.

Luckily, the Zabbix plugin for Grafana can put a facelift on the valuable data stored in Zabbix.  With this new data source, your end users can get the beautiful dashboard view they expect from a modern application.

Continue reading “Grafana: Connecting to a Zabbix datasource”

Zabbix: Alert to PagerDuty using Zabbix3

Having Zabbix send alert mails directly to user groups is typically outgrown as the system matures and the number of alerts increase, new lines of business and engineering groups are on-boarded, and on-call scheduling is implemented.

If you already use PagerDuty for on-call scheduling, then it makes perfect sense to have Zabbix create incidents in PagerDuty. While it is possible to use standard email to perform some level of integration,  the native library is the tightest integration you will find and supports multiple pager duty services.

The agent built by PagerDuty is especially well done, using their API to automatically create PagerDuty incidents as well as automatically mark them resolved if the trigger is only ephemeral (e.g. a temporary cpu spike).

Continue reading “Zabbix: Alert to PagerDuty using Zabbix3”