This article has been updated in October 2018 and is now tested for HAProxy 1.8.14. The reload functionality in HAProxy till now has always been “not perfect but good enough”, perhaps dropping a few connections under heavy load but within parameters everyone was willing to accept. And because of the potential impact, a reload was … HAProxy: Zero downtime reloads with HAProxy 1.8 on Ubuntu 16.04 with Systemd
This article has been updated in October 2018 and is now tested for HAProxy 1.8.14. The reload functionality in HAProxy till now has always been “not perfect but good enough”, perhaps dropping a few connections under heavy load but within parameters everyone was willing to accept. And because of the potential impact, a reload was … HAProxy: Zero downtime reloads with HAProxy 1.8 on Ubuntu 14.04
If your Salt Minion version is too far removed from the Salt Master version, you may find yourself with unexplained errors. This problem can be faced when the OS template you are deploying was packaged years earlier with an older Salt minion while the Salt Master has been kept up to date. But it can … SaltStack: Installing an older Salt Master or Minion for compatibility
Many developers like to use Vagrant from HashiCorp to standardize the workflow of virtual machines: creation, running, destroying, taking snapshots, etc.. Usually Vagrant is used for Linux hosts, but it also works with Windows as long as you prepare the template properly. In a previous article I went over the detailed steps to create a template image for … Windows: Windows 2012 Sysprep for Vagrant readiness
In the world of Linux containers where deployment takes on the order of seconds, even the best-case scenario for spinning up a new Windows host can seem like an eternity. Clearly, you don’t want to wait for the entire Windows install process each time you bring up a Windows guest OS. Even automated, this would … Ubuntu: Standing up a Windows 2012 instance on Ubuntu using Sysprep
Android is one of the leading platforms of the mobile industry. By installing an Android emulator on your Ubuntu desktop, you can bring this power to your desktop. More often than not, an Android emulator is used for custom development of mobile apps, but don’t overlook its utility as a way to access your favorite … Ubuntu: Installing the Genymotion Android emulator
The Tor project is free software that helps protect your privacy by making it difficult for a 3rd party to analyze your network requests or link your traffic back to your network access point. See the Tor overview page for reasons why this may be important to world citizens, corporations, or specific professions. Simplified, this … Ubuntu: Installing Tor on Ubuntu 14.04 and 16.04
SMTP mail relays exposed to the internet typically use a combination of SSL and authenticated SMTP to avoid abuse by malicious actors. This is an excellent choice from a security perspective, but makes smoke testing a bit more complex than just opening telnet.
Ansible is an agentless configuration management tool that helps operations teams manage installation, patching, and command execution across a set of servers. In this article I’ll describe how to deploy the latest release of Ansible using pip on Ubuntu 16.04, and then perform a quick validation against a client.
Ansible is an agentless configuration management tool that helps operations teams manage installation, patching, and command execution across a set of servers. Ansible was started as a Linux only solution, leveraging ssh to provide a management channel to a target server. However, starting at Ansible 1.7, support for Windows hosts was added by using Powershell … Ansible: Managing a Windows host using Ansible
Ansible is an agentless configuration management tool that helps operations teams manage installation, patching, and command execution across a set of servers. In this article I’ll describe how to deploy the latest release of Ansible using pip on Ubuntu 14.04, and then perform a quick validation against a client.
Zabbix low-level discovery (LLD) provides a way to create an array of related items, triggers, or graphs without needing to know the exact number of entities up front. The easiest way to populate the keys of a discovery item is to add a “UserParameter” in zabbix_agentd.conf, and then the Zabbix agent will invokes a script which returns the set … Zabbix: LLD low-level discovery returning multiple values
Syslog is a message logging standard has been around for decades, but has renewed popularity as a method of log capture with the advent of containerization and centralized logging solutions. Enabling an Ubutu 14.04 or 16.04 host to act as a syslog server only takes a few simple steps.
The Docker console commands for listing and viewing containers and images (ps, images, history, inspect) provides a wealth of information, but when you are managing hundreds of containers, a graph view of the container inventory and their dependencies can be critical for operations. Dockviz can help you visualize your containers and images by creating an PNG image … Docker: Visualizing image hierarchy and container dependency using dockviz
Update Oct 2022: This article has now been written for GoLang 1.19 on Ubuntu 22.04. Go has changed the way it handles SIGURG signals, and Systemd services no longer directly forward to syslog. Read my newer article here. The Go language with its simplicity, concurrency support, rich package ecosystem, and ability to compile down to a single … GoLang: Running a Go binary as a systemd service on Ubuntu 16.04
ElasticSearch very often serves as a repository for monitoring, logging, and business data. As such, integrations with external system are a requirement. The Go programming language with its convenient deployment binary and rich set of packages can easily serve as a bridge between these systems and the ElasticSearch server. We will use the olivere/elastic package for this purpose, it is … ELK: Connecting to ElasticSearch with a Go client
The Go language with its simplicity, concurrency support, rich package ecosystem, and ability to compile down to a single binary is an attractive solution for writing services on Ubuntu. However, the Go language does not natively provide a reliable way to daemonize itself. In this article I will describe how to take a couple of simple Go language programs, … GoLang: Running a Go binary as a SysV service on Ubuntu 14.04
The open-source Zabbix monitoring solution has a published, simple binary protocol that allows you to send metrics to the Zabbix server without relying on the Zabbix Agent – which makes it very convenient for integration with other parts of your infrastructure. In this article, I’ll show how to use the go-zabbix package for sending metrics to … Zabbix: Sending Zabbix metrics using a Go client
Downloading 3rd party packages from github is made very simple in the Go language with the import statement. But similar to other languages, the complexity of versions and inter-dependencies begs the use of a package manager for any projects that are non-trivial (think npm for Javascript, pip for Python, Maven for Java, etc.). Glide is a package manager for the Go … GoLang: Glide for Go language package management
The open-source Zabbix monitoring solution has a REST API that provides the ability for deep integrations with your existing monitoring, logging, and alerting systems. This fosters development of community-driven modules like Ryan Day’s zabbix Go language package, which is an easy way to automate Zabbix tasks like creating hosts and manipulating other back end structures. One of the nice things … Zabbix: Zabbix REST API using a Go client
Using 3rd party packages from github is made very simple in the Go language with the import statement. But one problem is that “go get” will always pull the HEAD of the master branch and there is no way to explicitly specify another branch. The ultimate answer would be to use a package dependency manager like Glide, which … GoLang: Vendor directory for github branches other than master
A nice feature of the Go language is the ability to build binaries for multiple platforms directly from a single source system. As an example, even from a development Windows 7 32-bit machine, you can build binaries for both 64 bit Linux and Windows 2012 Servers. Before Go 1.5, you needed a compiler for the target architecture, … GoLang: Cross Compiling for Linux and Windows platforms
The Go programming language has gotten considerable momentum, and the fact that it compiles down to machine code has made it popular in containers like Docker where a single executable binary fits the execution model perfectly. This article will detail installation on Ubuntu 14.04 with the standard hello world validation.
An issue that keeps coming up on the mailing lists as well as Stackoverflow[1,2] is how to merge multiple pillar files for use with a single state. The problem is that pillars using the same key overwrite each other, and there is no easy way to express the desire to merge instead. There are various workarounds, but all … SaltStack: Combine multiple pillar files under a single key
Configuration Management tools like SaltStack are invaluable for managing infrastructure at scale. Even in the growing world of containerization where immutable image deployment is the norm, those images need to be built in a repeatable and auditable fashion. This article will detail installation of the SaltStack master on Ubuntu 14.04, with validation using a single Minion. Note that Minion … SaltStack: Installing a Salt Master on Ubuntu 14.04
It is very common to have Logstash create time-based indexes in ElasticSearch that fit the format, <indexName>-YYYY.MM.DD. This means events submitted with @timestamp for that day all go to the same index. However, if you do not explicitly specify an index template that maps each field to a type, you can end up with unexpected query … ELK: Custom template mappings to force field types
When building complex, real-world Logstash filters, there can be a fair bit of processing logic. There are typically multiple grok patterns as well as fields used as flags for conditional processing. The problem is, these intermediate extracted fields and processing flags are often ephemeral and unnecessary in your ultimate persistent store (e.g. ElasticSearch), but they will be inserted … ELK: metadata fields in Logstash for grok and conditional processing
Logstash provides a powerful mechanism for listening to various input sources, filtering and extracting the fields, and then sending events to a persistence store like ElasticSearch. Installing Logstash on Ubuntu is well documented, so in this article I will focus on Ubuntu specific steps required for Logstash 2.x and 5.x.
Logstash has a rich set of filters, and you can even write your own, but often this is not necessary since there is a out-of-the-box filter that allows you to embed Ruby code directly in the configuration file. Using logstash-filter-ruby, you can use all the power of Ruby string manipulation to parse an exotic regular expression, … ELK: Using Ruby in Logstash filters
The open-source Zabbix monitoring solution has a REST API that provides the ability for deep integrations with your existing monitoring, logging, and alerting systems. This fosters development of community-driven modules like the py-zabbix Python module, which is an easy way to automate Zabbix as well as send/retrieve metrics.
Docker log collection can be done using various methods, one method that is particularly effective is having a dedicated container whose sole purpose is to automatically sense other deployed containers and aggregate their log events. This is the architectural model of logspout, an open-source project that acts as a router for the stdout/stderr logs of other containers. If you do … Docker: logspout for Docker log collection
Log rotation is an essential maintenance task for managed servers. The logrotate package available in the main Ubuntu repository is easily configurable and is invoked by the cron service for automated log retention.
ElastAlert from the Yelp Engineering group provides a very flexible platform for alerting on conditions coming from ElasticSearch. In a previous article I fully describe running interactively on an Ubuntu server, and now I’ll expand on that by running it at system startup using a System-V init script. One of the challenges of getting ElastAlert to run as a … ELK: Running ElastAlert as a service on Ubuntu 14.04
ElasticSearch’s Metricbeat is a lightweight shipper of both system and application metrics that runs as an agent on a client host. That means that along with standard cpu/mem/disk/network metrics, you can also monitor Apache, Docker, Nginx, Redis, etc. as well as create your own collector in the Go language. In this article we will describe installing … ELK: Installing MetricBeat for collecting system and application metrics
ElasticSearch’s commercial X-Pack has alerting functionality based on ElasticSearch conditions, but there is also a strong open-source contender from Yelp’s Engineering group called ElastAlert. ElastAlert offers developers the ultimate control, with the ability to easily create new rules, alerts, and filters using all the power and libraries of Python.
The open-source Zabbix monitoring solution has very lightweight agents that are easy to install on Ubuntu. Although the Ubuntu main repository has a build available, it is older and so we are going to choose to download and install the latest point version in this article. Unfortunately, the repo.zabbix.com cannot be added directly as an Ubuntu … Zabbix: Installing a Zabbix Agent on Ubuntu 14.04
By nature, the amount of data collected in your ElasticSearch instance will continue to grow and at some point you will need to prune or warehouse indexes so that your active collections are prioritized. ElasticDump can assist in moving your indexes either to a distinct ElasticSearch instance that is setup specifically for long term data, or exporting … ELK: ElasticDump and Python to create a data warehouse job
The Curator product from ElasticSearch allows you to apply batch actions to your indexes (close, create, delete, etc.). One specific use case is applying a retention policy to your indexes, deleting any indexes that are older than a certain threshold. Installation Start by installing Curator using apt and pip: $ sudo apt-get install python-pip -y … ELK: Using Curator to manage the size and persistence of your index storage
Although container based engines such as Docker are highly popularized for newer application deployment – there will still be widespread use of OS virtualization engines for years to come. One of the most popular virtualization engines for development purposes is the open-source VirtualBox from Oracle. This article will detail its installation on Ubuntu 14.04.
Building services using Spring Boot gives a development team a jump start on many production concerns, including logging. But unlike a standard deployment where logging to a local file is where the developer’s responsibility typically ends, with Docker we must think about how to log to a public space outside our ephemeral container space. The … Docker: Sending Spring Boot logging to syslog