Docker: logspout for Docker log collection

Docker log collection can be done using various methods, one method that is particularly effective is having a dedicated container whose sole purpose is to automatically sense other deployed containers and aggregate their log events.

This is the architectural model of logspout, an open-source project that acts as a router for the stdout/stderr logs of other containers.

If you do not have docker installed yet, see my article here.  Before moving on, you should be able to run the hello-world container.

Continue reading “Docker: logspout for Docker log collection”

SaltStack: Troubleshooting Basic Network Connectivity of Minion on Ubuntu

saltstack_logo-thumbnailWhen troubleshooting basic connectivity from your SaltStack minions to your Salt master, the first thing to remember is the basic flow – the minions initiate the connection to port 4505/4506 on the Salt master.

With this in mind, if you have modified /etc/salt/minion so that the master is explicitly set and logs are set to debug levels as shown below:

master: mysaltmaster
log_level_logfile: debug

And the minion key is still not showing up on the Salt master list (salt-key -L), and the minion log file (/var/log/salt/minion) is not providing any hints, you should try a basic network connectivity test using netcat.  From the console of the Salt minion:

Continue reading “SaltStack: Troubleshooting Basic Network Connectivity of Minion on Ubuntu”

Logstash: Testing Logstash grok patterns online

elastic-logstash-fwIn my previous posts, I have shown how to test grok patterns locally using Ruby on Linux and Windows.  This works well when your VM do not have full internet access, or only have console access, or any reason that you want to test it locally.

If you have access to a graphical web browser and the log file, there is a nice online grok constructor here and here. and by simply entering a sampling of the log lines and a grok pattern, you can verify that all the lines are parsed correctly.

Here is a small example to start you off:

Continue reading “Logstash: Testing Logstash grok patterns online”