SaltStack: Combine multiple pillar files under a single key

saltstack_logo-thumbnailAn issue that keeps coming up on the mailing lists as well as Stackoverflow[1,2] is how to merge multiple pillar files for use with a single state.  The problem is that pillars using the same key overwrite each other, and there is no easy way to express the desire to merge instead.

There are various workarounds, but all of these expect the human operator to know about these disparate sources and manually mend them together with a unifying sls file (using includes or anchors/references).

The state and pillar files in this article can be downloaded from my github page.

Continue reading “SaltStack: Combine multiple pillar files under a single key”

SaltStack: Installing a Salt Master on Ubuntu 14.04

saltstack_logo-thumbnailConfiguration Management tools like SaltStack are invaluable for managing infrastructure at scale.  Even in the growing world of containerization where immutable image deployment is the norm, those images need to be built in a repeatable and auditable fashion.

This article will detail installation of the SaltStack master on Ubuntu 14.04, with validation using a single Minion.  Note that Minion installation is not mandatory if using Salt SSH.

Continue reading “SaltStack: Installing a Salt Master on Ubuntu 14.04”

SaltStack: Creating a ZooKeeper External Pillar using Python

saltstack_logo-thumbnailSaltStack has the ability to create custom states, grains, and external pillars.  There is a long list of standard external pillars ranging from those which read from local JSON files, to those that pull from EC2, MongoDB, etcd, and MySQL.

In this article, we will use Apache ZooKeeper as the storage facility for our SaltStack pillar data.  ZooKeeper is used extensively for configuration management and synchronization of distributed applications, so it makes sense that it could serve as a central repository for pillar data.

Continue reading “SaltStack: Creating a ZooKeeper External Pillar using Python”

SaltStack: Running a masterless minion on Ubuntu

saltstack_logo-thumbnailIt may be hard to imagine on the development side, but there are instances where a deployed host is not accessible from the Salt Master in a production environment.  This forces a bit of creativity if you have a set of standard formulas you need to apply to the host.

For instance, imagine a host sitting in a highly restricted DMZ network. Even with the advent of Salt SSH for minionless administration, SSH access may only be opened from a jumpbox and not the Salt Master itself.  In cases like this, a Masterless Minion is a viable alternative.

Continue reading “SaltStack: Running a masterless minion on Ubuntu”

SaltStack: Keeping Salt Pillar data encrypted using GPG

saltstack_logo-thumbnailWhen automating software and infrastructure, it is not uncommon to need to supply a user id and password for installation or other operations.  While it is certainly possible to pass these plaintext credentials directly in the state, this is not best practice.

# not best practice!!!

testdb_user:
  mysql_user.present:
    - name: frank
    - password: "test3rdb"
    - host: localhost

There are several issues with this approach.

Continue reading “SaltStack: Keeping Salt Pillar data encrypted using GPG”

SaltStack: Setting a jinja2 variable from an inner block scope

saltstack_logo-thumbnailWhen using jinja2 for SaltStack formulas you may be surprised to find that your global scoped variables do not have ability to be modified inside a loop.  Although this is counter intuitive given the scope behavior of most scripting languages it is unfortunately the case that a jinja2 globally scoped variable cannot be modified from an inner scope.

As an example of a solution that will not work, let’s say you have a global flag ‘foundUser’ set to False, then want to iterate through a group of users, and if a condition is met inside the loop, then ‘foundUser’ would be set to True.

Continue reading “SaltStack: Setting a jinja2 variable from an inner block scope”

SaltStack: Troubleshooting Basic Network Connectivity of Minion on Ubuntu

saltstack_logo-thumbnailWhen troubleshooting basic connectivity from your SaltStack minions to your Salt master, the first thing to remember is the basic flow – the minions initiate the connection to port 4505/4506 on the Salt master.

With this in mind, if you have modified /etc/salt/minion so that the master is explicitly set and logs are set to debug levels as shown below:

master: mysaltmaster
log_level_logfile: debug

And the minion key is still not showing up on the Salt master list (salt-key -L), and the minion log file (/var/log/salt/minion) is not providing any hints, you should try a basic network connectivity test using netcat.  From the console of the Salt minion:

Continue reading “SaltStack: Troubleshooting Basic Network Connectivity of Minion on Ubuntu”

SaltStack: Validating States of Minion without Execution

saltstack_logo-thumbnailBefore running state.apply against a minion, especially in a production environment, a good sanity test can be to list the states that will be executed without actually running those states.

This can be done by adding tests=True to the end of the state command. For example, to check all the states that will be applied to a minion:

salt 'myminion' state.apply tests=True

Or to check which states would be run for the apache formula:

salt 'myminion' state.sls apache tests=True