Ubuntu: A centralized apt package cache using Apt-Cacher-NG

ubuntuIt is common in secure production datacenters for internal hosts to be forced to go through a reverse proxy for public internet access.  The same concept can be applied to apt package management, where setting up a centralized package proxy enables caching as well as security controls.

In a datacenter where you could have hundreds of host instances all needing the same package/kernel/security patch, having a cache of packages inside your network can save a significant amount of network bandwidth and operator time.

In this article, we’ll go through installation and configuration of Apt-Cacher-NG, a specialized reverse proxy for Debian based distributions that does whitelisting of repositories, precaching, and remapping to support caching for SSL repositories.

Continue reading “Ubuntu: A centralized apt package cache using Apt-Cacher-NG”

Ubuntu: A centralized apt package cache using squid-deb-proxy

ubuntuIt is common in secure production datacenters for internal hosts to be forced to go through a reverse proxy (e.g. Squid) for public internet access.  The same concept can be applied to apt package management, where setting up a centralized package proxy enables caching as well as security controls.

In a datacenter where you could have hundreds of host instances all needing the same package/kernel/security patch, having a cache of packages inside your network can save a significant amount of network bandwidth and operator time.

And just like an internet proxy that whitelists only specific domains, a package proxy can have a whitelist of apt repositories, as well as a blacklist of specific packages.

In this article we’ll go through installation and configuration of squid-deb-proxy, which is just a packaging of Squid3 with specific tunings for package caching.  Since most Security and Operations teams are familiar with Squid already, this makes it easier to get deployment approval versus other package caching solutions.

Continue reading “Ubuntu: A centralized apt package cache using squid-deb-proxy”

SaltStack: Installing an older Salt Master or Minion for compatibility

If your Salt Minion version is too far removed from the Salt Master version, you may find yourself with unexplained errors.

This problem can be faced when the OS template you are deploying was packaged years earlier with an older Salt minion while the Salt Master has been kept up to date.

But it can also happen with a relatively recent version Master like 2016.11, if you use the latest 2017.7 Minion which has major changes in the fileclient.

In this article I will show you how to use apt-get to install an earlier version of the Salt Master or Salt Minion.

Continue reading “SaltStack: Installing an older Salt Master or Minion for compatibility”

GoLang: Glide for Go language package management

Downloading 3rd party packages from github is made very simple in the Go language with the import statement. But similar to other languages, the complexity of versions and inter-dependencies begs the use of a package manager for any projects that are non-trivial (think npm for Javascript, pip for Python, Maven for Java, etc.).

Glide is a package manager for the Go programming language that can greatly ease the chore of package management by supporting package independence between projects, versioning, and non-master branches.

Continue reading “GoLang: Glide for Go language package management”

GoLang: Vendor directory for github branches other than master

Using 3rd party packages from github is made very simple in the Go language with the import statement.  But one problem is that “go get” will always pull the HEAD of the master branch and there is no way to explicitly specify another branch.

The ultimate answer would be to use a package dependency manager like Glide, which I describe in this article.  But if you cannot introduce Glide into your workflow yet then manually populating the vendor directory (enabled by default since 1.6) is a viable alternative.

Continue reading “GoLang: Vendor directory for github branches other than master”

Ubuntu: Silent package installation and debconf

If you have worked on deploying packages via apt-get, you are probably familiar with a couple of forms of interruption during the package installation and upgrade process.

The first is the text menu shown during package upgrades that informs you that a new configuration file is available and asks if you want to keep your current one, use the new one from the package maintainer, or show the difference.

The second is the occasional ASCII dialog that interrupts the install/upgrade and ask for essential information before moving forward.  The screenshot below is the dialog you get when installing MySQL or MariaDB, asking to set the initial root password for the database.

The problem, in this age of cloud scale, is that you often need completely silent installations and upgrades that can be pushed out via Configuration Management.  Even if this is a build for an immutable image, you would prefer a completely automated construction process instead of manual intervention each time you build an image.

Continue reading “Ubuntu: Silent package installation and debconf”

Node.js: Packaging modules for offline deployment using npm-bundle

nodejs-logoIn a production environment, it is common to have restricted internet access on the production deployment hosts.  This means that using the standard ‘npm install’ and pulling modules from the registry.npmjs.org repository is not an option.

Given the breadth of the dependency graph required for most modules, this packaging is something you want automated without needing to modify the package.json file by hand.

After various failed attempts at: using npmbox, scripts wrapping up ‘npm pack’, and archiving the entire node_modules directory – the npm-bundle module finally provided a proper solution.

Continue reading “Node.js: Packaging modules for offline deployment using npm-bundle”