In a production environment, it is common to have restricted internet access on production hosts. This means that using the standard ‘gem install’ and pulling gems from the rubygems.org repository directly on that host is not an option. What is needed is a way to use an internet-connected development host that matches the target production … Ruby: Copying gems to hosts with limited internet access
Although server hosts typically have no graphical desktop and only serve console-based clients, these machines still have the ability to serve a GUI display screen to a remote desktop if necessary. The X11 protocol makes it possible to send the graphical display to a remote graphical desktop. Beyond the ability to run GUI utilities on … Ubuntu: X11 forwarding to view GUI applications running on server hosts
If you are using tar to archive log files and find yourself needing to exclude certain extensions or older log files, tar can easily handle those type of situations. For example, to create a tar that excludes any rolled over logs that are zipped, use an ‘exclude’ like below: tar cvf mylogarchive.tar –exclude=*.gz –exclude=*.tgz * … Linux: Excluding files based on extension and age with tar
If you are running KVM on a console-only host OS, you can still do a full Ubuntu installation on a KVM guest VM by using serial port output and the Ubuntu Server text-based installer. Once deployed, this guest VM can be accessed by using ssh or ‘virsh console’, without the need for a graphics display. … KVM: Creating an Ubuntu VM with console-only access
Although there are utilities such as dpkg-deb for managing .deb packages, they can also be manipulated by the standard set of archival utilities: tar, ar, and gzip. This article will lead you through extracting the contents of a .deb file, making modifications to the installation scripts and default configuration files, then repackaging.
If you are using zip and find yourself needing to exclude a directory (.git, build, etc), the “-x” exclude switch can provide that functionality. Take the following directory structure: $ find . . ./two.txt ./skipme ./skipme/three.txt ./one.txt You can exclude the entire ‘skipme’ folder and everything in it with: zip -r myzip.zip * -x skipme/*
If you need KVM to bring up a full guest OS as quickly as possible, consider preparing a base disk image that can be used as the basis for any number of linked clones by using a backing file. This article is related to a previous article, “Cloning and fixing an Ubuntu host using KVM“. … KVM: Implementing linked clones with a backing file
Whether you need to flash the BIOS or check hardware compatibility for a virtualization engine, it is often necessary to gather details on your current hardware, BIOS, and CPU/DRAM feature set. In this article I’ll provide a starting list of commands you can use to gather this information.
If you are using the direct output of a command to feed a while loop in BASH, you may still want to take user input inside that loop. For example, if you need the user to confirm each line by pressing <ENTER>. Take a simple example like this: grep one test.txt | while read -r … Bash: Reading input from the console while looping over output of command
Update Nov 2021: I have written a newer article that deploys ESXi 7.0U1. If you are running KVM on an Ubuntu server, you already have an excellent Type 1 virtualization engine. Luckily, if you need to test something specific to VMware you can always run ESXi 6.7 nested inside a KVM virtual machine. In this … KVM: Deploying a nested version of VMware ESXi 6.7 inside KVM
Usually the purpose of cloning a KVM guest OS is to have it use the same binaries and configuration as the parent, but allow it to run as an independent entity. When first cloned, the whole guest OS is cloned including host name, network settings, etc. and for those reasons you cannot run the parent … KVM: Cloning and fixing Ubuntu host using KVM
There are multiple approaches to allowing a process to run as a non-root user but still provide access to privileged ports (<1024). There are applications like Apache that handle this by starting the master process as root, and then worker processes as a less privileged user. Another way is setting the privilege on a binary … iptables: Running service as non-root, iptables to forward from privileged port
In previous articles I have described how to use a Squid proxy as a bridge to outside services such as http and ftp. Having hosts in your private networks use a single access point to services in the outside world institutes the type of control often required in production data centers. Having a single host … Ubuntu: Using iptables to forward tcp and udp requests
If you see the message, “There was an error while downloading the metadata for this box”, with a 404 not found return message when doing a box update – make sure to check the URL listed in the Vagrant “metadata_url” files. For example, the “atlas.hashicorp.com” host has been deprecated in favor of “vagrantcloud.com” for some … Vagrant: Fixing “error while downloading the metadata for this box”
By default, KVM will use an older SeaBIOS x86 firmware for your virtual machines. If you want to use a more recent version of seaBIOS or want to drop the older BIOS standard and instead use the newer UUEFI specification (Unified Extensible Firmware Interface), KVM can support that with configuration changes. In this article, I … KVM: Alternate firmware BIOS for KVM
By default, KVM will use the SeaBIOS x86 firmware for virtual machines. This is typically installed as a dependent package which is years old. In this article, I will show how to built the latest SeaBIOS image from source.
OVMF is an open-source project that implements the Unified Extensible Firmware Interface (UEFI) specification. UUEFI is designed to eventually replace the BIOS firmware interface. In this article, I will show how to build the latest OVMF image from source.
Update September 2022: Validated these instructions on Ubuntu 22.04 If you are running an Ubuntu host, you have multiple choices for a virtualization hypervisor. I have written up several articles on using VirtualBox, but now let’s consider a bare metal hypervisor like KVM. KVM is a type 1 hypervisor implemented as a Linux kernel module … KVM: Bare metal virtualization on Ubuntu with KVM
Docker Compose gives us the ability to define and orchestrate multiple containers in order to construct a service. In this article, we will use Docker Compose to create a MongoDB server and then another container that is used exclusively as a MongoDB client. While it is entirely possible to manually create these containers, links, and … Docker: Using docker-compose to link a MongoDB server and client
In order to communicate with MongoDB using its default TCP protocol on port 27017, you will need a MongoDB client. There are many language bindings available, but in this article we’ll focus on the client available from the “mongodb-org-shell” Debian package.
Update Oct 2020: multi-stage builds now provide a standard way to leverage a fat build image, while allowing your published image to remain small. This article is still useful for comparing base image sizes. GoLang applications are a great architectural fit for a Docker container because of their single binary executable. But you need to … Docker: Base image when deploying a GoLang binary in a container
If you have Bash console commands that you type out frequently, consider creating an alias. By simply adding a “.bash_aliases” files in your home directory, you can easily define shortcuts to frequent commands. For example, if you frequently list all the files in a directory reverse sorted by date, you may end up typing “ls … Ubuntu: Creating an alias for common commands under bash
If you have issues with the Nautilus default file browser window not being resizable after it has been maximized, first try holding down “alt” while holding down the mouse button and dragging inside the window. If that does not work, you can use “wmctrl” from the console to fix the issue.
Although in modern architectures you typically see Spring Boot executable jars running as the primary process of a container, there are still many deployment scenarios where running the jar as a service at boot time is required. With Ubuntu 14.04, we can use SysV to run a Spring Boot application at boot time. This will … Java: Spring Boot application as a service using SysV on Ubuntu 14.04
Although in modern architectures you typically see Spring Boot executable jars running as the primary process of a container, there are still many deployment scenarios where running the jar as a service at boot time is required. With Ubuntu 16.04, we can use the built-in systemd supervisor to run a Spring Boot application at boot … Java: Spring Boot application as a service using systemd on Ubuntu 16.04
Logstash provides a powerful mechanism for listening to various input sources, filtering and extracting the fields, and then sending events to a persistence store like ElasticSearch. Installing Logstash on Ubuntu is well documented, so in this article I will focus on Ubuntu specific steps required for Logstash 6.x on Ubuntu 16.04.
There are numerous articles I’ve written where a certificate is a prerequisite for deploying a piece of infrastructure. This article will guide you through generating a self-signed certificate with SAN (Subject Alternative Name) and SAN wildcard entries, replacing the deprecated usage of CN=<FQDN>. In addition to the operational benefits of managing SAN, it is also … Ubuntu: Creating a self-signed SAN certificate using OpenSSL
There are numerous articles I’ve written where a certificate is a prerequisite for deploying a piece of infrastructure. This article will guide you through creating a trusted CA (Certificate Authority), and then using that to sign a server certificate that supports SAN (Subject Alternative Name). Operationally, having your own trusted CA is advantageous over a … Ubuntu: Creating a trusted CA and SAN certificate using OpenSSL
Postfix is a open source mail transfer agent. If communication the upstream mail relay is disrupted, email will build up in the pending mail queue until the root cause is resolved.
It is common in secure production datacenters for internal hosts to be forced to go through a reverse proxy for public internet access. The same concept can be applied to apt package management, where setting up a centralized package proxy enables caching as well as security controls. In a datacenter where you could have hundreds … Ubuntu: A centralized apt package cache using Apt-Cacher-NG
It is common in secure production datacenters for internal hosts to be forced to go through a reverse proxy (e.g. Squid) for public internet access. The same concept can be applied to apt package management, where setting up a centralized package proxy enables caching as well as security controls. In a datacenter where you could … Ubuntu: A centralized apt package cache using squid-deb-proxy
Getting a local project into a public repository on GitHub only takes a few steps. This is well documented on a GitHub help page. In this article, I’ll go through getting a local project called “myproject1” into a public GitHub repository.
Having your production servers go through a proxy like Squid for internet access can be an architectural best practice that provides network security as well as caching efficiencies. In a previous article, I showed how you can enforce whitelists for specific domains when using HTTP/HTTPS. Now let’s do the same thing for FTP connections, proxying … Squid: Enabling whitelisted FTP proxying using Squid
The Meltdown vulnerability affects Intel and some ARM (but not AMD) processor chips and can allow unprivileged access to memory in the kernel and other processes. Canonical has committed to kernel patches to address this issue and they are now available from the both the updates and security official Ubuntu repositories. In this article, I’ll step through patching an … Ubuntu: Testing the official released kernel patches for Meltdown CVE-2017-5754
The Meltdown vulnerability affects Intel and some ARM (but not AMD) processor chips and can allow unprivileged access to memory in the kernel and other processes. Canonical has committed to kernel patches to address this issue by January 9, 2018 and the first candidate kernel patches have now been released for Xenial and Trusty LTS. UPDATE Jan 11 … Ubuntu: Testing the first candidate kernel patches for Meltdown CVE-2017-5754
The Meltdown vulnerability affects Intel and some ARM (but not AMD) processor chips and can allow unprivileged access to memory in the kernel and other processes. Canonical has committed to kernel patches to address this issue by January 9, 2018. A paper coming out of Graz University of Technology in Austria and written by Daniel Gruss, Moritz Lipp, Michael … Ubuntu: Testing the KAISER kernel patch for Meltdown CVE-2017-5754
The Meltdown vulnerability affects Intel and some ARM (but not AMD) processor chips and can allow unprivileged access to memory in the kernel and other processes. Canonical has committed to kernel patches to address this issue by January 9, 2018. If you need to check your system, or perhaps have already patched your systems but want to … Ubuntu: Determine system vulnerability for Meltdown CVE-2017-5754
The Spectre vulnerability affects Intel, AMD, and ARM processor chips (each to various degrees) and can allow unprivileged access to memory in the kernel and other processes. Canonical has committed to kernel patches to address this issue by January 9, 2018. If you need to check your system, or perhaps have already patched your systems … Ubuntu: Determine system vulnerability for Spectre CVE-2017-5715 CVE-2017-5753
This article has been updated in October 2018 and is now tested for HAProxy 1.8.14. The reload functionality in HAProxy till now has always been “not perfect but good enough”, perhaps dropping a few connections under heavy load but within parameters everyone was willing to accept. And because of the potential impact, a reload was … HAProxy: Zero downtime reloads with HAProxy 1.8 on Ubuntu 16.04 with Systemd
This article has been updated in October 2018 and is now tested for HAProxy 1.8.14. The reload functionality in HAProxy till now has always been “not perfect but good enough”, perhaps dropping a few connections under heavy load but within parameters everyone was willing to accept. And because of the potential impact, a reload was … HAProxy: Zero downtime reloads with HAProxy 1.8 on Ubuntu 14.04