The Spring framework provides a proven and well documented model for the development of custom projects and services. The Spring Boot project takes an opinionated view of building production Spring applications, which favors convention over configuration. In this article we will explore how to configure a Spring Boot project to use the Simple Logging Facade … Spring: Spring Boot with SLF4J/Logback sending to syslog
Docker is a container platform that streamlines software delivery and provides isolation, scalability, and efficiency with less overhead than OS level virtualization. These instructions are taken directly from the official Docker for Ubuntu page, but I wanted to reiterate those tasks essential for installing the Docker Community Edition on Ubuntu 14.04 and 16.04.
Once you have a Squid proxy setup as described in my article here, the next challenge is configuring your Ubuntu servers so that they use this proxy by default instead of attempting direct internet connections. There are several entities we want using Squid by default: apt package manager, interactive consoles and wget/curl, and Java applications.
Having your production servers go through a proxy like Squid for internet access can be an architectural best practice that provides network security as well as caching efficiencies. For further security, denying access to all requests but an explicit whitelist of domains provides auditable control.
HAProxy is a high performance TCP/HTTP (Level 4 and Level 7) load balancer and reverse proxy. A common pattern is allowing HAProxy to be the fronting SSL-termination point, and then HAProxy determines which pooled backend server serves the request.
Nginx is a popular reverse proxy and load balancer that focuses on level 7 (application) traffic. A common pattern is allowing Nginx to be the fronting SSL-termination point, and then Nginx determines which pooled backend server is best available to serve the request.
Some web applications leave authentication as an orthogonal concern to the application – not including any kind of login functionality and instead leaving authentication as an operational concern. When this happens, a reverse proxy that has an LDAP integration can act as an architectural sentry in front of the web application and also fulfills the … Apache2: Enable LDAP authentication and SSL termination for Ubuntu
There are numerous articles I’ve written where a certificate is a prerequisite for deploying a piece of infrastructure. Here are the quick steps for installing a simple self-signed certificate on an Ubuntu server. If you instead need to create a certificate with SAN (Subject Alternative Name) support, read my article here. Some of you will … Ubuntu: Creating a self-signed certificate using OpenSSL on Ubuntu
Jenkins is the open-source automation server that is critical in building a continuous integration and delivery pipeline. It is extensible and has a wealth of plugins that integrate with numerous enterprise systems. Here are the detailed steps for installing a Jenkins server on Ubuntu.
Especially in enterprise application development, there can be 3rd party dependencies that are not available in public Maven repositories. These may be internal, business specific libraries or licensed libraries that have limitations on usage. When this is the case, you can either publish to a private Maven repository that controls authorization or you can put … Maven: Installing a 3rd party jar to a local or remote repository
An essential part of the standard build process for Java applications is having a set of repositories where project artifacts are stored. Artifact curation provides the ability to manage dependencies, quickly rollback releases, support compatibility of downstream projects, do QA promotion from test to production, support a continuous build pipeline, and provides auditability. JFrog puts … Maven: Installing a private Maven repository on Ubuntu using Artifactory
Java JMX (Java Management Extensions) is a standardized way of monitoring Java based applications. The managed resources (MBeans) are defined and exposed by the JVM, application server, and application – and offer a view into these layers that can provide invaluable monitoring data. But in order to report back the JMX data you must know … Monitoring: Java JMX exploration from the console using jmxterm
strace is a handy utility for tracing system, file, and network calls on a Linux system. It can produce trace output for either an already running process, or it can create a new process. Some of the most common troubleshooting scenarios are needing to isolate either the network or file system activity of a process. … Ubuntu: Using strace to get a view into file and network activity of a process
tcpdump comes standard on Ubuntu servers and is an invaluable tool in determining traffic coming in and out of a host. As network infrastructures have become more complex and security conscious, validating network flow from client hosts through potentially multiple proxies and ultimately to a destination host and port has become more important than ever. … Ubuntu: Using tcpdump for analysis of network traffic and port usage
Nginx is a powerful application level proxy server. Whether for troubleshooting or analysis, enabling log levels and custom formats for the access/error logs is a common requirement. Error Logs By default, only messages in the error category are logged. If you want to enable more details, then modify nginx.conf like: error_log file [level] Enabling debug … Nginx: Custom access log format and error levels
Enabling verbose logs for an AppDynamics machine or database agents can be invaluable for troubleshooting connectivity or network issues. Luckily, this is easily done by editing the conf/logging/log4j.xml file. By default, only the error level messages are sent to the logs: <root> <priority value=”error”/> <appender-ref ref=”FileAppender”/> </root> But you can modify this so that debug … AppDynamics: Enabling verbose debug logs for Agents
Update Jan 2023: I have written an updated article on doing a sysupgrade from older OpenWrt versions to 19.x. If you want to do an in-place sysupgrade of a 19.x or earlier snapshot, use the instructions from the newer article, but selecting a snapshot image instead of a stable release image. Although stables releases of … OpenWrt: Upgrading OpenWrt to the latest snapshot build
The PingFederate server provides best-in-class Identity Management and SSO. However, due to US laws governing export of cryptography, the default SSL protocols and cipher suites need to be configured to harden the solution. Below are the steps involved with making these post-installation changes.
As an exploration of AppDynamics’ APM functionality, you may find it useful to deploy a sample application that can quickly return back useful data. The Java Spring PetClinic connecting back to a PostgreSQL database provides a simple code base that exercises both database and application monitoring. In a previous article, I went over the detailed … AppDynamics: Java Spring PetClinic and PostgreSQL configured for monitoring
Update Feb 2023: enumerating the secure protocols and ciphers of a remote site can be done more efficiently by nmap, as described in my other article here. While enabling HTTPS is a important step in securing your web application, it is critical that you take steps to disable legacy protocols and low strength ciphers that … OpenSSL: Using OpenSSL to enumerate protocols and ciphers in use by web applications
Selenium is an open-source solution for automating the browser allowing you to run continuous integration tests, validate performance and scalability, and perform regression testing of web applications. This kind of automated testing is useful not only from desktop systems, but also from server machines where you may want to monitor availability or correctness of returned … Selenium: Running headless automated tests on Ubuntu
If you have worked on deploying packages via apt-get, you are probably familiar with a couple of forms of interruption during the package installation and upgrade process. The first is the text menu shown during package upgrades that informs you that a new configuration file is available and asks if you want to keep your … Ubuntu: Silent package installation and debconf
As an exploration of AppDynamics’ APM functionality, you may find it useful to deploy a sample application that can quickly return back useful data. The Java Spring PetClinic connecting back to a MySQL database provides a simple code base that exercises both database and application monitoring. We’ll deploy the Java Spring PetClinic unto Tomcat running … AppDynamics: Java Spring PetClinic and MySQL configured for monitoring
The AppDynamics Machine Agent is used not only to report back on basic hardware metrics (cpu/memory/disk/network), but also as the hook for custom plugins that can report back on any number of applications including: .NET, Apache, AWS, MongoDB, Cassandra, and many others. In this article, I’ll go over the details to install the Machine Agent … AppDynamics: Installing a Machine Agent on Ubuntu 14.04
The ElasticSearch stack (ELK) is popular open-source solution that serves as both repository and search interface for a wide range of applications including: log aggregation and analysis, analytics store, search engine, and document processing. Its standard web front-end, Kibana, is a great product for data exploration and dashboards. However, if you have multiple data sources … Grafana: Connecting to an ElasticSearch datasource
Zabbix is an open-source monitoring solution that provides metrics collection, dynamic indexes, alerting, dashboards, and an API for external integration. But graphing is arguably one Zabbix’s weak points; it still builds static images while other enterprise and consumer applications have set end users’ expectations for graph visualization and interactivity very high. Luckily, the Zabbix plugin … Grafana: Connecting to a Zabbix datasource
Grafana is an open-source visualization suite that is able to generate graphs and dashboards, in addition to alerting. It is designed to retrieve data from various backends including: Graphite, ElasticSearch, Prometheus, and Zabbix. This article will lead you through an installation of the latest stable version on Ubuntu 14.04.
Until Zabbix3, trend data was not available via the Zabbix API. This meant that you could retrieve the raw values of a key over time, but not the aggregated historical trends of that value (e.g. CPU average over 5 minute intervals). The only way to monitor trends was to look at the visual graph generated … Zabbix: Enabling API fetch of Trend data in Zabbix2
Having Zabbix send alert mails directly to user groups is typically outgrown as the system matures and the number of alerts increase, new lines of business and engineering groups are on-boarded, and on-call scheduling is implemented. If you already use PagerDuty for on-call scheduling, then it makes perfect sense to have Zabbix create incidents in … Zabbix: Alert to PagerDuty using Zabbix3
As part of normal long-term operations, the number of kernel images on your system will accumulate and take up disk space. This issue with space will be even more pronounced if /boot is mounted to its own smaller partition. With Ubuntu 16.04, ‘apt autoremove –purge’ and configuration of the unattended upgrades can ensure that old … Unbutu: Removing unused kernel images and headers
Oracle VirtualBox as a virtualization engine paired with Vagrant provides a cross-platform virtualization-agnostic workflow for Linux, Windows, and MacOS. It is light enough to allow a developer to setup, test, and tear down virtual infrastructure as part of a unit test. You may find yourself in a position where you have built a VM in … VMware: Exporting from Oracle VirtualBox/Vagrant to vCloud Director
The ELK stack (ElasticSearch-Logstash-Kibana), is a horizontally scalable solution with multiple tiers and points of extension and scalability. Because so many companies have adopted the platform and tuned it for their specific use cases, it would be impossible to enumerate all the novel ways in which scalability and availability had been enhanced by load balancers, … ELK: Architectural points of extension and scalability for the ELK stack
The heart of the ELK stack is Elasticsearch. In order to provide high availability and scalability, it needs to be deployed as a cluster with master and data nodes. The Elasticsearch cluster is responsible for both indexing incoming data as well as searches against that indexed data. Resources As described in the documentation, if there … ELK: Scaling an ElasticSearch Cluster
The most varied point in an ELK (Elasticsearch-Logstash-Kibana) stack is the mechanism by which custom events and logs will get sent to Logstash for processing. Companies running Java applications with logging sent to log4j or SLF4J/Logback will have local log files that need to be tailed. Applications running in containers may send everything to stdout/stderr, … ELK: Feeding the logging pipeline
If you are running an Ubuntu server for any extended period of time, security issues will arise that affect the kernel, distribution, or packages installed on that host. While there is a minimal stability risk with automatically applying security fixes, I feel those are dwarfed by the risks of running hosts that have known security … Ubuntu: Unattended Upgrades for security patches
Although the ELK stack has rich support for clustering, clustering is not supported over WAN connections due to Elasticsearch being sensitive to latency. There are also practical concerns of network throughput given how much data some installations index on an hourly basis. So as nice as it would be to have a unified, eventually consistent … ELK: Federated Search with a Tribe node
Kibana is the end user web application that allows us to query Elasticsearch data and create dashboards that can be used for analysis and decision making. Although Kibana can be pointed to any of the nodes in your Elasticsearch cluster, the best way to distribute requests across the nodes is to use a non-master, non-data … ELK: Pointing Kibana to a Client Node
SaltStack has the ability to create custom states, grains, and external pillars. There is a long list of standard external pillars ranging from those which read from local JSON files, to those that pull from EC2, MongoDB, etcd, and MySQL. In this article, we will use Apache ZooKeeper as the storage facility for our SaltStack … SaltStack: Creating a ZooKeeper External Pillar using Python
It may be hard to imagine on the development side, but there are instances where a deployed host is not accessible from the Salt Master in a production environment. This forces a bit of creativity if you have a set of standard formulas you need to apply to the host. For instance, imagine a host … SaltStack: Running a masterless minion on Ubuntu
Python is a language whose advantages are well documented, and the fact that it has become ubiquitous on most Linux distributions makes it well suited for quick scripting duties. In this article I’ll go through an example of using Python to read entries from a JSON file, and from each of those entries create a … Python: Using Python, JSON, and Jinja2 to construct a set of Logstash filters