If you are using awk with a relatively indexed NF variable (number of fields), and get an error like below, this is because the input being parsed does not have the number of fields expected. awk: run time error: negative field index $-1 The root problem is the awk expression was expecting a certain number … Bash: resolving awk run time error, negative field index
At the Bash command line interface, there is the concept of programmable completion and regular file/directory completion. This means that when you press the <TAB>, the alternatives can be provided by a custom program or the filesystem hierarchy. There is always the chance that a program may introduce undesirable behavior to your auto-completion, and if … Bash: falling back to file autocompletion if errors introduced by program autocompletion
If you find yourself low on disk space, check the size of the Systemd journal logs as a way to easily free up some space. $ sudo journalctl –disk-usage Archived and active journals take up 528.0M in the file system. # reduce size to 300 Mb sudo journalctl –vacuum-size=300M There are automatic jobs that trim … Linux: reducing disk usage of Systemd journal logs
Base64 encoding a string may seem like a straight-forward operation, but there are a couple of gotchas even when dealing with just simple ASCII strings. Avoid embedding a new line character into the encoding If you use the most straight forward method of Base64 encoding shown below, you have to remember that echo by default … Bash: avoiding newline artifacts when Base64 encoding a string
Mike Farah’s yq yaml processor has a a rich set of operators and functions for advanced usage. In this article, I will illustrate how to update deeply nested elements in yaml. This can be done for both known paths as well as arbitrarily deep paths. Sample yaml We will use the following yaml files to … yq: updating deeply nested elements
Mike Farah’s yq yaml processor has a a full-featured validation command that is very detailed in its reporting, but the yaml specification itself is very lenient, which means yq may accept scenarios you did not expect (e.g. an empty file). yq -v file.yaml >/dev/null ; echo “final result = $?” Luckily, the yq tips-and-tricks section … yq: validate yaml syntax
It is relatively easy to experiment with a base LLama2 model on Ubuntu, thanks to llama.cpp written by Georgi Gerganov. The llama.cpp project provides a C++ implementation for running LLama2 models, and works even on systems with only a CPU (although performance would be significantly enhanced if using a CUDA-capable GPU).
If you are using ssh private/public keypair authentication, and get an almost immediate error like below: $ ssh -i id_rsa.pub myuser@a.b.c.d -p 22 Received disconnect from a.b.c.d port 22:2: Too many authentication failures Disconnected from a.b.c.d port 22 Then try again using the ‘IdentitiesOnly‘ option. ssh -o ‘IdentitiesOnly yes’ -i id_rsa.pub myuser@a.b.c.d -p 22 The … Bash: fixing “Too many authentication failures” for ssh with private key authentication
If you want a variable to reference another variable in Bash, that is possible using a concept called indirect reference. Below is a simple example where ‘varname’ contains the name of another variable ‘foo’. # our variable foo=bar # our reference varname=foo # the following syntax will ERROR and DOES NOT work !!! echo “${$varname}” … Bash: indirect reference to evaluate a variable value
If your systemd service is failing with the following error message: XXX.service: Start request repeated too quickly The first thing to do is fix any underlying issues. Use ‘systemctl status <service>’, ‘journalctl -u <service>’, and search any log files produced by the service to understand why the service failed multiple times and exceeded its StartLimitBurst. … Ubuntu: resolving systemd error, “Start request repeated too quickly”
If there is a command that needs to run on startup/reboot, you can use the @reboot directive in cron to schedule it. Here is a simple example of logging the date to a file at startup. (crontab -l 2>/dev/null; echo “@reboot date >> ~/startup.log”) | sort -u | crontab – The sort command avoids duplicate … Bash: schedule a command that will be run at reboot using cron
Getting the sum of a list of numbers is a common scenario on the command line, whether you are checking local filesystem utilization or parsing infrastructure reports for total cpu counts. There are multiple ways this can be done, but one of the simplest is to use awk. Let’s use a a sequence of numbers … Bash: calculate sum from a list of numbers
If ssh private/public keypair authentication is failing, check the logs on the server side for permission errors. On Debian/Ubuntu check for these errors in “/var/log/auth.log”. # error if authorized_keys file has too wide a permission for others Authentication refused: bad ownership or modes for file /home/myuser/.ssh/authorized_keys # error if .ssh directory has too wide a … Bash: fixing SSH authentication error “bad ownership or modes for file/directory”
When parsing a string that is divided by a separator char, getting the first N values OR last N values is a common scenario when dealing with: IP address separated by periods, e.g. “10.11.12.13” File path separated by forward slash “/tmp/myfolder/subpath1/subpath2/subpath3” Fully qualified domain separated by periods “sub1.sub2.my.domain.com”
Docker is a container platform that streamlines software delivery and provides isolation, scalability, and efficiency with less overhead than OS level virtualization. These instructions are taken from the official Docker for Ubuntu page, but I fine-tuned them per Ubuntu22+ standards.
Although jwt.io has become a common online destination for decoding JWT, this can also be done locally using jq. # populate JWT variable JWT=… # decode with jq utility echo $JWT | jq -R ‘split(“.”) | .[0],.[1] | @base64d | fromjson’ Attribution of credit goes to this gist.
If you need to test for a file’s existence, content size, and whether it was recently modified, the ‘find‘ utility can provide this functionality in a single call. One scenario for this usage might be the cached results from a remote service call (database, REST service, etc). If fetching these results was a relatively costly … Bash: testing if a file exists, has content, and is recently modified
If you are upgrading from Ubuntu 20 to Ubuntu 22 using ‘do-release-upgrade’ and get a fatal error ‘Connection to the the Snap Store failed’, this may be resolved by removing the ‘lxd’ package which is a lightweight container supervisor. sudo /etc/init.d/lxd stop sudo rm -fr /var/lib/lxd sudo dpkg –force depends -P lxd; sudo dpkg –force … Ubuntu: ‘Connection to the Snap Store failed’ during upgrade from Ubuntu 20 to 22
While enabling HTTPS is a important step in securing your web application, it is critical that you take steps to disable legacy protocols and low strength ciphers that can circumvent the very security you are attempting to implement. The Qualys SSL test is popular for grading the overall security of a public site, but you … Linux: using nmap to check the secure protocols and ciphers of a site
If you need a way to simply encrypt and decrypt files or strings with a symmetric key (same password for encryption and decryption), openssl provides this functionality. Be sure to avoid weak ciphers such as 3DES, and employ salt as well as PBKDF2 iterations to reduce vulnerability to brute force attacks. Here is a simple … Linux: using openssl to encrypt and decrypt files and strings
If apt update throws warnings about invalid signature verification and NO_PUBKEY, you may need to migrate from using the deprecated system keyring to using a ‘signed-by’ attribute in your apt repo definition file. Here are examples of errors you might see when doing an ‘apt update’. W: An error occurred during the signature verification. The … Ubuntu: fixing apt NO_PUBKEY errors by converting deprecated keyring to signed-by attribute
If you are attempting to ssh to a server and receive an error like below, it means the server side ssh daemon only supports a cryptographically weaker algorithm. Unable to negotiate with 192.168.2.1 port 22: no matching host key type found. Their offer: ssh-rsa If you still wish to connect, you can either provide this … Linux: ssh client throwing unable to negotiate error
If you have ever considered moving from WordPress to the Hugo static site generator, you can preview this migration by running Hugo on your local Ubuntu host. This will allow you to assess whether you can find suitable replacements for the WordPress plugins you have come to rely upon, validate your content syntax, and tweak … Hugo: exporting a WordPress blog to a static Hugo site on Ubuntu
If you need to extract the Nth occurrence of a match in a file (given definitive block separators), awk provides a convenient way to express the extraction. For example, a chained pem certificate will have multiple certification definitions with unique starting and ending marker lines. Here is how you would extract the second certificate. awk … Bash: awk to extract Nth match from file based on line separator
If you have ever considered moving from WordPress to the Jekyll static site generator, you can preview this migration by running jekyll on your local Ubuntu host. This will allow you to assess whether you can find suitable replacements for the WordPress plugins you have come to rely upon, validate your content syntax, and tweak … Jekyll: exporting a WordPress blog to a static Jekyll site on Ubuntu
If you have warning messages coming from apt about an invalid signature verification and EXPKEYSIG, then it is likely that a signing key for one of the remote apt repo has expired. Below is an example coming from an expired podman package at the “download.opensuse.org” repo. W: An error occurred during the signature verification. The … Ubuntu: fixing apt invalid signature warnings
If you have Dropbox installed on your Linux desktop and have recently started seeing this warning message from apt: http://linux.dropbox.com/ubuntu/dists/disco/Release.gpg: Key is stored in legacy trusted.gpg keyring (/etc/apt/trusted.gpg), see the DEPRECATION section in apt-key(8) for details This can be resolved by adding the Dropbox PGP key to the ‘signed-by’ of the apt repo definition (as … Ubuntu: fix apt warning for Dropbox with key in legacy keyring
Mike Farah’s yq yaml processor has a a rich set of operators and functions for advanced usage. In this article, I will illustrate how to take a block of yaml from one file to populate a specific location in another. Install yq Here are the steps for installing the latest yq on Snap enabled systems … yq: replace section of one yaml file with content section of another
Github Actions provide the ability to define a build workflow directly in Github. The workflow steps are defined as yaml and can be triggered by various events, including a code push, branch, or tagging in the repository. In this article I will detail the steps of creating a statically-linked GoLang binary that is automatically built … Github: automated Github release of GoLang binary using Github Actions
If you are using Linux with graphical X interface and need to copy to the clipboard, the ‘xclip‘ utility works similar to ‘pbcopy’ on a Mac terminal and will place the content unto your clipboard. Here is the basic usage: sudo apt install xclip -y # copies to clipboard echo “hello” | xclip -selection clipboard … Linux: xclip to place content on the clipboard
The Go language with its simplicity, concurrency support, rich package ecosystem, and ability to compile down to a single binary is an attractive solution for writing services on Ubuntu. However, the Go language does not natively provide a reliable way to daemonize itself. In this article I will describe how to take a couple of simple Go language programs … GoLang: Running a Go binary as a systemd service on Ubuntu 22.04
Update Sept 2024: changed the page parsed for latest version, validated installation of go1.23.1 on Ubuntu22.04 The Go programming language consistently ranks as one of the most popular languages in developer surveys. In fact, Kubernetes as well as most of the CNF projects are written in Go. And it compiles down to machine code, which … GoLang: Installing the Go Programming language on Ubuntu 22.04
If you need to quickly standup a secure HTTPS server on a machine, perhaps to validate that a client is sending the proper method or parameters, the socat utility can assist. In this article we will use “socat.local” as the FQDN of our certificate and host. Maybe you already have a pem key+certificate for this … Linux: socat used as secure HTTPS web server
In environments where certificates are manually deployed, reloading TLS certs is often only done annually when the certificate is near expiration. This long lapse in time often means that someone else has inherited the task of renewal, and the original key in use may even be in question. Luckily, openssl provides a way to validate … Linux: openssl to validate whether private key and TLS certificate match
sed is an excellent utility for doing substitutions, but if you need to work across multiple files, then you need to pair it with a command that can provide a list of candidate files. This is where find comes in handy. For example, to replace all instances of ‘zebra’ with ‘horse’ recursively in the current … Linux: sed to replace across multiple files in directory
When using a private key on the client to ssh into a remote server with the matching public certificate in ~/.ssh/authorized_keys, a common failure message from the client is: Permission denied (publickey) The most common reasons for this is private key permissions issues (chmod 600), a misconfiguration of authorized_keys, or trying to send the wrong … Linux: ssh-keygen to check whether ssh private key and public cert are keypair
The ClientAliveInterval and ClientAliveMaxCount settings in “/etc/sshd/sshd_config” work together to control the timeout value of an ssh session on the server side. But under BASH, to keep idle client sessions from timing out, you also need to set the ‘TMOUT’ variable or you will see messages like below when disconnected. timed out waiting for input: … Bash: extend timeout for idle ssh sessions using TMOUT
In this article I will demonstrate how to create an Ubuntu 22 template in vCenter. Then use Terraform to create a vSphere VM based on this template. The VM template creation is done by manually stepping through the Ubuntu server ISO installation wizard, followed by a set of preparation steps. Then Terraform is used to … Terraform: creating an Ubuntu 22 template and then guest VM in vCenter
In order to expose KVM virtual machines on the same network as your bare-metal Host, you need to enable bridged networking. In this article, I’ll show how to implement KVM bridged networking on Ubuntu 22.04 using Netplan. This bridged network will expose the KVM Guest OS as a peer on the upstream network, with no … KVM: Creating a bridged network with NetPlan on Ubuntu 22.04
The Microsoft .NET SDK is an open-source development platform for developing applications across multiple architectures and operating systems. In this article, I will show you how to install the .NET SDK on Ubuntu 20.04 and then create/compile/run a simple web application. Ubuntu 22 will have the dotnet-sdk available in the default Ubuntu apt repositories, but … Ubuntu: Installing .NET SDK 6 on Ubuntu 20.04