Fabian Lee : Software Engineer

Bash: fixing “Too many authentication failures” for ssh with private key authentication

December 21, 2023
Categories: Linux

If you are using ssh private/public keypair authentication, and get an almost immediate error like below: $ ssh -i id_rsa.pub myuser@a.b.c.d -p 22 Received disconnect from a.b.c.d port 22:2: Too many authentication failures Disconnected from a.b.c.d port 22 Then try again using the ‘IdentitiesOnly‘ option. ssh -o ‘IdentitiesOnly yes’ -i id_rsa.pub myuser@a.b.c.d -p 22 The … Bash: fixing “Too many authentication failures” for ssh with private key authentication

Bash: testing if a file exists, has content, and is recently modified

May 10, 2023
Categories: Linux

If you need to test for a file’s existence, content size, and whether it was recently modified, the ‘find‘ utility can provide this functionality in a single call. One scenario for this usage might be the cached results from a remote service call (database, REST service, etc). If fetching these results was a relatively costly … Bash: testing if a file exists, has content, and is recently modified

Bash: performing multiple substitutions with a single sed invocation

March 25, 2021
Categories: Scripting

Instead of stringing together sed multiple times in a pipeline, it is also possible to make multiple substitutions with a single invocation of sed. Consider the following example which replaces the word ‘hello’ as well as ‘quick’ in the paragraph: $ sed “s/hello/goodbye/g; s/quick/slow/g” <<EOF hello, world! hello, universe! the quick brown fox EOF goodbye, … Bash: performing multiple substitutions with a single sed invocation

Bash: Renaming files using shell parameter expansion

January 29, 2020
Categories: Scripting

Shell parameter expansion provides various ways to manipulate strings, and a convenient way to succinctly express renaming a set of files. In its simplest form, parameter expansion is simply ${parameter}. But look at these examples: $ mystr=”TheQuickBrownFox.jpg” # chop off last 4 digits $ echo ${mystr:0:-4} TheQuickBrownFox # truncate end of string ‘.jpg’ $ echo … Bash: Renaming files using shell parameter expansion

Bash: Using shell or environment variables in awk output

January 19, 2020
Categories: Linux

If you are in the middle of a text processing pipeline, and need to insert a shell or environment variable into the output of awk, you can use the “-v” flag. Here are two files containing animal classifications: $ echo -e “shark=fish\ndolphin=mammal” > ocean.txt $ echo -e “dog=mammal\neagle=bird” > land.txt By passing the loop variable … Bash: Using shell or environment variables in awk output

Bash: Skipping lines at the top or bottom of a stream

January 18, 2020
Categories: Linux

Using head to get the first lines of a stream, and tail to get the last lines in a stream is intuitive. But if you need to skip the first few lines of a stream, then you use tail “-n +k” syntax. And to skip the last lines of a stream head “-n -k” syntax. … Bash: Skipping lines at the top or bottom of a stream

Bash: Running command on quoted list of parameters using xargs

September 30, 2019
Categories: Linux, Scripting

The typical usage for xargs is feeding it multiple lines of input, and having it run commands using each line as an argument. But there are other cases when you have a single line separated by a delimiter or even a list of optionally quoted args, and I’ll show in this article how to handle … Bash: Running command on quoted list of parameters using xargs

Linux: Using sed to insert lines before or after a match

October 28, 2018
Categories: Linux

The sed utility is a powerful utility for doing text transformations. In this article, I will provide an example of how to insert a line before and after a match using sed, which is a common task for customizing configuration files. I have also written a related article on setting and replacing values in a … Linux: Using sed to insert lines before or after a match