If you are using awk with a relatively indexed NF variable (number of fields), and get an error like below, this is because the input being parsed does not have the number of fields expected. awk: run time error: negative field index $-1 The root problem is the awk expression was expecting a certain number … Bash: resolving awk run time error, negative field index
sed is an excellent utility for doing substitutions, but if you need to work across multiple files, then you need to pair it with a command that can provide a list of candidate files. This is where find comes in handy. For example, to replace all instances of ‘zebra’ with ‘horse’ recursively in the current … Linux: sed to replace across multiple files in directory
If you can conform to a bit of naming convention, one way to easily render template files in Bash is to have the template and bash variables names match. Then you can either use sed or envsubst to do the substitution. For example, here are two variables being defined and then placeholders embedded into the … Bash: render template from matching bash variables
If you are doing a substitution with sed but need to exclude a specific line or pattern, that can be accomplished by prefixing an exclusion and using “!”. For example, to substitute “hello” for “goodbye” in the following content, but to skip any line containing “world”, use syntax like the following: $ sed “/world/! s/hello/goodbye/g” … Bash: sed substitution with an exclusion pattern
Instead of stringing together sed multiple times in a pipeline, it is also possible to make multiple substitutions with a single invocation of sed. Consider the following example which replaces the word ‘hello’ as well as ‘quick’ in the paragraph: $ sed “s/hello/goodbye/g; s/quick/slow/g” <<EOF hello, world! hello, universe! the quick brown fox EOF goodbye, … Bash: performing multiple substitutions with a single sed invocation
If you need to output only lines before (or after) a unique line identified by a regular expression, then here are examples of using sed. All examples in this article can be found in github. Consider the following lines of content in the variable “$lines” $ echo “$lines” ape swings frog jumps logs dog barks … Bash: output all lines before/after line identified by regex
Whether looking at differences in filenames, installed packages, etc. it can be useful to calculate the difference between two Bash arrays. SiegeX on stackoverflow.com offered the following function using awk, and I have built a full example available on github. function arraydiff() { awk ‘BEGIN{RS=ORS=” “} {NR==FNR?a[$0]++:a[$0]–} END{for(k in a)if(a[k])print k}’ <(echo -n “${!1}”) <(echo … Bash: Difference between two arrays
YAML is a popular syntax for configuration, and it is common to have certificate definitions embedded in these files. But since the cert is typically Base64 PEM encoded, it means you can’t easily view its attributes (subject, expiration date, etc) and so you are left with the manual task of copy-pasting it out, saving as … Bash: Examining each certificate in a yaml file using sed and openssl
sed is a powerful utility for transforming text. One of the nice tricks with sed is the ability to reuse capture groups from the source string in the replacement value you are constructing. For example, if you have have the following kernel parameters in “/etc/default/grub” $ grep GRUB_CMDLINE_LINUX_DEFAULT /etc/default/grub GRUB_CMDLINE_LINUX_DEFAULT=”quiet splash” And wanted to append … Bash: Appending to existing values using sed capture group
If you have to ssh to a remote system to produce a block of json, it may be preceded and suffixed with login and system messages. Or you may have a program that sends json as the main part of its output, but also sends debug type messages to stdout that need to be cleaned. … Linux: sed to cleanup json that has errant text surrounding it
The sed utility is a powerful utility for doing text transformations. In this article, I will provide an example of how to insert a line before and after a match using sed, which is a common task for customizing configuration files. I have also written a related article on setting and replacing values in a … Linux: Using sed to insert lines before or after a match