Bash: avoiding newline artifacts when Base64 encoding a string

Base64 encoding a string may seem like a straight-forward operation, but there are a couple of gotchas even when dealing with just simple ASCII strings.

Avoid embedding a new line character into the encoding

If you use the most straight forward method of Base64 encoding shown below, you have to remember that echo by default writes a newline character to stdout, which means the Base64 encoded value includes that newline.

# this has a newline char embedded into base64 encoding
$ echo "This is my string" | base64
VGhpcyBpcyBteSBzdHJpbmcK

# this does NOT have a newline char embedded
$ echo -en "This is my string" | base64
VGhpcyBpcyBteSBzdHJpbmc=

Avoid multiple lines of output from Base64 encoding

If your string is longer than even a few words, you may notice that the base64 outputs multiple lines of data (76 character default column width).  While this is desired for some purposes (like certificate data or email MIME), it can produce invalid syntax for others.  Use the ‘-w0’ flag to avoid line wrap in the base64 output.

# example of column wrap at 76 chars
$ echo -en "This is my string that needs to be encoded and is long enough to wrap" | base64
VGhpcyBpcyBteSBzdHJpbmcgdGhhdCBuZWVkcyB0byBiZSBlbmNvZGVkIGFuZCBpcyBsb25nIGVu
b3VnaCB0byB3cmFw

# instead, use '-w0' flag to output single line
$ echo -en "This is my string that needs to be encoded and is long enough to wrap" | base64 -w0
VGhpcyBpcyBteSBzdHJpbmcgdGhhdCBuZWVkcyB0byBiZSBlbmNvZGVkIGFuZCBpcyBsb25nIGVub3VnaCB0byB3cmFw

One real-world example where this multi-line syntax could cause failure is when retrieving an OAuth2 Access Token using a Basic ‘Authorization’ header.

# credentials for Basic authorization formatted as <user>:<secret>
$ credentials='TestUserForSandbox-qatest-api-client1:dukh2eFBm!2C@F2qCz7yrwa1'

# multiple lines are generated, which would make the curl fail
$ creds_base64_BAD=$(echo -en "$credentials" | base64)
$ echo "$creds_base64_BAD"
VGVzdFVzZXJGb3JTYW5kYm94LXFhdGVzdC1hcGktY2xpZW50MTpkdWtoMmVGQm0hMkNARjJxQ3o3
eXJ3YTE=

# token retrieval will fail with invalid credentials
curl -X POST $token_URL -H "Authorization: Basic $creds_base64_BAD" --data "grant_type=client_credentials&scope=my"

# single line generated, which is required for valid syntax in 'Authorization' header
$ creds_base64_GOOD=$(echo -en "$credentials" | base64 -w0)

REFERENCES

man page for ‘echo’

man page for ‘base64’

wikipedia Base64

NOTES

The ‘Authorization’ header can also be generated by curl since this use-case is so common

curl -v --user $credentials -X POST $token_URL --data "grant_type=client_credentials&scope=my"