jq has a very powerful expression language for querying and transforming json. But when dealing with complex data structures and responses in the real-world, well-tested happy path queries can fail. One common error is: jq: error (at <stdin>:1): Cannot index array with string “…” The root cause is that your query is attempting to pull … jq: Resolving “Cannot index array with string” errors from jq
The need to configure a specific pod’s container arguments is a common Kubernetes administration task. As examples, you might need to enable verbose logging, set an explicit value to override a default, or configure a host name or port set in a container’s arguments. In the example below, we are targeting the ‘metrics-server’ in the … Kubernetes: patching container arguments array with kubectl and jq
The power of kustomize lies in its ability to transform yaml, and to that end it has built-in support for the JSON 6902 RFC specification. This RFC specifies the syntax for adding, removing, replacing, and moving elements and values in a yaml file. And since Kubernetes manifests are yaml files, it makes perfect sense to … Kubernetes: kustomize transformations with patchesJson6902
If you have a complex JSON data structure or perhaps an array of rich data structures as the result of module output, you may want to extract this into a workable list or dictionary you can take action on. For this article, I will emulate the resulting output from the ‘stat‘ module when it checks … Ansible: extracting a list or dictionary from a complex data structure
Most of the modern cloud platforms and utilities have us manipulate either JSON or YAML configuration files. And when you start dealing with real world scenarios with hundreds of lines of embedded data structures it is too difficult and error-prone to manually inspect indentation levels to determine the exact dotted or json path to an … Python: converting JSON to dot notation for easier path determination
Specifying input variables in the “terraform.tfvars” file in HCL syntax is commonly understood. But if the values you need are already coming from a json source, it might make more sense to feed those directly to Terraform. Here is an example where the simple variable “a” is provided via an external json file. # … Terraform: using json files as input variables and local variables
Although the GCP console provides a manual interface for creating service accounts and assigning roles, it can also be done via the gcloud CLI. Using gcloud, even the json key file for the service account can be generated, which is essential for automation. The full Bash script, create_serviceaccount.sh can be found on github. But here … GCP: Creating gcp service account with IAM roles using gcloud
In a previous post, I described how to install and use the ‘govc‘ tool – which is a command line tool for working with VMware vCenter. When you do a listing on a virtual machine “govc ls -l -json <vmpath>”, you are returned back a very detailed JSON data structure that contains all the information … Python: JSONPath to extract vCenter information using govc
There are many tools that utilize json, and when it is relatively simple you can use standard modules or even custom coding to pull out the desired portions. But once these data structures reach a certain level of complexity you really should consider a Python module that implements JSONPath (analogous to xPath for XML). The … Python: Querying JSON files with JSONPath using jsonpath_rw_ext
If you have to ssh to a remote system to produce a block of json, it may be preceded and suffixed with login and system messages. Or you may have a program that sends json as the main part of its output, but also sends debug type messages to stdout that need to be cleaned. … Linux: sed to cleanup json that has errant text surrounding it
The Cloud Foundry CLI makes it easy to authenticate into a target API endpoint to consume the Cloud Controller API (CAPI). The “cf login” command allows you to pass or interactively provide the user, password, org, and space. Once logged in, if you want to run raw commands, the “cf curl” can execute a request … CloudFoundry: Authenticating and running CF API commands using curl
Syslog is a message logging standard has been around for decades, but has renewed popularity as a method of log capture with the advent of containerization and centralized logging solutions. Enabling an Ubutu 14.04 or 16.04 host to act as a syslog server only takes a few simple steps.
Python is a language whose advantages are well documented, and the fact that it has become ubiquitous on most Linux distributions makes it well suited for quick scripting duties. In this article I’ll go through an example of using Python to read entries from a JSON file, and from each of those entries create a … Python: Using Python, JSON, and Jinja2 to construct a set of Logstash filters