filter

Python: exploring the use of startswith against a list: tuple, regex, list comprehension, lambda


Categories: Python

As with any programming language, Python has a multitude of ways to accomplish the same task.  In this article, I will explore the idea of taking a string and checking if it ‘startswith’ any of the strings from a predetermined list. As context for the example code, RFC1918 sets out several IPv4 ranges that can … Python: exploring the use of startswith against a list: tuple, regex, list comprehension, lambda

GCP: Analyzing members of IAM role using gcloud filtering and jq


Categories: Scripting

Although  the GCP console provides a nice interface for displaying which user/service account is in which IAM security role (IAM & Admin > IAM), it can be difficult to analyze using gcloud get-iam-policy because of the inner array of ‘members’ returned. However, if you use the flattening ability of gcloud, it becomes much easier to … GCP: Analyzing members of IAM role using gcloud filtering and jq

ELK: metadata fields in Logstash for grok and conditional processing


Categories: Linux, Logging

When building complex, real-world Logstash filters, there can be a fair bit of processing logic.  There are typically multiple grok patterns as well as fields used as flags for conditional processing. The problem is, these intermediate extracted fields and processing flags are often ephemeral and unnecessary in your ultimate persistent store (e.g. ElasticSearch), but they will be inserted … ELK: metadata fields in Logstash for grok and conditional processing

ELK: Using Ruby in Logstash filters


Categories: Logging, Scripting

Logstash has a rich set of filters, and you can even write your own, but often this is not necessary since there is a out-of-the-box filter that allows you to embed Ruby code directly in the configuration file. Using logstash-filter-ruby, you can use all the power of Ruby string manipulation to parse an exotic regular expression, … ELK: Using Ruby in Logstash filters