ELK: Custom template mappings to force field types

It is very common to have Logstash create time-based indexes in ElasticSearch that fit the format, <indexName>-YYYY.MM.DD.  This means events submitted with @timestamp for that day all go to the same index.

However, if you do not explicitly specify an index template that maps each field to a type, you can end up with unexpected query results.  The reason is that without explicit mappings, the index (that is created fresh each day) uses its best judgement to assign field types based on the first event inserted.

In this article, I’ll show you how to create explicit custom index templates so that field types are uniform across your time-series indexes.

Continue reading “ELK: Custom template mappings to force field types”

Nginx: Custom access log format and error levels

Nginx is a powerful application level proxy server.  Whether for troubleshooting or analysis, enabling log levels and custom formats for the access/error logs is a common requirement.

Error Logs

By default, only messages in the error category are logged.  If you want to enable more details, then modify nginx.conf like:

error_log file [level]

Enabling debug level on Linux would usually look like:

error_log /var/log/nginx/error.log debug;

Access Logs

Access logs and their format are also customized in nginx.conf.  By default, if no format is specified then the combined format is used.

access_log file [format]

Continue reading “Nginx: Custom access log format and error levels”

SaltStack: Creating a Custom Grain using Python

saltstack_logo-thumbnailSaltStack grains are used for relatively static information such as operating system, IP address, and other system properties.  They are also useful for targeting minions, for example whether a system is part of  dev/test/prod, or a flag on whether it falls under LifeScience or HIPAA regulation.

In this article we will implement a custom grain that determines whether a host is part of development, test, or production environment based on a simplistic naming scheme.   This custom grain will be written in Python.

Continue reading “SaltStack: Creating a Custom Grain using Python”

EMC OnDemand: Best Practices for Custom Methods

The concept of custom methods which run directly on the Java Method Server has proven an extremely useful extension point for Documentum developers and solutions architects.  Whether used in a workflow activity to integrate with an enterprise message queue or as an action for Webtop users who need temporarily escalated privileges to apply legal retention, custom Java methods have become a key customization in most customer environments. Features include:

  • Lightweight invocation of methods as compared to dmbasic and external Java methods that require execution
  • DFC operations execute on the same host as the Content Server which minimizes the effects of network latency and throughput
  • Can be configured to run as the repository owner which allows them elevated privileges to content when necessary
  • Provide the logic for workflow auto-activities, able to utilize any Java library including the DFC
  • Provide the logic for custom job/methods, again able to utilize the full power of Java and its libraries

Continue reading “EMC OnDemand: Best Practices for Custom Methods”