ELK: Scaling an ElasticSearch Cluster

elasticsearch-logoThe heart of the ELK stack is Elasticsearch.  In order to provide high availability and scalability, it needs to be deployed as a cluster with master and data nodes.  The Elasticsearch cluster is responsible for both indexing incoming data as well as searches against that indexed data.

Resources

As described in the documentation, if there is one absolutely critical resource it is memory.  Keeping the heap size less than 32G will allow you to use compressed object pointers which is preferred.  Swapping memory takes a big hit, so minimize swappiness on your Linux host.

Continue reading “ELK: Scaling an ElasticSearch Cluster”

Ubuntu: Using a swap file instead of swap partition for virtualized server VMs

ubuntuBefore virtualization, there was a stronger argument for using a swap partition instead of a swap file for servers.  A fragmented swap file could lead to performance issues that a statically sized and placed partition did not have consider.

But once virtualization comes into play, unless you go to great lengths to segment your storage pools, that swap partition is not guaranteed to be either statically sized or statically placed on a physical platter.  And at that point, you should consider using a swap file which provides more flexibility in sizing and capacity planning.

Here are instructions for adding a 16Gb swap file to Ubuntu:

Continue reading “Ubuntu: Using a swap file instead of swap partition for virtualized server VMs”