ELK: Federated Search with a Tribe node

elasticsearch-logoAlthough the ELK stack has rich support for clustering, clustering is not supported over WAN connections due to Elasticsearch being sensitive to latency.  There are also practical concerns of network throughput given how much data some installations index on an hourly basis.

So as nice as it would be to have a unified, eventually consistent cluster span across your North America and European datacenters, that is not currently a possibility.  Across availability zones in the same AWS datacenter will work, but not across different regions.

Federated Search

But first let’s consider why we want a distributed Elasticsearch cluster in the first place.  It is not typically for geo failover or disaster recovery (because we can implement that separately in each datacenter), but more often because we want end users to have a federated search experience.

We want end users to go to a single Kibana instance, regardless of which cluster they want to search, and be able to execute a search query against the data.  A Tribe node can bridge two distinct clusters for this purpose.

Continue reading “ELK: Federated Search with a Tribe node”