Hi All, this document gives you an overview of how ELK works.
ELK stands for Elastic Search, Logstash and Kibana.
How does it work?
Logs from the server can be shipped to the logstash server using file beat or any other alternative software. Or you can use a logstash individual server to do this same operation.
Once logs are received to logstash it goes through the input and filter section. In the filter section of the logstash configuration file, you can map values and filter unwanted the string using the same filter, these all depend on the configuration mentioned in the configuration file.
Once the logs are passed through the input and filter section, it goes to the output section in the conf file. This section decides where the logs need to be shipped. The logs will be sent and saved in the elasticsearch index format.
Even you can add conditional statements in the output section.
Here is the sample configuration file,
Elasticsearch is a kind of database where the data is getting saved. By default, it listens on port 9200 and 9300. Even elasticsearch support cluster setup, where you can create master nodes and data nodes.
Kibana is the visualizer where you can see the data stored in the elasticsearch. Kibana connects to the 9200 port of elasticsearch and that how the data is getting displayed in the dashboard. Kibana has support for creating dashboards and widgets. In the later version of kibana, it has multiple features included such as ARM, monitoring, Machine Learning, etc. It’s all dependent on which version you use.
Kibana supports various kinds of diagram such bar chart, heat map, pie chart, geolocation map, etc
You can refer to the below block diagram to see how it works.
Here is the document in which you can see how to set up a highly available ELK stack.build highly available ELK stack
© 2019, Techrunnr. All rights reserved.