Read the first item in this Table of Contents if you haven't been here before.
Table of Contents
- The ELKBeats Stack: Sounds Like a Good Idea ...
- The ELKBeats Stack: the Ground Work
- The ELKBeats Stack: L is for Logstash
- The ELKBeats Stack: E is for Elasticsearch
- The ELKBeats Stack: K is for Kibana
- The ELKBeats Stack: Getting E, L, and K to play nice together
- The ELK Stack with Beats: Feeding Logstash with Beats (Insecure - so far)
- The ELK Stack with Beats: Securing the Beats-to-Logstash Connection
Putting it Together
Change the output stanza of /etc/logstash/conf.d/apache.conf, adding a feed to elasticsearch:
output { stdout { codec => rubydebug } elasticsearch { } # yup, empty }
You can keep, comment out, or remove the stdout feed: I'm leaving it active until I trust the setup a bit more.
Restart logstash with systemctl restart logstash
. If you're getting a logstash error in the log, remember to try running this:
# /opt/logstash/bin/logstash --configtest --config /etc/logstash/conf.d/apache.conf
Configuration OK
This shows a passing configuration. The failure messages aren't terribly helpful, but do offer a bit of direction.
Hit "http://localhost/" a few times to feed logstash some data. Then use curl or wget to visit elasticsearch at "localhost:9200/_cat/indices" (Kibana is available remotely: this is not, has to be fetched from localhost). Before I properly configured logstash so it could fetch the Apache log file, I was getting this:
green open .kibana 1 0 1 0 3.1kb 3.1kb
After correctly configuring logstash:
green open logstash-2016.03.03 5 0 46 0 98.4kb 98.4kb green open .kibana 1 0 1 0 3.1kb 3.1kb
Once that's working, it's time to try kibana: visit "http://localhost:5601". It will ask you about its configuration, but if it has no data to work with it won't actually let you complete the configuration. If it does have data, you can proceed and start to tinker with it.
Continue to The ELK Stack with Beats: Feeding Logstash with Beats (Insecure - so far), the next article in this series.
Bibliography
(This is the same Bibliography for all of the "ELKBeats Stack" articles.)
- https://en.wikipedia.org/wiki/Elasticsearch
- https://en.wikipedia.org/wiki/Kibana
- https://www.linode.com/docs/databases/elasticsearch/webserver-logs-with-elk-stack ... this is an excellent set of instructions that's significantly out-of-date (old URLs/addresses), which was nevertheless my main source of information
- https://www.elastic.co/guide/en/logstash/current/config-examples.html
- http://www.webupd8.org/2014/03/how-to-install-oracle-java-8-in-debian.html (with the caveat that as of 2016-03, my instructions are more accurate than theirs ...)
- https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-discovery-zen.html
- Getting Kibana Up and Running
- Elasticsearch Getting Started
- Elasticsearch Reference >> Installation
- Elasticsearch Repositories (at elastic.co)
- Getting Started with Logstash
- https://www.elastic.co/guide/en/beats/libbeat/1.1/elasticsearch-installation.html
- Logstash Repositories (at elastic.co)
- How To Install Elasticsearch, Logstash, and Kibana (ELK Stack) on Ubuntu 14.04, Digital Ocean's uneven guide to this same subject, occasionally helpful but big on "install this" and short on "understand"
- http://main.justinflowers.ca/web/wordpress/?p=19