Wiki source code of ELK - ElasticSearch, LogStash, Kibana
Last modified by Normann P. Nielsen on 2016/04/13 19:06
Hide last authors
| author | version | line-number | content |
|---|---|---|---|
| |
1.1 | 1 | = Testings = |
| 2 | |||
| 3 | |= Object|= Comment / Link|= Status | ||
| 4 | | **ELK Stack**| An excellent Guide for Ubuntu 14.04 is at [[https://www.digitalocean.com/community/tutorials/how-to-install-elasticsearch-logstash-and-kibana-elk-stack-on-ubuntu-14-04]]| **** | ||
| 5 | | **TopBeat **| [[https://www.digitalocean.com/community/tutorials/how-to-gather-infrastructure-metrics-with-topbeat-and-elk-on-ubuntu-14-04]]** **| | ||
| 6 | | **PacketBeat**| [[https://z0z0.me/monitor-your-servers-with-elasticsearch-2-x-and-beats-and-display-it-in-kibana/]]| | ||
| 7 | | **GeoIP Support**| [[https://www.digitalocean.com/community/tutorials/how-to-map-user-location-with-geoip-and-elk-elasticsearch-logstash-and-kibana]]| Gave some [[field mapping>>https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping.html]] challenges, and I had to delete the filebeat index values. | ||
| 8 | [[The Great Mapping Refactoring>>https://www.elastic.co/blog/great-mapping-refactoring]] | ||
| 9 | | **Tomcat Log Parsing**| [[https://blog.lanyonm.org/articles/2014/01/12/logstash-multiline-tomcat-log-parsing.html]]| | ||
| 10 | | **Shield (Security)**| [[https://www.elastic.co/guide/en/shield/current/kibana.html#using-kibana4-with-shield]]| | ||
| 11 | |||
| 12 | |||
| 13 | |||
| 14 | = Tips = | ||
| 15 | |||
| 16 | {{success}}Make sure the server time is correct for all servers :) as in use [[NTP>>https://help.ubuntu.com/lts/serverguide/NTP.html]]. | ||
| 17 | |||
| 18 | I have divided stuff a bit, please read: | ||
| 19 | |||
| 20 | = [[Logstash - Tips and Troubleshooting>>Main.information-technology.elk-elasticsearch-logstash-kibana.logstash-tips-and-troubleshooting]] = | ||
| 21 | |||
| 22 | = [[Elasticsearch - Tips and Troubleshooting>>Main.information-technology.elk-elasticsearch-logstash-kibana.elasticsearch-tips-and-troubleshooting]] ={{/success}} | ||
| 23 | |||
| 24 | = | ||
| 25 | |||
| 26 | = | ||
| 27 | |||
| 28 | = Sample filebeat.yml config for my Confluence Server = | ||
| 29 | |||
| 30 | |||
| 31 | {{code}} | ||
| 32 | paths: | ||
| 33 | - /var/log/auth.log | ||
| 34 | - /var/log/syslog | ||
| 35 | document_type: syslog | ||
| 36 | input_type: log | ||
| 37 | - | ||
| 38 | paths: | ||
| 39 | - /var/log/apache2/www.mos-eisley.dk-*.log | ||
| 40 | document_type: apache | ||
| 41 | input_type: log | ||
| 42 | - | ||
| 43 | paths: | ||
| 44 | - /data/www/Fordor.log | ||
| 45 | - /data/www/Baghus.log | ||
| 46 | document_type: camfileslog | ||
| 47 | input_type: log | ||
| 48 | {{/code}} | ||
| 49 | |||
| 50 | |||
| 51 | |||
| 52 | = Sample filebeat.yml config for my Alfresco Server = | ||
| 53 | |||
| 54 | |||
| 55 | {{code}} | ||
| 56 | paths: | ||
| 57 | - /var/log/auth.log | ||
| 58 | - /var/log/syslog | ||
| 59 | document_type: syslog | ||
| 60 | input_type: log | ||
| 61 | - | ||
| 62 | paths: | ||
| 63 | - /var/log/apache2/alfresco.mos-eisley.dk-*.log | ||
| 64 | - /var/log/apache2/elk.mos-eisley.dk-*.log | ||
| 65 | document_type: apache | ||
| 66 | input_type: log | ||
| 67 | - | ||
| 68 | paths: | ||
| 69 | - /var/log/pingkaf.txt | ||
| 70 | document_type: pinglog | ||
| 71 | input_type: log | ||
| 72 | {{/code}} | ||
| 73 | |||
| 74 | |||
| 75 | |||
| 76 | = Other Stuff: = | ||
| 77 | |||
| 78 | [[http://www.slideshare.net/aca_it/monitor-your-atlassian-stack-like-the-nsa]] | ||
| 79 | |||
| 80 | [[Elasticsearch CRUD>>https://www.digitalocean.com/community/tutorials/how-to-interact-with-data-in-elasticsearch-using-crud-operations]] | ||
| 81 | |||
| 82 | [[The Great Mapping Refactoring>>https://www.elastic.co/blog/great-mapping-refactoring]] | ||
| 83 | |||
| 84 | [[Embedding Visualisations>>https://www.youtube.com/watch?v=xj3dUMI263Y]] | ||
| 85 | |||
| 86 | [[A bit of logstash cooking>>https://home.regit.org/2014/01/a-bit-of-logstash-cooking/]] | ||
| 87 | |||
| 88 | [[ELK - 3 THINGS I WISH I'D KNOWN>>http://blog.scottlogic.com/2014/12/19/elk-3-things-i-wish-id-known.html]] | ||
| 89 | |||
| 90 | [[Little Logstash Lessons - Part I: Using grok and mutate to type your data>>https://www.elastic.co/blog/little-logstash-lessons-part-using-grok-mutate-type-data]] | ||
| 91 | |||
| 92 | [[5 Logstash Pitfalls You Need to Avoid>>http://logz.io/blog/5-logstash-pitfalls-and-how-to-avoid-them/]] |