#GlueCon 2014 Notes: Love your Logs with Elasticsearch ELK – Jordan Sissel, Elasticsearch
Love your Logs with Elasticsearch ELK – Jordan Sissel, Elasticsearch
- Been in operations for 10 years, recently switched to programming but always been one
- What is a log? Time + Data
- Logs come with a scale problem – we can read a few lines, but can’t consume lots of them at once
- We got here because of printf
- Complex logging rules that make things look pretty to humans break analytics (and add more complexity to a simple printf) (cough MySQL)
- Structured logs, such as JSON, makes it easier to capture logs, parse with one line and no regex, and then analyze
- Just use a standard time format (e.g. ISO8601)
- ELK is 3 open source apps: Elasticsearch, Logstash, and Kibana
- “If a new user has a bad time, it’s a bug”
- Processing and Transport – a universal glue that connects inputs to outputs with filtering along the way
- Graph everything and look for shapes to find anomalies
- Use tools that allow you to explore those anomalies to find the root cause