#GlueCon 2014 Notes: Love your Logs with Elasticsearch ELK – Jordan Sissel, Elasticsearch
Love your Logs with Elasticsearch ELK – Jordan Sissel, Elasticsearch Been in operations for 10 years, recently switched to programming but always been one What is a log? Time + Data Logs come with a scale problem – we can read a few lines, but can’t consume lots of them at once We got here because of printf Complex logging rules that make things look pretty to humans break analytics (and add more complexity to a simple printf) (cough MySQL) Structured logs, such as JSON, makes it easier to capture logs, parse with one line and no regex, and then analyze Just use a standard time format (e.g. ISO8601) ELK is 3 open source apps: Elasticsearch, Logstash, and Kibana “If a new user has a bad time, it’s a bug” Processing and Transport – a universal glue that connects inputs to outputs with filtering along the way Graph everything and look for shapes to find anomalies Use tools that allow you to explore those anomalies to find the root cause
Love your Logs with Elasticsearch ELK – Jordan Sissel, Elasticsearch
- Been in operations for 10 years, recently switched to programming but always been one
- What is a log? Time + Data
- Logs come with a scale problem – we can read a few lines, but can’t consume lots of them at once
- We got here because of printf
- Complex logging rules that make things look pretty to humans break analytics (and add more complexity to a simple printf) (cough MySQL)
- Structured logs, such as JSON, makes it easier to capture logs, parse with one line and no regex, and then analyze
- Just use a standard time format (e.g. ISO8601)
- ELK is 3 open source apps: Elasticsearch, Logstash, and Kibana
- “If a new user has a bad time, it’s a bug”
- Processing and Transport – a universal glue that connects inputs to outputs with filtering along the way
- Graph everything and look for shapes to find anomalies
- Use tools that allow you to explore those anomalies to find the root cause