Download as PDF, PPTX
![PromQL - Examples
@phennex
rate(api_http_requests_total[5m])
errors{job=“foo”} / total{job=“foo”}
Source: https://promcon.io/2016-berlin/talks/prometheus-design-and-philosophy/](https://image.slidesharecdn.com/monitoringwithprometheus-161102054822/85/Monitoring-with-prometheus-40-320.jpg)
![PromQL - Examples
@phennex
rate(api_http_requests_total[5m])
errors{job=“foo”} / total{job=“foo”}
Source: https://promcon.io/2016-berlin/talks/prometheus-design-and-philosophy/](https://image.slidesharecdn.com/monitoringwithprometheus-161102054822/75/Monitoring-with-prometheus-40-2048.jpg)
Uploaded byKasper Nissen
Monitoring with prometheus
The document provides an overview of monitoring with Prometheus, covering its purpose, architecture, metrics collection, and alerting strategies. It emphasizes the importance of monitoring various system aspects including latency, traffic, errors, and saturation, while also introducing Prometheus as a pull-based monitoring system created by ex-Google engineers. Additionally, it includes practical suggestions for improving alerting processes and references for further information on the topic.
More Related Content
Similar to Monitoring with prometheus
![UiPath Automation Suite Installation (Hands-On) [2/3]](https://cdn.slidesharecdn.com/ss_thumbnails/automationsuitecommunitysession2-251015095633-a6d862f1-thumbnail.jpg?width=600ounds&width=560&fit=bounds)
Monitoring with prometheus
- 1.
- 2. Hi! My name isKasper @phennex
- 3. What am Igoing to cover? @phennex + + + Monitoring - why and what? Prometheus - an introduction Short demo
- 4.
- 5.
- 6. What to monitor? @phennex Analyzinglong-term trends @phennex
- 7. What to monitor? @phennex Comparingover time or experiment groups @phennex
- 8.
- 9. What to monitor? @phennex Buildingdashboards @phennex
- 10. @phennex Conducting ad hocretrospective analysis @phennex
- 11.
- 12.
- 13. What to monitor? @phennex Hosts CPU,Memory, I/O, Network, Filesystem @phennex
- 14. What to monitor? @phennex Containers CPU,Memory, I/O, Restarts, Throttling @phennex
- 15. What to monitor? @phennex Applications Throughput,Latency @phennex
- 16. The Four GoldenSignals @phennex Site Reliability Engineering - How Google Runs Production Systems
- 17. What to monitor? @phennex Latency Thetime it takes to service a request. Important to distinguish between the latency of successful and failed requests. @phennex
- 18. What to monitor? @phennex Traffic Ameasure of how much demand is being placed on your system, measured in a high-level system-specific metric. @phennex
- 19. What to monitor? @phennex Errors Therate of requests that fail, either explicitly (e.g. HTTP 500s), implicitly (HTTP 200 success with wrong content) @phennex
- 20. What to monitor? @phennex Saturation How“full” your service is. A measure of your system fraction, emphasizing the resources that are most constrained (e.g. in a memory-constrained system, show memory) @phennex
- 21.
- 22. What to monitor? @phennex Prometheus Prometheuswas presented to be the protector and benefactor of mankind. @phennex
- 23. Prometheus @phennex + + + + Heavily inspired byBorgmon Built by ex-Googlers at SoundCloud Pull-based (scrapes at regular intervals) Many integration possibilities
- 24. The 2nd projectin CNCF
- 25. What is Prometheus? @phennex + + + + + + Monitoringsystem and Timeseries Database Instrumentation Metrics collection and storage Querying Alerting Dashboard / Graphing / Trending Source: https://promcon.io/2016-berlin/talks/prometheus-design-and-philosophy/
- 26. Prometheus focus on @phennex + + Operationalsystems monitoring Dynamic cloud environments Source: https://promcon.io/2016-berlin/talks/prometheus-design-and-philosophy/
- 27. Prometheus does notdo @phennex + + + + + + Raw log / event collection (use ELK stack) Request tracing (use opentracing.io) “Magic” anomaly detection Durable long-term storage Automatic horizontal scaling User / auth management
- 28. Prometheus Architecture @phennex Long-lived jobs PushgatewayAlertmanagerShort-lived jobs Grafana
- 29. The Data model @phennex <metricname>{<label name>=<label value>, …} api_http_requests_total{method="POST", handler="/messages"} Notation: Example: Every time series is uniquely identified by its metric name and a set of key- value pairs, also known as labels.
- 30. How to getmetrics? @phennex Directly instrumented Not Directly instrumented Exporter Source: https://promcon.io/2016-berlin/talks/so-you-want-to-write-an-exporter/
- 31.
- 32.
- 33. Official Prometheus Exporters @phennex Node/systemmetrics exporter AWS CloudWatch exporter Blackbox exporter Collectd exporter Consul exporter Graphite exporter HAProxy exporter InfluxDB exporter JMX exporter Memcached exporter Mesos task exporter MySQL server exporter SNMP exporter StatsD exporter
- 34. 3rd party exporters @phennex Databases Aerospikeexporter ClickHouse exporter CouchDB exporter MongoDB exporter PgBouncer exporter PostgreSQL exporter ProxySQL exporter Redis exporter RethinkDB exporter SQL query result set metrics exporter
- 35. 3rd party exporters @phennex Hardwarerelated apcupsd exporter IoT Edison exporter IPMI exporter knxd exporter Ubiquiti UniFi exporter Messaging systems NATS exporter NSQ exporter RabbitMQ exporter RabbitMQ Management Plugin exporter Mirth Connect exporter
- 36. 3rd party exporters @phennex Storage Cephexporter ScaleIO exporter HTTP Apache exporter Nginx metric library Passenger exporter Varnish exporter WebDriver exporter APIs Docker Hub exporter GitHub exporter OpenWeatherMap exporter Rancher exporter Speedtest.net exporter Logging Google's mtail log data extractor Grok exporter Other monitoring systems Cloud Foundry Firehose exporter scollector exporter Heka dashboard exporter Heka exporter Munin exporter New Relic exporter Miscellaneous BIG-IP exporter BIND exporter BOSH exporter Jenkins exporter Meteor JS web framework exporter Minecraft exporter module PowerDNS exporter rTorrent exporter SMTP/Maildir MDA blackbox prober Xen exporter
- 37. PromQL @phennex + + + Non-SQL Query Language Betterfor metrics computation Only does reads Source: https://promcon.io/2016-berlin/talks/prometheus-design-and-philosophy/
- 38. PromQL - Operators @phennex +(addition) == (equal) - (substraction) != (not-equal) * (multiplication) > (greater-than) / (division) < (less-than) % (modulo) >= (greater-or-equal) ^ (exponentiation) <= (less-or-equal) and (intersection) or (union) unless (complement) … and vector matching Source: https://prometheus.io
- 39. PromQL - AggregationOperators @phennex sum stddev bottomk min stdvar topk max count quantile avg count_values Source: https://prometheus.io
- 40. PromQL - Examples @phennex rate(api_http_requests_total[5m]) errors{job=“foo”}/ total{job=“foo”} Source: https://promcon.io/2016-berlin/talks/prometheus-design-and-philosophy/
- 41.
- 42.
- 43. What to monitor? @phennex Symptom-basedalerting Be proactive @phennex
- 44. What to monitor? @phennex Preventalert fatigue - Use ticketing systems (Avoid email spam) - Warning are tasks like new features @phennex
- 45. What to monitor? @phennex Providerunbooks - Keep them concise - Explanation, hints, links - Dynamic - include recent observations @phennex
- 46. What to monitor? @phennex Practiceoutages “Firedrills”, “Gamedays” - repeat regularly @phennex
- 48. @phennex Start being proactive. Dontbe firefighters.
- 49. … and remember… @phennex
- 50. Hope is NOTa strategy @phennex Source: Site Reliability Engineering, How Google Runs Production Systems (2016), B. Beyer et al.
- 51. If you wannaknow more… @phennex - prometheus.io - promcon.io - The Site Reliability Engineering book - Podcasts: - https://dev.to/sedaily/prometheus-monitoring-with-brian-brazil - https://dev.to/sedaily/the-art-of-monitoring-with-james-turnbull (prefers push based opposite prometheus) - https://dev.to/sedaily/prometheus-with-julius-volz
- 52. @phennex The 3rd projectin CNCF opentracing.io
- 53.