KEMBAR78
Opentelemetry - From frontend to backend | PDF
Vienna DevOps & Security Meetup
From front- to backend with OpenTelemetry
$ whoami - Sebastian
Sebastian Poxhofer
Software Engineer @ Whizus
Github: @secustor
Open Source Enthusiast & Contributor
Agenda
- Context
- Concepts
- Lookout
- Demo
- QA
Why should I care?
What’s out there? / The state of monitoring
- Microservices lead to a more complex debugging process
- OSS
- Observability pillars are separated
- Proprietary products
- Heavy vendor lock in, especially if SDKs are used
- Public clouds
- Mostly support only their own products
- Limited imports available
- Product changes require system redesigns
- Metadata is not standardized
What is observability?
- We need to answer questions about our systems.
What characteristics did the queries that timed out at 500ms share in common? Service
versions? Browser plugins?
- Instrumentation produces data.
- Querying data answers our questions.
Origins
One protocol to rule them all
The pillars of observability (signals):
- Traces ( GA )
- Metrics ( BETA )
- Logs ( Draft/Experimental )
The container for monitoring?
But why OpenTelemetry and not Y?
- Semantic conventions for metadata
- Open standard
- Supported by :
- + 300 companies
- + 2,000 contributors
- + 100,000 contributions in the last year
- CNCF ( incubating )
- FAANG ( MAANA? 🤔) + M
CONCEPTS
Components
- Open Telemetry Protocol ( OTLP )
- Instrumentation libraries
- Auto instrumentation
- Manual instrumentation
- Collector
Context propagation
- HTTP/2
- GRPC
- Messaging
- Depends on instrumentation
- Kafka
- RabbitMQ
- ...
Instrumentations
Supported Languages:
- Golang
- Java
- JavaScript/TypeScript
- .NET
- Swift
- C++
- Python
- ...
Distributions:
- Core
- Contrib
- Instrumentation
The collector pipeline
Collector components
Receiver:
- OTLP
- AWS ( xRay, Container
Insights )
- GCP ( Sub & Span)
- Jaeger/Zipkin
- Prometheus
- Filelog
- Kafka
- …
Processor:
- Batch
- MemoryLimiter
- GroupByAttribute
- GroupByTrace
- Filter
- SpanMetrics
- K8Attributes
- ...
Exporter:
- OTLP
- AWS ( xRay, CloudWatch, …)
- GCP
- AzureMonitor
- Jaeger/Zipkin
- Kafka
- Loki
- Splunk
- …
Gateway vs Agent deployment
-
Source: OpenTelemetry documentation
What’s next?
Status
- Like Kubernetes in 2015/16
- Basics are there, advanced stuff needs some work
- Most SDKs GA
- Auto instrumentation available for Java and JS
- Opentelemetry Operator in development
- Java agent injection possible
Lookout
- Agent Management Protocol
- eBPF tracing
- .NET auto instrumentation ( ❤ Datadog )
Demo
Applications
Applications
Trace flow
Metrics flow
Log flow
Links
- Instrumentations: https://opentelemetry.io/registry/
- Collector
- Receivers:
https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver
- Exporters:
https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/exporter
- Demo: https://github.com/secustor/opentelemetry-meetup
We are hiring!
Go to https://whizus.com
Q&A

Opentelemetry - From frontend to backend

  • 1.
    Vienna DevOps &Security Meetup From front- to backend with OpenTelemetry
  • 2.
    $ whoami -Sebastian Sebastian Poxhofer Software Engineer @ Whizus Github: @secustor Open Source Enthusiast & Contributor
  • 3.
    Agenda - Context - Concepts -Lookout - Demo - QA
  • 4.
  • 5.
    What’s out there?/ The state of monitoring - Microservices lead to a more complex debugging process - OSS - Observability pillars are separated - Proprietary products - Heavy vendor lock in, especially if SDKs are used - Public clouds - Mostly support only their own products - Limited imports available - Product changes require system redesigns - Metadata is not standardized
  • 6.
    What is observability? -We need to answer questions about our systems. What characteristics did the queries that timed out at 500ms share in common? Service versions? Browser plugins? - Instrumentation produces data. - Querying data answers our questions.
  • 7.
  • 8.
    One protocol torule them all The pillars of observability (signals): - Traces ( GA ) - Metrics ( BETA ) - Logs ( Draft/Experimental )
  • 9.
    The container formonitoring?
  • 10.
    But why OpenTelemetryand not Y? - Semantic conventions for metadata - Open standard - Supported by : - + 300 companies - + 2,000 contributors - + 100,000 contributions in the last year - CNCF ( incubating ) - FAANG ( MAANA? 🤔) + M
  • 11.
  • 12.
    Components - Open TelemetryProtocol ( OTLP ) - Instrumentation libraries - Auto instrumentation - Manual instrumentation - Collector
  • 13.
    Context propagation - HTTP/2 -GRPC - Messaging - Depends on instrumentation - Kafka - RabbitMQ - ...
  • 14.
    Instrumentations Supported Languages: - Golang -Java - JavaScript/TypeScript - .NET - Swift - C++ - Python - ... Distributions: - Core - Contrib - Instrumentation
  • 15.
  • 16.
    Collector components Receiver: - OTLP -AWS ( xRay, Container Insights ) - GCP ( Sub & Span) - Jaeger/Zipkin - Prometheus - Filelog - Kafka - … Processor: - Batch - MemoryLimiter - GroupByAttribute - GroupByTrace - Filter - SpanMetrics - K8Attributes - ... Exporter: - OTLP - AWS ( xRay, CloudWatch, …) - GCP - AzureMonitor - Jaeger/Zipkin - Kafka - Loki - Splunk - …
  • 17.
    Gateway vs Agentdeployment - Source: OpenTelemetry documentation
  • 18.
  • 19.
    Status - Like Kubernetesin 2015/16 - Basics are there, advanced stuff needs some work - Most SDKs GA - Auto instrumentation available for Java and JS - Opentelemetry Operator in development - Java agent injection possible
  • 20.
    Lookout - Agent ManagementProtocol - eBPF tracing - .NET auto instrumentation ( ❤ Datadog )
  • 21.
  • 22.
  • 23.
  • 24.
  • 25.
  • 26.
  • 27.
    Links - Instrumentations: https://opentelemetry.io/registry/ -Collector - Receivers: https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver - Exporters: https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/exporter - Demo: https://github.com/secustor/opentelemetry-meetup
  • 28.
    We are hiring! Goto https://whizus.com
  • 29.