For a long time, adding observability to an application meant picking a vendor first. You installed their agent, used their SDK, and sent data in their format. If you later wanted to switch tools, or move from a hosted service to something you ran yourself, you rewrote your instrumentation.
OpenTelemetry changed that. It gives you one set of APIs, SDKs, and a wire protocol for metrics, traces, and logs, and lets you send that data anywhere: a commercial platform, an open-source stack on your own servers, or both at once.
What OpenTelemetry is
OpenTelemetry, often shortened to OTel, is an open-source observability framework run under the Cloud Native Computing Foundation. It formed in 2019 when two earlier projects, OpenTracing and OpenCensus, merged, and it has since become one of the CNCF's most active projects.
It's important to be clear about what OpenTelemetry isn't. It isn't a database, a dashboard, or an alerting system. It doesn't store your data or show it to you. OpenTelemetry is the layer that generates, collects, processes, and ships telemetry; you still need backends to store and visualize it.
That separation is the point. Instrument your code once with OpenTelemetry, and you can change what's behind it without touching the application.
The three signals: Traces, metrics, and logs
Traces follow a single request as it moves through your system. A trace is made of spans, each representing one operation, such as an HTTP request, a database query, or a call to another service, with timings and attributes. In a system with several services, traces show you exactly where a slow request spent its time.
Metrics are numeric measurements over time: request rates, error counts, latency histograms, queue depths, memory usage. They're cheap to store and ideal for dashboards and alerts.
Logs are timestamped records of events. OpenTelemetry's contribution here is correlation: logs emitted inside a traced request can carry the trace and span IDs, so you can jump from a slow trace straight to the log lines it produced.
The project is also working on continuous profiling as a fourth signal, but traces, metrics, and logs are the core today.
The main components
The API and SDKs are what you use inside your application. There are official SDKs for most major languages, including Java, Python, Go, JavaScript and Node.js, .NET, Ruby, PHP, Rust, and C++. The API defines how you create spans and record metrics; the SDK handles sampling, batching, and exporting.
Auto-instrumentation, sometimes called zero-code instrumentation, adds telemetry to common frameworks and libraries without code changes. For Java, Python, Node.js, and .NET, you can often get useful traces from web frameworks, HTTP clients, and database drivers just by running your app with the OpenTelemetry agent.
OTLP, the OpenTelemetry Protocol, is the wire format for sending telemetry. It runs over gRPC, usually on port 4317, or HTTP, usually on port 4318. Many backends accept OTLP directly; check support for each signal and the required endpoint.
The Collector is a standalone service that receives telemetry, processes it, and exports it to one or more backends. It's the piece that makes a self-hosted setup flexible.
Semantic conventions are the shared naming rules for attributes, such as http.request.method or service.name, so data from different languages and libraries lines up consistently.
Why the Collector matters
You could have every application send telemetry straight to your backends, but in practice the Collector is worth running.
It decouples applications from backends. Your apps send OTLP to the Collector, and only the Collector needs to know where data goes. Changing backends means changing one config file.
It processes data on the way through. The Collector can batch data for efficiency, drop noisy spans, strip sensitive attributes, add metadata like environment or host names, and sample traces so you don't store every single request.
It translates between formats. The contrib distribution of the Collector includes receivers for Prometheus scrapes, host metrics, log files, and many other sources, plus exporters for a wide range of backends. That lets you bring older systems into the same pipeline.
A Collector config has three main parts: receivers that accept data, processors that transform it, and exporters that send it on. Pipelines tie them together for each signal.
Choosing self-hosted backends
The most common open-source stack pairs OpenTelemetry with tools from the Prometheus and Grafana ecosystems.
Prometheus stores metrics. Recent versions can accept OTLP metrics directly, as well as scraping targets in the traditional way. Grafana Tempo or Jaeger stores traces; both accept OTLP. Grafana Loki stores logs, and recent versions accept OTLP logs natively. Grafana ties it together with dashboards, queries, and links between signals, so you can click from a metric spike to the traces behind it and then to the related logs.
If you'd rather run one system than four, there are all-in-one, OpenTelemetry-native platforms such as SigNoz, OpenObserve, and Uptrace. They're simpler to operate, at the cost of less flexibility in mixing and matching components.
Try a self-hosted OpenTelemetry stack
For a complete starting point, Grafana publishes docker-otel-lgtm, which bundles a Collector and the observability backends. Its maintainers describe it as a development, demonstration, and testing image. Use it to learn the data flow; plan a separately configured deployment for production.
Step 1: Run the learning environment
With Docker installed on a Linux VPS, run:
docker run -d --name otel-lgtm \
-p 127.0.0.1:3000:3000 \
-p 127.0.0.1:4317:4317 \
-p 127.0.0.1:4318:4318 \
grafana/otel-lgtm:latest
docker logs -f otel-lgtm
This learning command uses the current image; record its version or digest when reproducing a setup. It doesn't configure durable production storage. Bound ports are accessible only on the host's loopback interface.
Step 2: Access Grafana securely
From your workstation, create an SSH tunnel, replacing the login and hostname:
ssh -L 3000:127.0.0.1:3000 [email protected]
Open Grafana at http://127.0.0.1:3000 on your workstation. Check the project's current README for demo credentials and change them before wider access. The image provisions its data sources for you.
Step 3: Instrument a Python application
For an existing Python app running directly on the same VPS, install instrumentation in that app's virtual environment:
python3 -m venv .venv
. .venv/bin/activate
# Install your application's dependencies before bootstrapping instrumentation.
pip install opentelemetry-distro opentelemetry-exporter-otlp
opentelemetry-bootstrap -a install
export OTEL_SERVICE_NAME=my-app
export OTEL_EXPORTER_OTLP_ENDPOINT=http://127.0.0.1:4318
export OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf
opentelemetry-instrument python app.py
Replace app.py with your app's entry point and generate requests. Check for traces in Grafana's Explore view. Exact signals depend on the SDK and installed instrumentation; traces appearing doesn't mean application logs and custom metrics are automatically covered.
If the app runs in another container, its localhost is a different network namespace. Put both containers on a private Docker network and use the Collector container's service name and port instead. Remote applications need a secured private route to ingestion.
Step 4: Design the production deployment
A production stack needs explicit backend configuration, storage, retention, authentication, backups, and resource limits. Use each backend's supported deployment examples, pin compatible versions, and test ingestion and queries for all three signals before relying on the system.
For a separate Collector, configure receivers, processors, exporters, and a pipeline for each signal. Prometheus requires its OTLP receiver to be enabled if you're using push ingestion; Tempo and Loki require compatible receiver and storage settings. Keep these services private rather than exposing unauthenticated ingestion endpoints.
Securing and sizing the stack
Don't expose the Collector's OTLP ports to the open internet. Keep them on a private network, a VPN, or behind a firewall that only allows your application servers, and enable TLS and authentication if telemetry crosses networks you don't control.
Plan storage and retention. Traces are the biggest consumer, so use sampling to keep a representative share rather than every request, and set retention periods in Tempo, Loki, and Prometheus that match how far back you actually look. A week or two of traces and a few weeks of logs is common for smaller setups.
Size the server for the volume. Start with a small test workload and measure actual memory, CPU, and disk consumption. There is no reliable universal RAM minimum for a stack with several backends; ingestion rate, cardinality, and retention determine sizing. As volume grows, split the backends onto separate machines or move trace and log storage to object storage, which Tempo and Loki both support.
And remember that observability needs observing too. An uptime check on Grafana and the Collector, using something like Uptime Kuma, tells you when the system you rely on to find problems has a problem of its own.
Wrapping up
OpenTelemetry is a vendor-neutral standard for generating and collecting traces, metrics, and logs. It doesn't store or display data itself; it gives you consistent instrumentation, a common protocol, and a Collector that can send telemetry wherever you want.
Paired with Prometheus, Tempo, Loki, and Grafana, or an all-in-one platform like SigNoz, it makes a fully self-hosted observability stack practical for teams of any size, with your telemetry data staying on infrastructure you control.
Thanks for reading! If you're planning to run your own observability stack, xTom provides enterprise-grade dedicated servers and colocation, while V.PS offers scalable, NVMe-powered KVM VPS for smaller deployments. We also offer IP transit, shared hosting, and general IT services.
Ready to discuss your infrastructure needs? Contact our team to explore the right solution for your projects.
Frequently asked questions about OpenTelemetry
What is OpenTelemetry used for?
OpenTelemetry is used to generate, collect, and export telemetry data, meaning traces, metrics, and logs, from applications and infrastructure, so you can understand performance and troubleshoot problems.
Is OpenTelemetry a replacement for Prometheus or Grafana?
No. OpenTelemetry collects and ships data, while Prometheus stores metrics and Grafana visualizes data. They work together, and recent Prometheus versions can receive OpenTelemetry metrics directly.
What is the OpenTelemetry Collector?
The Collector is a standalone service that receives telemetry, processes it with steps like batching, filtering, and sampling, and exports it to one or more backends. It decouples your applications from your storage systems.
Do I need to change my code to use OpenTelemetry?
Not always. Auto-instrumentation for languages like Java, Python, Node.js, and .NET adds telemetry to common frameworks and libraries without code changes. Custom spans and metrics do require adding SDK calls.
What ports does OpenTelemetry use?
OTLP over gRPC typically uses port 4317, and OTLP over HTTP typically uses port 4318.
Can I self-host OpenTelemetry for free?
Yes. OpenTelemetry and common backends like Prometheus, Tempo, Loki, Jaeger, and Grafana are open source. Your main costs are the servers and storage you run them on.