Kokil Thapa - Professional Web Developer in Nepal
Freelancer Web Developer in Nepal with 15+ Years of Experience

Kokil Thapa is an experienced full-stack web developer focused on building fast, secure, and scalable web applications. He helps businesses and individuals create SEO-friendly, user-focused digital platforms designed for long-term growth.

Distributed Tracing with OpenTelemetry and Jaeger

By Kokil Thapa | Last reviewed: September 2026

When a checkout fails or a booking API times out, logs alone rarely tell you which hop broke the chain. Distributed Tracing with OpenTelemetry and Jaeger gives you a shared request ID, span timings, and service maps so you can see latency hop by hop. On production Laravel stacks I maintain, tracing has cut mean time to recovery more than any dashboard tweak. This guide walks through instrumentation, Jaeger deployment, and the reading habits that actually find root cause — not just pretty graphs. If you already instrumented one service, see the companion piece on instrumenting an app with OpenTelemetry for deeper SDK notes.

What is Distributed Tracing with OpenTelemetry and Jaeger?

Distributed tracing records one logical request as it crosses processes, queues, and databases. Each unit of work becomes a span with a start time, duration, attributes, and optional events. Spans share a trace ID; parent-child links form a tree you can render as a flame graph or Gantt chart.

OpenTelemetry (OTel) is the vendor-neutral standard for traces, metrics, and logs. You instrument once with OTel APIs and SDKs, then export to Jaeger, Grafana Tempo, or a commercial backend without rewriting code. Jaeger is an open-source trace backend originally from Uber. It ingests OTLP or Jaeger-native spans, indexes them, and serves a UI for search and dependency graphs.

The split matters in practice. OTel owns propagation and instrumentation. Jaeger owns storage, query, and visualization. You can swap Jaeger for another backend later while keeping the same OTel code paths — a pattern I prefer on long-lived client systems where vendor lock-in is expensive.

OpenTelemetry + Jaeger Trace FlowWeb AppLaravel / PHPAPI ServiceREST / JSONWorkerQueue jobMySQLClient spanOpenTelemetry SDK + auto-instrumentationW3C traceparent headers on every outbound callOTel Collector (batch, filter, route)Receives OTLP gRPC or HTTPJaeger — storage, search, service map UI
Distributed Tracing with OpenTelemetry and Jaeger: services emit spans, the collector batches exports, Jaeger indexes traces for search.

Three concepts show up in every trace you will debug:

  • Trace ID — one 128-bit identifier for the whole request chain.
  • Span ID — one segment inside the trace; each HTTP call or DB query is usually one span.
  • Context propagation — headers such as traceparent carry IDs across network hops per the W3C Trace Context spec.

Without propagation, each service creates an orphan trace. You get four unrelated timelines instead of one story. That failure mode is common right after teams add tracing to only the edge gateway.

How do you instrument a PHP or Laravel app with OpenTelemetry?

PHP tracing uses the OpenTelemetry PHP SDK plus optional auto-instrumentation extensions. For Laravel 12 or 13 on PHP 8.3+, Composer packages and middleware give you HTTP server spans without wrapping every controller by hand.

Install dependencies

On a Laravel 12 project running PHP 8.3 or higher, add the core SDK and OTLP exporter:

composer require \
  open-telemetry/opentelemetry \
  open-telemetry/exporter-otlp \
  open-telemetry/sdk

composer require --dev open-telemetry/opentelemetry-auto-laravel

Register a service provider that boots the tracer only when tracing is enabled. Never ship 100% trace sampling to high-traffic production without budget for storage — start with a ratio sampler.

Configure environment variables

OTEL_SERVICE_NAME=booking-api
OTEL_TRACES_EXPORTER=otlp
OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf
OTEL_EXPORTER_OTLP_ENDPOINT=http://otel-collector:4318
OTEL_PHP_AUTOLOAD_ENABLED=true
OTEL_TRACES_SAMPLER=parentbased_traceidratio
OTEL_TRACES_SAMPLER_ARG=0.1

The parentbased_traceidratio sampler keeps entire traces intact once the root span is sampled. A random per-span sampler breaks parent-child links and produces misleading flame graphs.

Add manual spans around business logic

Auto-instrumentation covers HTTP, Guzzle, and PDO on many setups. Custom spans belong on payment callbacks, PDF generation, and third-party API calls — the places I see latency hide on API development projects.

use OpenTelemetry\API\Trace\Span;
use OpenTelemetry\API\Trace\StatusCode;

$tracer = $GLOBALS['otel_tracer'];

$span = $tracer->spanBuilder('payment.verify')
    ->setAttribute('gateway', 'khalti')
    ->startSpan();

$scope = $span->activate();

try {
    $result = $gateway->verify($payload);
    $span->setAttribute('payment.status', $result['status']);
} catch (\Throwable $e) {
    $span->setStatus(StatusCode::STATUS_ERROR, $e->getMessage());
    $span->recordException($e);
    throw $e;
} finally {
    $scope->detach();
    $span->end();
}

Propagate context on outbound HTTP calls. Guzzle middleware from the auto-instrumentation package injects traceparent when configured. For raw curl or legacy SDKs, read the current span context and set headers yourself.

Instrument queue workers separately

Laravel queue workers are separate PHP processes. Each job dispatch should inject trace context into the job payload or message headers. On a booking platform like Adventure Third Pole Trek, a slow confirmation email job looked like an API timeout until worker spans showed a 12-second SMTP handshake.

Run workers with the same OTEL_* variables as FPM pools. A mismatch here produces traces that stop at the web tier — one of the most common gaps on enterprise Laravel deployments.

Trace Context PropagationBrowserEdge NginxLaravelPayment API1. GET /checkout — no trace yet2. Edge creates root span + traceparent3. Laravel continues parent span ID4. Outbound call carries same trace IDHeaderstraceparent:00-abc-def-01tracestate:vendor=t1Same traceall four hops
W3C traceparent headers bind HTTP hops into one Distributed Tracing with OpenTelemetry and Jaeger trace tree.

How do you deploy Jaeger and the OpenTelemetry Collector?

Run Jaeger behind an OpenTelemetry Collector in production. The collector receives OTLP from many apps, batches spans, strips sensitive attributes, and forwards to Jaeger storage. Direct app-to-Jaeger works for demos; collectors scale better on real traffic.

Docker Compose stack for staging

This minimal stack suits a staging server on Ubuntu 24 with Docker. Jaeger all-in-one embeds memory storage — fine for dev, not for long retention.

services:
  jaeger:
    image: jaegertracing/all-in-one:1.64
    ports:
      - "16686:16686"
      - "4317:4317"
      - "4318:4318"
    environment:
      COLLECTOR_OTLP_ENABLED: "true"

  otel-collector:
    image: otel/opentelemetry-collector-contrib:0.120.0
    volumes:
      - ./otel-collector-config.yaml:/etc/otelcol-contrib/config.yaml
    ports:
      - "4317:4317"
      - "4318:4318"
    depends_on:
      - jaeger

Collector config forwarding traces to Jaeger:

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317
      http:
        endpoint: 0.0.0.0:4318

processors:
  batch:
    timeout: 5s
    send_batch_size: 1024
  attributes:
    actions:
      - key: http.request.header.authorization
        action: delete

exporters:
  otlp/jaeger:
    endpoint: jaeger:4317
    tls:
      insecure: true

service:
  pipelines:
    traces:
      receivers: [otlp]
      processors: [batch, attributes]
      exporters: [otlp/jaeger]

Point Laravel and worker processes at http://otel-collector:4318. Open the Jaeger UI on port 16686 and search by service name.

Production storage and retention

Memory storage loses data on restart. For production, use Jaeger with Elasticsearch or badger-backed persistent storage. Retention of seven to fourteen days is typical for teams without a dedicated observability budget — roughly Rs 15,000–40,000/month (~USD 110–295) in cloud storage at moderate volume.

Pair traces with metrics and logs through the same OTel Collector where possible. That alignment mirrors the approach in Prometheus alerting setups and reduces context switching during incidents.

Trace Sampling Decision TreeIncoming root spanHead sample?Keep 10%ratio samplerDrop traceno storage costError or slow?Tail samplealways keepProduction default: 10% head samplingplus tail rules for errors and p99 latency
Sampling strategy for Distributed Tracing with OpenTelemetry and Jaeger: ratio at the head, tail rules for errors and slow traces.

How do you read Jaeger traces to find production bottlenecks?

Jaeger’s UI is only useful if you search with intent. Start from a symptom — elevated p95 on POST /api/orders, a spike in 502 responses, or a user-reported trace ID from your logs — then narrow down.

Search workflow

  1. Open Jaeger UI and pick the service name from the dropdown.
  2. Filter by operation, minimum duration, or tags such as http.status_code=500.
  3. Sort by longest duration and open the slowest trace first.
  4. Switch to the flame graph view and identify the widest bar — that span consumed the most time.
  5. Compare a failing trace with a healthy trace of the same operation side by side.

Log correlation closes the loop. Include trace_id in structured JSON logs via a Monolog processor. When Kibana or Loki shows an error line, paste the trace ID into Jaeger and you jump straight to spans — faster than grep across twelve microservices. Validate exported JSON payloads with a JSON formatter when debugging attribute shape mismatches.

What the span attributes should tell you

Good attributes turn traces into actionable data. At minimum, capture HTTP method, route name, response status, database statement type (not raw SQL with PII), queue name, and external host. On legal-tech portals where document generation runs synchronously, a child span named dompdf.render often explains multi-second page loads that APM averages hide.

The Jaeger System Architecture or dependency graph view shows call edges weighted by traffic. A thick arrow from your API to an SMS gateway means that integration deserves a circuit breaker — the same class of failure described in API rate limiting and abuse prevention.

SignalLogs aloneMetrics aloneOpenTelemetry + Jaeger traces
Pinpoints slow downstream hopRarely — unless every service logs the same request IDOnly aggregate latency per serviceYes — per-span duration in one view
Shows serial vs parallel callsNoNoYes — Gantt layout reveals concurrency
Cost at moderate trafficLowLowMedium — storage grows with sample rate
Setup effort on Laravel monolithAlready presentModerateModerate — one OTel SDK + collector
Best for microservicesInsufficientGood for SLOsEssential for cross-service debugging

Traces complement — not replace — logs and metrics. Use Prometheus for SLIs, logs for exact error messages, and Jaeger for latency topology. That trio is the baseline I recommend during testing and optimization engagements.

Jaeger Flame Graph ReadingPOST /api/booking — 842 ms totalauth.middleware — 12 mscontroller.store — 820 msvalidateSQL SELECT — 780 msHealthy traceSQL span ~ 40 msSlow traceMissing index on joinWidest bar = biggest time sinkFix the red span first, redeploy, compare new tracesPair with MySQL slow query log for exact SQL
Reading Jaeger flame graphs: the widest span in Distributed Tracing with OpenTelemetry and Jaeger points to the latency fix.

When should you choose OpenTelemetry and Jaeger over other stacks?

Pick OTel plus Jaeger when you run multiple services in PHP, Node.js, or Go and need one trace format across all of them. OTel’s single SDK surface beats maintaining Jaeger-only clients, Zipkin clients, and vendor agents in parallel.

Jaeger fits teams that want self-hosted trace storage on their own Linux servers — common for Nepal-based businesses avoiding per-host SaaS pricing. Managed APM (Datadog, New Relic, Honeycomb) wins when you want traces, metrics, and log analytics in one bill and you accept vendor cost. Honeycomb-style high-cardinality trace analysis is stronger for event-heavy systems than stock Jaeger UI, but Jaeger is free and runs on the same EC2 box as your app.

Skip full distributed tracing when you operate a single Laravel monolith with no outbound microservices and no queue workers. Structured logs plus slow-query logging and speed optimization may be enough until you split services or add heavy async work.

Add tracing before you need it — not after a week of unexplained timeouts. The instrumentation cost is lowest while the codebase is still a monolith. Retrofitting propagation across fifteen internal APIs is painful, as teams learning the saga pattern for distributed transactions often discover too late.

Operational ownership matters. Jaeger needs disk, backups, and collector upgrades — work that falls under Linux system administration on many small teams. If nobody owns retention policy, traces fill disk silently until the collector starts dropping spans.

For greenfield platforms — marketplaces, booking engines, client portals — I treat OTel hooks as part of the initial scaffold. The same applies to AI-assisted workflows where external LLM calls need span-level cost attribution, a topic adjacent to AIOps for modern infrastructure.

Key Takeaways

  • Instrument with the OpenTelemetry PHP SDK, export OTLP to a collector, and store traces in Jaeger — keep propagation on every outbound HTTP call and queue job.
  • Use parentbased_traceidratio near 10% in production, plus tail sampling rules for errors and slow traces so storage stays predictable.
  • Run the OpenTelemetry Collector to batch, scrub sensitive headers, and forward spans — do not expose Jaeger directly to the public internet.
  • Correlate logs with trace_id so Jaeger search starts from a single error line instead of manual service-by-service grep.
  • Read flame graphs for the widest span first; compare slow and healthy traces before changing indexes or code paths.
  • Adopt tracing early on monoliths that will split into services — retrofitting W3C context across many internal APIs is expensive.

People Also Ask

What is the difference between OpenTelemetry and Jaeger?

OpenTelemetry defines APIs, SDKs, and the OTLP export protocol for generating and shipping telemetry. Jaeger is a backend that receives those spans, stores them, and provides search UI and service dependency graphs. You write code against OTel; Jaeger is one of several backends that can receive the data.

Does Laravel support OpenTelemetry natively?

Laravel does not ship built-in tracing as of Laravel 13, but community auto-instrumentation packages wrap HTTP kernel, routing, database, and queue events with OpenTelemetry spans. You still configure exporters and samplers through environment variables and a service provider bootstrapped at application start.

How much overhead does tracing add in production?

Well-sampled tracing typically adds single-digit millisecond overhead per request on PHP 8.3+ when using batch export through a local collector. Cost rises with 100% sampling, synchronous export, and heavy custom attributes on hot paths. Measure p95 before and after on staging with realistic load.

Can Jaeger receive traces without the OpenTelemetry Collector?

Yes. Applications can export OTLP directly to Jaeger when COLLECTOR_OTLP_ENABLED=true on modern Jaeger images. Collectors remain recommended for production because they centralize sampling, PII redaction, and multi-backend routing without redeploying every app.

Ship tracing before the next outage forces it

Distributed Tracing with OpenTelemetry and Jaeger turns cross-service guesswork into a repeatable debug workflow: instrument, propagate, sample wisely, and read flame graphs with purpose. Start on staging with Docker Compose, add log correlation, then roll sampling to production once retention and disk budgets are clear. If you want help wiring OTel into a Laravel platform, queue workers, and deployment pipeline, contact us or review how we approach custom software development and ongoing support and maintenance for production systems.

Frequently Asked Questions

Distributed tracing records one logical request as it crosses processes, queues, and databases. Each unit of work becomes a span with start time, duration, and attributes; spans share a trace ID and form a parent-child tree you can render as a flame graph. OpenTelemetry is the vendor-neutral standard for emitting that telemetry. Jaeger ingests OTLP spans, indexes them, and provides search UI and dependency graphs. OTel owns propagation and instrumentation; Jaeger owns storage and visualization.

OpenTelemetry defines APIs, SDKs, and the OTLP export protocol for generating and shipping telemetry. Jaeger is a backend that receives those spans, stores them, and provides search UI and service dependency graphs. You write code against OTel; Jaeger is one of several backends that can receive the data.

At moderate traffic with seven to fourteen days retention, expect roughly Rs 15,000–40,000 per month (~USD 110–295) for cloud storage, depending on sample rate and span volume.

Laravel 13 does not ship built-in tracing. Community auto-instrumentation packages wrap the HTTP kernel, routing, database, and queue events with OpenTelemetry spans on Laravel 12 or 13 running PHP 8.3 or higher. You still register a service provider, set OTEL_* environment variables, and configure exporters yourself. The setup is moderate effort — one SDK plus a collector — but it is not a core framework feature you enable with a single config flag.

Install open-telemetry/opentelemetry, open-telemetry/exporter-otlp, and open-telemetry/sdk via Composer, plus open-telemetry/opentelemetry-auto-laravel as a dev dependency. Register a provider that boots the tracer only when tracing is enabled. Set OTEL_SERVICE_NAME, OTEL_TRACES_EXPORTER=otlp, OTEL_EXPORTER_OTLP_ENDPOINT pointing at your collector, and OTEL_PHP_AUTOLOAD_ENABLED=true. Add manual spans around payment callbacks, PDF generation, and third-party API calls where auto-instrumentation misses business logic.

Use parentbased_traceidratio at around 0.1 so roughly ten percent of traces stay intact from root to leaf. Random per-span sampling breaks parent-child links and produces misleading flame graphs. Never ship one hundred percent sampling on high-traffic production without storage budget. Add tail sampling rules for errors and slow traces so incidents remain visible even when the head ratio is low.

Queue workers run as separate PHP processes from your FPM pool. Without injecting trace context into the job payload or message headers at dispatch time, each worker starts an orphan trace disconnected from the web request. Run workers with the same OTEL_* variables as FPM. On a booking platform, a slow confirmation email job once looked like an API timeout until worker spans exposed a twelve-second SMTP handshake hidden from the web tier.

Applications can export OTLP directly to Jaeger for demos and small setups. In production, run a collector in front of Jaeger. It receives spans from many apps, batches exports, strips sensitive attributes like authorization headers, and forwards to Jaeger storage. Collectors scale better under real traffic and keep Jaeger off the public internet. Point Laravel and workers at http://otel-collector:4318, not Jaeger directly.

Well-sampled tracing typically adds single-digit millisecond overhead per request on PHP 8.3+ with batch export through a local collector. Overhead rises with full sampling, synchronous export, or heavy custom attributes.

On Ubuntu 24 staging, use Docker Compose with jaegertracing/all-in-one:1.64 and otel/opentelemetry-collector-contrib:0.120.0. Enable COLLECTOR_OTLP_ENABLED on Jaeger, mount a collector config with OTLP receivers on ports 4317 and 4318, a batch processor, attribute scrubbing, and an otlp/jaeger exporter. Open the Jaeger UI on port 16686. For production, replace Jaeger memory storage with Elasticsearch or badger-backed persistence so traces survive restarts.

Start from a symptom — elevated p95 on an endpoint, a spike in 502 responses, or a trace ID from logs. Search Jaeger by service name, filter by operation, minimum duration, or tags like http.status_code=500. Open the slowest trace, switch to the flame graph, and inspect the widest bar first — that span consumed the most time. Compare a failing trace with a healthy trace of the same operation side by side before changing indexes or code.

Context propagation carries trace and span IDs across network hops using W3C headers such as traceparent. Each service reads incoming context and creates child spans under the same trace ID. Without propagation, every service generates orphan traces — four unrelated timelines instead of one request story. This failure mode is common when teams instrument only the edge gateway. Guzzle middleware from the auto-instrumentation package injects traceparent on outbound HTTP when configured.

Choose OTel plus Jaeger when you run multiple services in PHP, Node.js, or Go and need one trace format across all of them. Jaeger suits teams wanting self-hosted storage on their own Linux servers, avoiding per-host SaaS pricing common for Nepal-based businesses. Managed APM like Datadog, New Relic, or Honeycomb wins when you want traces, metrics, and log analytics in one bill. Skip full distributed tracing for a single Laravel monolith with no outbound microservices and no queue workers.

Include trace_id in structured JSON logs via a Monolog processor. When Kibana or Loki shows an error line, paste that trace ID into Jaeger search and jump straight to the full span tree. This closes the loop faster than grepping across a dozen microservices. The same trace ID appears in every span of the request chain, so one log line anchors the entire call tree in the Jaeger UI.

At minimum, record HTTP method, route name, response status, database statement type without raw SQL containing PII, queue name, and external host. Good attributes turn flame graphs into fixes instead of decoration. On legal-tech portals where document generation runs synchronously, a child span named dompdf.render often explains multi-second page loads that aggregate APM averages hide. Validate exported JSON payloads with a formatter when debugging attribute shape mismatches.

Share this article

0 Comments

Leave a comment

Your email is not published. Comments appear once they have been read. Sign in to have your details filled in.

Quick Contact Options
Choose how you want to connect me: