Kokil Thapa - Professional Web Developer in Nepal
Freelancer Web Developer in Nepal with 15+ Years of Experience

Kokil Thapa is an experienced full-stack web developer focused on building fast, secure, and scalable web applications. He helps businesses and individuals create SEO-friendly, user-focused digital platforms designed for long-term growth.

The Four Golden Signals of Monitoring

By Kokil Thapa | Last reviewed: September 2026

Production breaks in predictable ways. Requests slow down before they fail. Queues fill before disks do. The Four Golden Signals of Monitoring cut through metric noise and tell you whether users are actually suffering. Google introduced this framework in the Site Reliability Engineering book, and it still fits Laravel APIs, WooCommerce stores, and bare Ubuntu servers in 2026. If you run a support and maintenance contract or self-host on Linux, these four numbers belong on every dashboard.

What Are The Four Golden Signals of Monitoring?

The framework comes from Google SRE practice. It answers one question: is the service healthy for real users right now? You do not need hundreds of charts. You need four signal types that map directly to user experience.

  • Latency — time to complete work, including queue wait inside the service.
  • Traffic — demand level: requests per second, orders per minute, active sessions.
  • Errors — failed work: HTTP 5xx, payment callback rejections, job retries exhausted.
  • Saturation — how full the bottleneck resource is: CPU, memory, DB connections, queue depth.

Latency and errors tell you users are hurting. Traffic gives context—a spike explains higher latency. Saturation often predicts the next outage. Together they form a minimal observability contract. This pairs cleanly with the broader split between logs, metrics, and traces described in our guide on observability vs monitoring.

Four Golden SignalsUser-FacingLatencyResponse timeErrorsFailed requestsSystem ContextTrafficDemand loadSaturationResource useProduction ServiceWeb app, API, queue worker, databaseAlert when signals degrade together
The Four Golden Signals of Monitoring split user pain (latency, errors) from demand and capacity (traffic, saturation).

On a booking platform like Adventure Third Pole Trek, latency covers checkout and Livewire updates. Traffic tracks concurrent trek searches during peak season. Errors catch failed Khalti callbacks. Saturation watches MySQL connection pool and Redis queue depth.

How Do You Measure Latency as a Golden Signal?

Latency is not average response time alone. Averages hide slow tail requests that ruin checkout. Measure latency at the service boundary—the edge your users hit.

Use percentiles, not means

Track p50, p95, and p99 for HTTP handlers and API endpoints. A p99 of 2 seconds with a p50 of 120 ms means 1% of users wait painfully long. That 1% often includes payment flows.

# Prometheus histogram example (nginx or app middleware)
http_request_duration_seconds_bucket{le="0.1"} 8420
http_request_duration_seconds_bucket{le="0.5"} 9910
http_request_duration_seconds_bucket{le="1.0"} 9985
http_request_duration_seconds_bucket{le="+Inf"} 10000
http_request_duration_seconds_sum 892.4
http_request_duration_seconds_count 10000

In Grafana, graph histogram_quantile(0.99, rate(http_request_duration_seconds_bucket[5m])). Set alerts on p99 crossing SLO thresholds, not on CPU alone. Our Prometheus metrics fundamentals article walks through histogram setup in more depth.

Separate success latency from error latency

Failed requests often return fast—a 401 in 5 ms skews your latency chart downward. Filter by status code or result label:

histogram_quantile(0.95,
  sum(rate(http_request_duration_seconds_bucket{status=~"2.."}[5m])) by (le, route)
)

For background jobs, latency means queue wait plus processing time. A Laravel job that waits 10 minutes then runs in 2 seconds still feels broken to the user waiting on email.

Latency Measurement PathClientLoad BalancerEdge latencyApp ServerHandler timeDatabaseQuery timeMeasure at user boundary firstThen drill into slow spans internallyp50 baselineTypical userp95 warningDegraded UXp99 alertSLO breach
Golden-signal latency starts at the user boundary; internal spans explain where time is lost.

How Do You Track Traffic, Errors, and Saturation Together?

Traffic without latency context misleads. A traffic drop might mean DNS failure, not happy users leaving. Errors without traffic context hides rare but critical paths. Saturation without traffic misses idle resources that still matter at peak.

Traffic: quantify demand

Use counters that match business meaning:

  • HTTP: rate(http_requests_total[5m]) by route and method.
  • API: requests per consumer, per endpoint, with auth label stripped of secrets.
  • Queues: enqueue rate vs dequeue rate on Laravel Horizon or Redis.
  • eCommerce: orders per minute, cart submissions, payment attempts.

Compare today vs same hour last week. Dashain and Tihar spikes on Nepal-facing sites can double traffic without code changes. Plan capacity from that pattern.

Errors: count what users cannot complete

HTTP 5xx is the obvious error signal. Also track:

  1. 4xx on critical paths (checkout 422 vs blog 404).
  2. Payment gateway timeouts and webhook signature failures.
  3. Queue job failures after max retries.
  4. Health check failures on Laravel health endpoints.
# Error rate as golden signal
sum(rate(http_requests_total{status=~"5.."}[5m]))
/
sum(rate(http_requests_total[5m]))

Alert on error rate above 1% for five minutes, not on single 500s. One bad deploy can spike briefly; sustained rate means rollback.

Saturation: watch the bottleneck

Saturation asks how much headroom remains on the limiting resource. CPU at 40% can still mean saturation if the DB connection pool is maxed.

ResourceSaturation signalTypical alert threshold
CPUUtilisation vs request latency riseSustained > 80% with p95 latency up
MemoryAvailable RAM, PHP-FPM pool exhaustionOOM kills or swap use
DiskI/O wait, free space on /var> 85% full or iowait > 30%
DatabaseActive connections, slow query log rateConnections > 80% of max_connections
QueueDepth, oldest job ageDepth growing 15+ minutes

I've seen Laravel apps where Redis memory hit 100% while CPU looked fine. Sessions and cache evictions caused random logouts. That is saturation doing its job as an early warning.

Signal Correlation FlowTraffic UpDemand spikeSaturationPool near limitLatency Upp99 degradesErrors RiseTimeouts and 503 responsesFix saturation before errors become an outageScale pool, add workers, optimise query
Traffic spikes expose saturation first; latency and errors follow if you do not act on capacity signals.

Which Monitoring Tools Implement the Four Golden Signals?

You can implement golden signals on almost any stack. Pick tools your team will actually run at 2 a.m.

Prometheus and Grafana

Prometheus is the default choice for custom apps. Exporters cover MySQL, Redis, Nginx, and PHP-FPM. Grafana dashboards group the four signals per service. See our full Prometheus and Grafana stack guide for install steps on Ubuntu 22/24.

Netdata and Nagios

Netdata gives fast saturation views with minimal config—good on single-server Laravel hosts. Nagios suits check-based alerting on disk, HTTP, and queue depth. Neither replaces application-level error rates; add app metrics either way.

Cloud and APM options

Managed APM (Datadog, New Relic, Honeycomb) maps golden signals out of the box. Cost runs Rs 15,000–50,000/month (~USD 110–370) for small fleets. Self-hosted Prometheus on a Rs 3,000/month VPS (~USD 22) works when you accept setup time. For Linux system administration clients, I often start with Netdata plus one Prometheus job for the Laravel app.

ToolLatencyTrafficErrorsSaturationBest fit
Prometheus + GrafanaHistogramsCounter ratesStatus labelsNode exporterLaravel, APIs, multi-service
NetdataApp chartsConnection countsLog patternsCPU/RAM/disk nativeSingle VPS, quick wins
Nagios / IcingaPlugin checksRequest pluginsHTTP check failResource pluginsLegacy infra, simple alerts
ZabbixItem historySNMP, agentTrigger on codesTemplatesMixed OS estates

Whatever you choose, store dashboards next to runbooks. A chart without "what to do next" wastes the alert.

How Do You Apply Golden Signals to Laravel and API Applications?

Framework apps need instrumentation at middleware and job boundaries. Server metrics alone miss logic errors and N+1 query latency.

Laravel: middleware and Horizon

Add request timing middleware that records histogram metrics or logs structured timing:

public function handle(Request $request, Closure $next)
{
    $start = microtime(true);
    $response = $next($request);
    $duration = microtime(true) - $start;

    Metrics::observe('http_request_duration_seconds', $duration, [
        'route' => $request->route()?->getName() ?? 'unknown',
        'status' => $response->getStatusCode(),
    ]);

    return $response;
}

For queues, export Horizon metrics: jobs per minute, failed jobs, wait time. Saturation shows up as queue:work processes at 100% CPU with growing Redis list length.

On production Laravel 12 or 13 apps, pair this with complete Prometheus setup and health routes that return JSON for synthetic checks.

REST APIs: per-consumer SLOs

API traffic should break down by client ID or API key hash. One partner sending bulk sync can spike latency for everyone else. Rate limiting—covered in our API rate limiting guide—protects saturation.

Error golden signals for APIs include:

  • HTTP 5xx rate by endpoint version (/api/v1 vs /api/v2).
  • Timeout rate on outbound payment or SMS calls.
  • Webhook delivery failures and retry exhaustion.

Our API monitoring with Prometheus post shows scrape configs for Laravel Sanctum-protected metrics endpoints.

WordPress and WooCommerce

WooCommerce 11.1 on WordPress 7.1 needs different labels. Traffic means add-to-cart and checkout attempts. Latency covers TTFB on shop pages and admin-ajax calls. Errors include failed Stripe or eSewa redirects. Saturation hits PHP-FPM pm.max_children during flash sales—common on florist stores like Petals Qatar.

Laravel Dashboard LayoutLatency Panelp95 by route name/checkout, /api/bookingTarget: p95 < 800msTraffic Panelreq/s and jobs/minHorizon throughputCompare week over weekErrors Panel5xx rate and failed jobsPayment callback failsAlert: > 1% for 5 minSaturation PanelDB pool, Redis memPHP-FPM active workersQueue depth trendOne row per service — same layout everywhere
Standard dashboard layout for The Four Golden Signals of Monitoring on a Laravel production app.

Alert rules that actually page you

Combine signals to reduce noise:

  1. User impact: p99 latency > 2s AND error rate > 0.5% for 5 minutes.
  2. Capacity: saturation > 85% AND traffic within 20% of weekly peak.
  3. Queue backlog: oldest job age > 10 minutes AND traffic above baseline.

Send saturation-only alerts to Slack. Page on-call only when errors or latency confirm user pain. Tune with testing and optimization load tests before festival traffic.

Validate JSON health responses during deploys with a JSON formatter in your runbook—misformatted probes look like outages.

Key Takeaways

  • The Four Golden Signals of Monitoring are latency, traffic, errors, and saturation—measure all four, not just CPU.
  • Use p95 and p99 latency on success paths; averages hide checkout pain.
  • Alert on error rates and sustained latency, not single failed requests.
  • Watch saturation on the real bottleneck: DB pool, queue depth, or PHP-FPM workers.
  • Instrument Laravel and APIs at middleware and job boundaries, not only the server.
  • Correlate signals before paging—traffic spikes explain latency; saturation predicts tomorrow's outage.

People Also Ask

What is the difference between golden signals and RED/USE methods?

RED (Rate, Errors, Duration) applies to request-driven services—it maps directly to traffic, errors, and latency. USE (Utilisation, Saturation, Errors) applies to resources like CPU and disks. Golden signals cover both user-facing and resource views in one framework. Use RED for your API and USE for MySQL host metrics.

How many metrics do you need for golden signal monitoring?

Start with roughly 10–20 time series per service: one latency histogram, request counter by status, and four to six saturation gauges. Add labels for route, queue, or payment provider only where you will alert. Too many labels explode cardinality and slow Prometheus.

Can you use golden signals without Kubernetes?

Yes. The framework predates Kubernetes and works on a single Ubuntu VPS running Apache, PHP-FPM 8.4, and MySQL 8.4. Exporters and app middleware supply the same signals whether you have one server or forty.

When should you alert on saturation alone?

Alert on saturation alone when headroom is low during predictable peaks—disk above 90%, connection pool above 85%, or queue depth monotonically increasing. Otherwise pair saturation with latency or errors to avoid noise during off-peak idle states.

Build Dashboards That Match How Your App Actually Fails

The Four Golden Signals of Monitoring stay relevant because users care about wait time, broken flows, and unavailable pages—not obscure internal gauges. Start with one service, four panels, and three alert rules tied to real SLOs. Expand to APIs, workers, and payment paths once the baseline catches a real incident.

If you want golden-signal dashboards wired into your Laravel app, WooCommerce store, or enterprise application stack, I can help design metrics, alerts, and runbooks that fit your hosting budget. Read more on the Ubuntu server monitoring guide, browse the Notary Nepal portfolio for legal-tech uptime patterns, or contact us to review your current setup.

Frequently Asked Questions

Latency, traffic, errors, and saturation. Latency is how long work takes. Traffic is demand level. Errors are failed requests. Saturation is how close bottleneck resources are to their limits.

Start with roughly 10–20 time series per service: one latency histogram, a request counter by status, and four to six saturation gauges. Add route or queue labels only where you will alert.

Yes. The framework predates Kubernetes and works on a single Ubuntu VPS with Apache, PHP-FPM 8.4, and MySQL 8.4. Exporters and app middleware supply the same signals on one server or many.

RED (Rate, Errors, Duration) applies to request-driven services and maps directly to traffic, errors, and latency. USE (Utilisation, Saturation, Errors) applies to resources like CPU and disks. Golden signals cover both user-facing and resource views in one framework. In practice, use RED for your API endpoints and USE for MySQL host metrics. That split keeps dashboards readable without duplicating charts that say the same thing twice.

Measure at the service boundary—the edge users hit—not deep inside internal calls. Track p50, p95, and p99 using Prometheus histograms, not averages that hide slow checkout tails. Separate success latency from error latency, because a fast 401 skews charts downward. For Laravel background jobs, latency means queue wait plus processing time. Alert on p99 crossing SLO thresholds, not CPU alone. Golden-signal latency starts at the user boundary; internal spans explain where time is lost.

Averages hide slow tail requests that ruin checkout and payment flows. A p99 of two seconds with a p50 of 120 ms means one percent of users wait painfully long—and that one percent often includes payment paths. Graph histogram_quantile(0.99, rate(http_request_duration_seconds_bucket[5m])) in Grafana and set alerts on p99, not means. On production Laravel apps I have instrumented, the mean looked healthy while p99 exposed slow paths that only appeared under real load.

Traffic without latency context misleads—a drop might mean DNS failure, not happy users leaving. Errors without traffic context hides rare but critical paths. Saturation without traffic misses idle resources that still matter at peak. Quantify traffic with HTTP request rates, API requests per consumer, Horizon enqueue versus dequeue rates, and orders per minute on eCommerce paths. Count errors as 5xx rate, payment failures, and exhausted job retries. Watch saturation on the real bottleneck: DB connections, queue depth, or PHP-FPM workers—not CPU alone.

Prometheus and Grafana are the default for custom apps, with exporters for MySQL, Redis, Nginx, and PHP-FPM. Netdata gives fast saturation views on single-server Laravel hosts with minimal config. Nagios and Icinga suit check-based alerting on disk, HTTP, and queue depth. Managed APM options like Datadog, New Relic, and Honeycomb map golden signals out of the box. Zabbix fits mixed OS estates. Neither Netdata nor Nagios replaces application-level error rates—you still need app metrics from middleware or Horizon.

Managed APM on Datadog, New Relic, or Honeycomb runs roughly Rs 15,000–50,000 per month (~USD 110–370) for small fleets. Self-hosted Prometheus on a Rs 3,000 per month VPS (~USD 22) works when you accept setup and maintenance time instead of managed pricing. For Linux administration clients I often start with Netdata plus one Prometheus job for the Laravel application. The trade-off is who responds at 2 a.m.: managed tools cost more monthly but reduce operational burden on small teams.

Alert on saturation alone when headroom is low during predictable peaks—disk above ninety percent, connection pool above eighty-five percent, or queue depth increasing monotonically. Otherwise pair saturation with latency or errors to avoid noise during off-peak idle states when high utilisation does not hurt users. Send saturation-only alerts to Slack. Page on-call only when errors or latency confirm user pain. Traffic spikes expose saturation first; latency and errors follow if you ignore capacity signals.

Instrument at middleware and job boundaries, not only server metrics. Add request timing middleware recording histogram metrics with route and status labels. Export Horizon metrics for jobs per minute, failed jobs, and wait time. Saturation appears as queue workers at full CPU with growing Redis list length. On Laravel 12 or 13 apps, pair this with Prometheus health routes returning JSON for synthetic checks. Server metrics alone miss logic errors and N+1 query latency that show up first in p99 at the middleware layer.

WooCommerce 11.1 on WordPress 7.1 needs commerce-specific labels. Traffic means add-to-cart and checkout attempts, not just page views. Latency covers TTFB on shop pages and admin-ajax calls. Errors include failed Stripe or eSewa redirects and payment callback rejections. Saturation often hits PHP-FPM pm.max_children during flash sales—common on florist stores handling international orders. Server CPU can look comfortable while checkout fails because every FPM worker is busy serving cart fragments.

Combine signals to reduce noise. Page on user impact when p99 latency exceeds two seconds AND error rate exceeds 0.5% for five minutes. Trigger capacity alerts when saturation exceeds eighty-five percent AND traffic sits within twenty percent of weekly peak. Flag queue backlog when oldest job age exceeds ten minutes AND traffic is above baseline. Alert on error rate above one percent for five minutes, not single 500 responses. Tune rules with load tests before festival traffic on Nepal-facing sites where Dashain and Tihar double demand.

Saturation asks how much headroom remains on the limiting resource—not overall CPU alone. CPU at forty percent can still mean saturation if the database connection pool is maxed. Typical alert thresholds include CPU sustained above eighty percent with p95 latency rising, disk above eighty-five percent full or iowait above thirty percent, database connections above eighty percent of max_connections, and queue depth with oldest job age growing fifteen plus minutes. I have seen Laravel apps where Redis memory hit one hundred percent while CPU looked fine, causing random logouts from session evictions.

CPU is a vanity metric when it is not the bottleneck. A Laravel app can show forty percent CPU while MySQL connections or PHP-FPM workers are maxed, or Redis memory is full causing session evictions. Golden signals require latency, traffic, errors, and saturation together—latency and errors tell you users are hurting, traffic gives context for spikes, and saturation predicts the next outage. Alert on user-visible degradation and the real limiting resource, not a green CPU chart that hides a full connection pool.

Share this article

0 Comments

Leave a comment

Your email is not published. Comments appear once they have been read. Sign in to have your details filled in.

Quick Contact Options
Choose how you want to connect me: