API Monitoring with Prometheus and Grafana

By Kokil Thapa | Last reviewed: September 2026

Your API can return 200 OK while latency quietly doubles and error rates creep up. API monitoring with Prometheus and Grafana fixes that gap by collecting time-series metrics from your application and infrastructure, storing them in Prometheus, and visualising trends in Grafana before users open support tickets. On production Laravel and payment-gateway integrations I've maintained, scrape-based monitoring caught webhook timeouts and queue backlogs that log tailing alone missed. This guide walks through instrumentation, scrape config, RED dashboards, and alerts you can deploy on Ubuntu with PHP 8.3+ or Laravel 12/13.

For teams shipping REST API development in Nepal or abroad, Prometheus fits pull-model observability well. You control retention, queries, and alert rules without locking into a single SaaS vendor. Grafana sits on top as the visual layer. Together they answer the questions operators actually ask: which endpoint is slow, which tenant hits rate limits, and did the last deploy change p95 latency.

What Is API Monitoring with Prometheus and Grafana?

Prometheus is a time-series database built around periodic HTTP scrapes. Your API (or a sidecar exporter) exposes metrics in OpenMetrics text format. Prometheus pulls those samples every 15–60 seconds and stores them with labels such as method, route, and status. Grafana queries Prometheus with PromQL and renders panels, heatmaps, and tables.

Grafana does not collect data itself in this stack. It reads from Prometheus (and optionally Loki or Tempo if you add logs and traces later). That separation keeps concerns clean: Prometheus owns storage and alerting rules; Grafana owns presentation.

The stack differs from passive log aggregation. Metrics are numeric samples you can aggregate, rate, and alert on in milliseconds. That makes it ideal for SLO tracking and capacity planning on APIs that handle bookings, payments, or document uploads.

Core components you will deploy

• Instrumented API — exposes counters, gauges, and histograms at /metrics.
• Prometheus server — scrapes targets, evaluates recording rules, fires alerts.
• Grafana — dashboards, variables, annotations for deploy markers.
• Alertmanager — routes notifications to Slack, email, or PagerDuty. See the dedicated Alertmanager alerting guide for routing trees.
• Exporters — optional sidecars for Nginx, MySQL 9.7, or Redis 8.10 when the app cannot expose infra metrics directly.

How Do You Instrument a REST API for Prometheus?

Instrumentation means emitting metrics at request boundaries. For HTTP APIs, the RED method is the standard starting point: Rate (requests per second), Errors (failed requests), and Duration (latency distribution). Google SRE calls these part of the four golden signals; for APIs, RED maps cleanly onto route-level labels.

In PHP and Laravel applications, the promphp/prometheus_client_php library registers a registry and renders OpenMetrics output. Wrap middleware around your API routes to observe each request.

Laravel middleware example

<?php
namespace App\Http\Middleware;

use Closure;
use Prometheus\CollectorRegistry;
use Prometheus\Storage\Redis;

class PrometheusMetrics
{
    public function handle($request, Closure $next)
    {
        $registry = new CollectorRegistry(new Redis([
            'host' => env('REDIS_HOST', '127.0.0.1'),
        ]));

        $counter = $registry->getOrRegisterCounter(
            'api', 'http_requests_total',
            'Total HTTP requests',
            ['method', 'route', 'status']
        );

        $histogram = $registry->getOrRegisterHistogram(
            'api', 'http_request_duration_seconds',
            'Request latency',
            ['method', 'route'],
            [0.05, 0.1, 0.25, 0.5, 1, 2.5, 5, 10]
        );

        $start = microtime(true);
        $response = $next($request);
        $duration = microtime(true) - $start;

        $route = $request->route()?->getName() ?? 'unknown';
        $status = (string) $response->getStatusCode();

        $counter->inc([$request->method(), $route, $status]);
        $histogram->observe($duration, [$request->method(), $route]);

        return $response;
    }
}

Register a dedicated metrics route outside normal auth middleware. Protect it with network ACLs or basic auth so metric endpoints are not public. Pair this pattern with Laravel API best practices so labels stay bounded — never use raw user IDs as label values.

Metrics route controller

<?php
use Prometheus\CollectorRegistry;
use Prometheus\RenderTextFormat;
use Prometheus\Storage\Redis;

Route::get('/metrics', function () {
    $registry = new CollectorRegistry(new Redis(['host' => '127.0.0.1']));
    $renderer = new RenderTextFormat();
    return response($renderer->render($registry->getMetricFamilySamples()))
        ->header('Content-Type', RenderTextFormat::MIME_TYPE);
})->middleware('metrics.ip_allowlist');

On booking APIs I've worked on, I also track business counters: payment_callbacks_total, queue_jobs_failed_total, and webhook_retries_total. Those catch integration failures that still return HTTP 200 to the caller. Validate payloads with your JSON formatter tool during development, but production health belongs in Prometheus counters.

How Do You Configure Prometheus to Scrape API Metrics?

Prometheus reads targets from prometheus.yml. Each job defines scrape interval, path, and labels applied to every series from that target. Keep scrape intervals at 15s for APIs under active development and 30–60s for stable production unless you have a clear reason to go faster.

global:
  scrape_interval: 15s
  evaluation_interval: 15s

scrape_configs:
  - job_name: 'laravel-api'
    metrics_path: /metrics
    static_configs:
      - targets: ['api.internal:8080']
        labels:
          env: production
          service: bookings-api

  - job_name: 'nginx-exporter'
    static_configs:
      - targets: ['nginx-exporter:9113']

  - job_name: 'redis-exporter'
    static_configs:
      - targets: ['redis-exporter:9121']

rule_files:
  - /etc/prometheus/rules/api-alerts.yml

alerting:
  alertmanagers:
    - static_configs:
        - targets: ['alertmanager:9093']

For multi-host setups, file-based service discovery or Docker labels scales better than hard-coded IPs. On Ubuntu 22/24 servers I manage, Prometheus runs as a systemd unit with config deployed through the same GitLab CI pipeline used for application releases. That mirrors the approach in our Ubuntu server monitoring guide.

Verify scrapes before building dashboards

1. Open http://prometheus:9090/targets and confirm state is UP.
2. Run up{job="laravel-api"} in the Prometheus graph UI.
3. Query rate(api_http_requests_total[5m]) after generating traffic.
4. Check for label cardinality with count by (route) (api_http_requests_total).

If targets show DOWN, check firewall rules between Prometheus and the API host. The scrape is an inbound HTTP GET from Prometheus to your app. UFW must allow that path on the metrics port. Official reference: Prometheus configuration documentation.

Which Grafana Dashboards Matter Most for API Health?

Start with one overview dashboard and one drill-down per critical service. The overview shows global RPS, error percentage, and p95 latency. Drill-downs filter by route, status code, or deployment version using Grafana template variables.

Import community dashboard ID 10826 as a starting skeleton, then replace generic node metrics with your api_http_* series. Our Grafana dashboards practical guide covers variables, units, and annotation layers for deploy markers.

Essential PromQL queries for API panels

# Request rate by route
sum by (route) (rate(api_http_requests_total[5m]))

# Error ratio (5xx)
sum(rate(api_http_requests_total{status=~"5.."}[5m]))
  / sum(rate(api_http_requests_total[5m]))

# p95 latency
histogram_quantile(0.95,
  sum by (le, route) (rate(api_http_request_duration_seconds_bucket[5m]))
)

# Apdex-style threshold (500ms)
sum(rate(api_http_request_duration_seconds_bucket{le="0.5"}[5m]))
  / sum(rate(api_http_request_duration_seconds_count[5m]))

Add rows for dependency health: MySQL slow queries, Redis memory, queue depth. On a legal-tech client portal with document uploads, correlating API latency with disk I/O caught a storage bottleneck that application logs masked. Cross-link infra panels with the full Prometheus and Grafana monitoring stack article for node-level exporters.

For small teams in Nepal running APIs on a single EC2 or local VPS (Rs 3,000–8,000/month hosting, ~USD 22–60), self-hosted Prometheus on the same network segment is often cheaper than SaaS APM at moderate traffic. Budget Alertmanager notification channels before you budget Grafana Enterprise plugins.

How Should You Alert on API Metrics Without Alert Fatigue?

Alerts belong in Prometheus rule files, not Grafana UI alone. Grafana can alert, but centralising in Prometheus keeps evaluation co-located with the data and feeds Alertmanager for deduplication and silencing.

groups:
  - name: api-alerts
    rules:
      - alert: ApiHighErrorRate
        expr: |
          sum(rate(api_http_requests_total{status=~"5.."}[5m]))
            / sum(rate(api_http_requests_total[5m])) > 0.05
        for: 10m
        labels:
          severity: critical
        annotations:
          summary: "API error rate above 5% for 10 minutes"

      - alert: ApiLatencyP95High
        expr: |
          histogram_quantile(0.95,
            sum(rate(api_http_request_duration_seconds_bucket[5m]))
          ) > 2
        for: 15m
        labels:
          severity: warning
        annotations:
          summary: "API p95 latency above 2 seconds"

      - alert: ApiMetricsTargetDown
        expr: up{job="laravel-api"} == 0
        for: 2m
        labels:
          severity: critical
        annotations:
          summary: "Prometheus cannot scrape API metrics"

Use the for clause to avoid paging on single blips. Pair threshold alerts with SLO burn-rate rules when traffic justifies the complexity. Route critical payment endpoints — like those described in Laravel rate limiting and throttling — to a separate Alertmanager receiver with shorter for windows.

Alert routing principles

• Warning → Slack channel; no phone call.
• Critical → on-call rotation with acknowledgement timeout.
• Info → ticket queue for next business day.
• Silence alerts during planned deploys using Alertmanager silences, not by disabling rules.

Grafana alerting docs are useful for panel thresholds, but production API SLOs should live in version-controlled YAML beside your Prometheus config. Reference: Grafana alerting documentation.

What Production Gotchas Break API Monitoring Stacks?

Most failures I see are operational, not PromQL syntax errors. Card label explosion is the silent killer. Every unique URL path as a label creates a new time series. Use named routes, not $request->path(), and cap cardinality below a few thousand active series per job.

Second: scraping through a load balancer without sticky awareness double-counts if each backend exposes independent counters. Either scrape one canary instance or export aggregated metrics from the gateway layer. If you terminate TLS at Kong or another API gateway, expose gateway-level RED metrics there and treat app metrics as a secondary signal.

Third: histogram bucket boundaries must match your SLO thresholds. Default Go buckets top out at 10 seconds. PHP APIs serving PDF generation or image transforms need buckets up to 30–60 seconds or your p99 flatlines at the bucket ceiling.

Fourth: secure the metrics endpoint. Unauthenticated /metrics leaks route names and traffic patterns. Restrict by IP, mTLS, or internal VPC only. This sits alongside proper API authentication with Passport or Sanctum on public routes — different surfaces, same security mindset.

Fifth: after deploys, reload PHP-FPM so opcache picks up middleware changes. I've debugged "metrics stuck at zero" that was simply stale PHP workers after a symlink swap. The same deploy pipeline notes apply in Linux system administration engagements.

Gateway and multi-service patterns

When APIs sit behind Traefik or Nginx, run nginx-prometheus-exporter or enable Traefik's built-in metrics endpoint as a separate scrape job. Correlate edge 502 rates with application 500 rates to distinguish proxy misconfig from code regressions. For design context, read REST API design best practices before you label routes inconsistently across services.

On Adventure Third Pole Trek, a Laravel + Livewire booking API benefited from queue-depth metrics alongside HTTP RED. Failed job counters predicted checkout failures minutes before HTTP error rates moved. That pattern applies to any async API processing payments or notifications.

If you need a lighter host baseline while building Prometheus coverage, Netdata zero-config monitoring fills gaps on CPU and disk. Migrate critical SLIs to Prometheus once histogram instrumentation lands.

Key Takeaways

• Expose OpenMetrics at a protected /metrics endpoint and instrument every API route with RED counters and histograms.
• Keep Prometheus labels low-cardinality: use route names, never user IDs or full URL paths.
• Build Grafana overview plus drill-down dashboards with p95 latency and error-ratio panels tied to deploy annotations.
• Define Prometheus alert rules with for durations and route severity-specific Alertmanager receivers.
• Scrape through internal networks only, reload PHP-FPM after deploys, and tune histogram buckets to your real SLOs.
• Complement HTTP metrics with business counters for webhooks, queues, and third-party API callbacks.

People Also Ask

Can Prometheus monitor APIs behind Kubernetes?

Yes. Use Kubernetes pod annotations prometheus.io/scrape: "true" and prometheus.io/port so Prometheus discovers pods automatically. ServiceMonitor CRDs with the Prometheus Operator are the managed alternative on clusters. The scrape model stays identical: HTTP pull against /metrics.

Do you need Grafana if Prometheus already has a UI?

Prometheus graphs single queries adequately for debugging. Grafana adds dashboard variables, team sharing, annotations, and mixed data sources. For any API watched by non-engineers — product owners, support leads — Grafana is worth the extra container.

How is API monitoring different from uptime checks?

Uptime probes hit one synthetic URL every few minutes. Prometheus scrapes real request metrics from production traffic continuously. You see per-route latency distributions and error mixes that a binary up/down check cannot surface.

What metrics should a Laravel API export first?

Start with http_requests_total, http_request_duration_seconds, and queue_jobs_failed_total. Add payment callback and external API latency histograms next if your app integrates gateways like eSewa, Khalti, or Stripe.

Ship API Monitoring You Can Act On

API monitoring with Prometheus and Grafana turns opaque production behaviour into graphs and alerts you can trust. Instrument with RED metrics, scrape on a steady interval, dashboard p95 and error ratio before launch day, and wire Alertmanager before the first incident — not after. If you want help instrumenting a Laravel API, hardening scrape paths, or folding monitoring into your deploy pipeline, contact us or explore testing and optimization services and ongoing support and maintenance. For a broader platform view, see building RESTful APIs with Laravel and custom software development options.