Kokil Thapa - Professional Web Developer in Nepal
Freelancer Web Developer in Nepal with 15+ Years of Experience

Kokil Thapa is an experienced full-stack web developer focused on building fast, secure, and scalable web applications. He helps businesses and individuals create SEO-friendly, user-focused digital platforms designed for long-term growth.

Prometheus: Metrics Monitoring Fundamentals

By Kokil Thapa | Last reviewed: September 2026

Production outages rarely announce themselves with a friendly error page. They show up as slow queries, rising queue depth, or disk usage creeping toward full. Prometheus: Metrics Monitoring Fundamentals gives you a repeatable way to watch those signals before users complain. Prometheus is a pull-based time-series database built for operational metrics. It stores numeric samples with labels, lets you query them with PromQL, and feeds alerts through Alertmanager. If you run Ubuntu server monitoring or maintain Laravel apps on shared EC2, this model fits real infrastructure without a heavyweight APM bill.

What is Prometheus and how does metrics monitoring work?

Prometheus watches your systems by asking them for metrics on a schedule. That pull model differs from agents that push logs or traces upstream. A Prometheus server stores samples in its local TSDB. Exporters and instrumented apps expose an HTTP /metrics endpoint. Grafana—or plain PromQL—turns those numbers into dashboards and alerts.

On client projects I maintain with Deployer 7 and GitLab CI, a small Prometheus instance on the same VPC catches PHP-FPM saturation and MySQL slow-query pressure early. The pattern scales from one Ubuntu box to a modest Kubernetes cluster without rewriting your app.

Prometheus Metrics Monitoring ArchitectureNode ExporterCPU, disk, RAMApp Metrics/metrics HTTPMySQL ExporterConnectionsBlackboxHTTP probesPrometheus ServerScrape · TSDB · PromQLGrafanaDashboardsAlertmanagerPager, emailRecording rulesPre-aggregate
Pull-based Prometheus metrics monitoring: exporters expose HTTP endpoints; Prometheus scrapes, stores, queries, and alerts.

The official Prometheus overview documentation describes four core components: the main server, client libraries, push gateway for short jobs, and exporters for third-party data. Alertmanager handles deduplication, grouping, and routing. For a full stack walkthrough, see the companion guide on Prometheus and Grafana as a full monitoring stack.

Core concepts you must internalise

  • Metric name — identifies what you measure, such as http_requests_total.
  • Labels — key/value dimensions like method="GET" and status="500".
  • Sample — a float value plus a millisecond timestamp.
  • Scrape interval — how often Prometheus polls each target, commonly 15s or 30s.
  • Job and instance — Prometheus adds these labels so you can group targets.

Prometheus is strong for infrastructure and request-level RED metrics. It is weaker as a long-term log store or distributed trace backend. Pair it with Loki or Jaeger when you need those signals, as covered in multi-cloud observability for metrics, logs, and traces.

What are the four Prometheus metric types and when should you use each?

Every exported series belongs to one of four types defined in the Prometheus metric types reference. Picking the wrong type produces misleading graphs. A counter plotted as a gauge hides growth; a gauge treated as a counter breaks rate().

Metric typeBehaviourTypical usePromQL hint
CounterMonotonic; only increases or resets to zeroHTTP requests, bytes sent, jobs processedrate(http_requests_total[5m])
GaugeUp or down freelyMemory usage, queue depth, open connectionsDirect value or avg_over_time()
HistogramObservations in configurable bucketsRequest latency, payload sizehistogram_quantile(0.99, ...)
SummaryPre-computed quantiles on clientLegacy SDK quantiles; prefer histograms todaysummary_quantile (rare in modern setups)

Histograms cost more cardinality because each bucket becomes its own time series. Keep bucket boundaries sensible. Five to ten buckets usually beat fifty micro-buckets that explode label cardinality on busy APIs.

Choose the Right Prometheus Metric TypeWhat are you measuring?Always increasing?Use CounterCurrent level?Use GaugeLatency spread?Use HistogramCounter exampleorders_placed_total{status="paid"}rate() gives orders per secondHistogram examplehttp_request_duration_secondsp99 via histogram_quantile()
Prometheus metrics monitoring fundamentals: counters for totals, gauges for current state, histograms for latency distributions.

The RED method fits HTTP services well: Rate (requests per second), Errors (failed requests), and Duration (latency histogram). The USE method suits nodes: Utilisation, Saturation, and Errors. Both map cleanly onto Prometheus types and appear throughout API monitoring with Prometheus and Grafana.

How do you install and configure Prometheus on Ubuntu?

Most production stacks I touch run Ubuntu 22 or 24 with Apache and PHP-FPM. Prometheus installs cleanly alongside them. Download a release from the official Prometheus GitHub releases page, or use your distro package if you prefer managed updates.

Install Prometheus server

  1. Create a dedicated system user: sudo useradd --no-create-home --shell /usr/sbin/nologin prometheus.
  2. Download and extract the latest tarball into /opt/prometheus.
  3. Place config at /etc/prometheus/prometheus.yml and set ownership to the prometheus user.
  4. Enable a systemd unit that starts /opt/prometheus/prometheus --config.file=/etc/prometheus/prometheus.yml.
  5. Open port 9090 only on your admin network or VPN—not the public internet.
# /etc/prometheus/prometheus.yml (minimal production starter)
global:
  scrape_interval: 15s
  evaluation_interval: 15s

scrape_configs:
  - job_name: 'prometheus'
    static_configs:
      - targets: ['localhost:9090']

  - job_name: 'node'
    static_configs:
      - targets: ['10.0.1.12:9100', '10.0.1.13:9100']
        labels:
          env: 'production'

  - job_name: 'laravel-app'
    metrics_path: '/metrics'
    static_configs:
      - targets: ['app.internal:443']
    scheme: https
    tls_config:
      insecure_skip_verify: false

Install node_exporter on each Linux host for CPU, memory, disk, and network series. For MySQL 9.7 or MariaDB 12.3, add mysqld_exporter with a read-only database user. Blackbox exporter probes HTTPS endpoints—useful alongside Laravel health checks and uptime monitoring.

Validate before you trust dashboards

After starting the service, hit http://localhost:9090/targets. Every target should show UP. A common mistake is scraping through a firewall that blocks the Prometheus host. Another is wrong metrics_path when the app mounts metrics elsewhere.

If you manage servers for Nepal clients with tight budgets, this stack often beats paid APM at Rs 15,000–25,000/month (~USD 110–185). You still need someone to tune alerts—that is where Linux system administration and support and maintenance contracts pay off.

Prometheus Scrape and Storage PipelineDiscoverystatic, DNSScrapeHTTP GETParsetext formatRelabeldrop, renameTSDBlocal storeRetention: default 15 days local; remote_write for long archiveBlocks compacted on disk under /var/lib/prometheusPromQL queriesGraph · API · rulesAlert rulesFire to Alertmanager
Each scrape cycle in Prometheus metrics monitoring: discover targets, pull metrics, parse, relabel, and write to the TSDB.

How do you instrument a Laravel or PHP application for Prometheus?

Exporters cover infrastructure. Application metrics need code-level hooks. On Laravel 12 or 13 apps, I expose a protected /metrics route that returns Prometheus text format. Middleware increments counters and observes histograms per request.

Middleware pattern for HTTP metrics

<?php
// app/Http/Middleware/PrometheusMetrics.php
namespace App\Http\Middleware;

use Closure;
use Illuminate\Http\Request;
use Prometheus\CollectorRegistry;
use Prometheus\Storage\Redis;

class PrometheusMetrics
{
    public function handle(Request $request, Closure $next)
    {
        $start = microtime(true);
        $response = $next($request);
        $duration = microtime(true) - $start;

        $registry = new CollectorRegistry(new Redis(['host' => '127.0.0.1']));
        $counter = $registry->getOrRegisterCounter(
            'app', 'http_requests_total', 'HTTP requests',
            ['method', 'route', 'status']
        );
        $counter->inc([
            $request->method(),
            $request->route()?->getName() ?? 'unknown',
            (string) $response->getStatusCode(),
        ]);

        $histogram = $registry->getOrRegisterHistogram(
            'app', 'http_request_duration_seconds', 'Request duration',
            ['method', 'route'],
            [0.05, 0.1, 0.25, 0.5, 1, 2.5, 5]
        );
        $histogram->observe($duration, [
            $request->method(),
            $request->route()?->getName() ?? 'unknown',
        ]);

        return $response;
    }
}

Redis 8.10 as the collector backend avoids race conditions when PHP-FPM runs multiple workers. Protect the metrics route with IP allowlisting or mTLS. Never expose it publicly without auth. On booking platforms like Adventure Third Pole Trek, queue depth and payment callback latency deserve their own gauges and histograms.

Business counters—bookings created, payments confirmed—belong in Prometheus too. They connect technical signals to revenue impact. Keep label cardinality low: use route names, not raw URLs with IDs.

Pushgateway for cron and batch jobs

Short-lived Artisan commands may finish between scrapes. Push metrics to Pushgateway before exit, then let Prometheus scrape the gateway. Do not push long-running service metrics that way. The gateway becomes a single point of stale data if misused.

For deeper setup steps, read monitoring with Prometheus and Grafana: complete setup. Compare with Netdata zero-config monitoring or Nagios for servers if you need a baseline before committing to PromQL.

How does PromQL help you query and alert on metrics?

PromQL is Prometheus's query language. You use it in Grafana panels, alert rules, and the built-in graph UI. Master a handful of functions first. That covers most on-call pages.

Queries every operator should know

# Request rate per route (5-minute window)
sum by (route) (rate(app_http_requests_total[5m]))

# Error ratio for 5xx responses
sum(rate(app_http_requests_total{status=~"5.."}[5m]))
/
sum(rate(app_http_requests_total[5m]))

# p99 latency from histogram
histogram_quantile(0.99,
  sum by (le, route) (rate(app_http_request_duration_seconds_bucket[5m]))
)

# Disk will fill in 4 hours at current write rate
predict_linear(node_filesystem_avail_bytes{mountpoint="/"}[1h], 4*3600) < 0

Alert rules live in files referenced from prometheus.yml. Route firing alerts through Alertmanager for grouping and silencing. A practical walkthrough sits in alerting with Prometheus Alertmanager.

# /etc/prometheus/rules/laravel.yml
groups:
  - name: laravel
    rules:
      - alert: HighErrorRate
        expr: |
          sum(rate(app_http_requests_total{status=~"5.."}[5m]))
          / sum(rate(app_http_requests_total[5m])) > 0.05
        for: 5m
        labels:
          severity: critical
        annotations:
          summary: "5xx error rate above 5%"

      - alert: QueueBacklog
        expr: app_queue_jobs_waiting > 500
        for: 10m
        labels:
          severity: warning

Run promtool check rules /etc/prometheus/rules/laravel.yml before reload. Bad syntax silences alerts silently until someone notices the gap.

Prometheus Alerting WorkflowMetric thresholdrule expr firesPrometheusfor: 5m waitAlertmanagergroup, inhibitNotifySlack, emailGood alert designSymptom-based: "checkout 5xx high" not "CPU > 80%"Runbook link in annotationTest with amtool alert testSilence during planned deploy window
Prometheus metrics monitoring fundamentals extend into Alertmanager for grouped, deduplicated notifications with runbook links.

Cardinality is the silent killer. Every unique label combination creates a new time series. High-cardinality labels—user IDs, session tokens, uncapped URL paths—can OOM a Prometheus server. Cap labels at design time. Use logs for per-user detail instead.

Enterprise teams often wire Prometheus into CI via testing and optimization pipelines. Load tests export request rates that become regression baselines. That pairs well with rate-limit tuning from API rate limiting and abuse prevention.

Key Takeaways

  • Prometheus pulls metrics over HTTP on a fixed interval—install exporters first, then instrument apps.
  • Use counters for totals, gauges for current values, and histograms for latency; avoid high-cardinality labels.
  • Protect /metrics endpoints and restrict port 9090 to admin networks.
  • Learn five PromQL patterns: rate(), error ratios, histogram_quantile(), predict_linear(), and recording rules.
  • Route alerts through Alertmanager with symptom-based rules and linked runbooks.
  • Pair Prometheus with Grafana for dashboards and with logs/traces when metrics alone are not enough.

People Also Ask

Is Prometheus free for production use?

Yes. Prometheus, Alertmanager, node_exporter, and Grafana are open source. Your costs are server compute, storage, and engineer time for setup and on-call tuning. That model suits budget-conscious Nepal deployments where paid APM would run Rs 20,000+/month (~USD 150).

What is the difference between Prometheus and Grafana?

Prometheus collects and stores metrics and evaluates alert rules. Grafana visualises data from Prometheus and other sources. You can query Prometheus directly, but Grafana gives you shareable dashboards most teams expect.

How long does Prometheus keep metrics?

Default local retention is 15 days, controlled by the --storage.tsdb.retention.time flag. For longer history, use remote_write to Thanos, Cortex, or VictoriaMetrics. Plan retention against disk size—each million active series consumes meaningful storage.

Can Prometheus monitor Laravel queues and scheduled jobs?

Yes. Export queue depth and job duration as gauges and histograms from Artisan workers. Short cron tasks should push to Pushgateway or expose a batch summary metric on completion so scrapes do not miss ephemeral jobs.

Build observability that survives production traffic

Understanding Prometheus: Metrics Monitoring Fundamentals means you can see failure before it becomes downtime. Start with node_exporter on every host, add application counters on your busiest Laravel routes, and write three alert rules that page on user-visible symptoms. Expand into the full stack guides on this site, validate JSON alert payloads with the JSON formatter tool, and review shipped work in the portfolio. When you want hands-on help wiring Prometheus into Deployer-managed releases or a multi-app EC2 host, contact us or explore enterprise application development for a monitoring plan built around your stack.

Frequently Asked Questions

Prometheus is a pull-based time-series database built for operational metrics. It polls HTTP endpoints on a fixed scrape interval, stores numeric samples with labels in a local TSDB, and lets you query them with PromQL. Exporters and instrumented apps expose a /metrics endpoint; Alertmanager handles deduplication, grouping, and routing of alerts. On client projects I maintain with Deployer 7 and GitLab CI, a small Prometheus instance on the same VPC catches PHP-FPM saturation and MySQL slow-query pressure before users notice. The model scales from one Ubuntu box to a modest Kubernetes cluster without rewriting your application.

Yes. Prometheus, Alertmanager, node_exporter, and Grafana are open source. Your costs are server compute, storage, and engineer time for setup and on-call alert tuning.

Counters track monotonic totals such as HTTP requests or bytes sent—use rate() to see growth over time. Gauges represent values that move up and down freely, like memory usage, queue depth, or open connections. Histograms bucket observations for request latency and payload size; histogram_quantile() gives you p99 latency. Summaries pre-compute quantiles on the client but add cardinality; prefer histograms in modern setups. Picking the wrong type produces misleading graphs—a counter plotted as a gauge hides growth, and a gauge treated as a counter breaks rate(). Keep histogram buckets sensible: five to ten usually beats fifty micro-buckets on busy APIs.

On Ubuntu 22 or 24, create a dedicated prometheus system user, download the release tarball into /opt/prometheus, and place config at /etc/prometheus/prometheus.yml with ownership set to that user. Enable a systemd unit pointing at the config file, set scrape_interval and evaluation_interval (commonly 15s), and define scrape_configs for prometheus itself, node_exporter targets, and any Laravel apps exposing /metrics over HTTPS. Install node_exporter on each Linux host for CPU, memory, disk, and network series. Open port 9090 only on your admin network or VPN—not the public internet. After starting the service, check http://localhost:9090/targets and confirm every target shows UP before trusting dashboards.

Prometheus collects and stores metrics and evaluates alert rules. Grafana visualises data from Prometheus and other sources. You can query Prometheus directly, but Grafana gives you shareable dashboards most teams expect.

Default local retention is 15 days, controlled by the --storage.tsdb.retention.time flag. For longer history, use remote_write to Thanos, Cortex, or VictoriaMetrics.

Exporters cover infrastructure; application metrics need code-level hooks. On Laravel 12 or 13 apps, expose a protected /metrics route returning Prometheus text format. Middleware increments counters and observes histograms per request, using Redis 8.10 as the collector backend to avoid race conditions when PHP-FPM runs multiple workers. Label by route names, not raw URLs with IDs, to keep cardinality low. Protect the route with IP allowlisting or mTLS—never expose it publicly without auth. Business counters like bookings created or payments confirmed belong here too; they connect technical signals to revenue impact on platforms such as booking systems.

Master a handful of patterns before writing complex alert rules. sum by (route) (rate(app_http_requests_total[5m])) gives request rate per route. Divide 5xx rate by total rate for error ratio. histogram_quantile(0.99, sum by (le, route) (rate(app_http_request_duration_seconds_bucket[5m]))) extracts p99 latency from histograms. predict_linear(node_filesystem_avail_bytes{mountpoint="/"}[1h], 4*3600) < 0 warns when disk will fill at the current write rate. Alert rules live in files referenced from prometheus.yml and route through Alertmanager for grouping and silencing. Always run promtool check rules before reload—bad syntax silences alerts silently until someone notices the gap.

Every unique label combination creates a new time series. High-cardinality labels—user IDs, session tokens, uncapped URL paths—can OOM a Prometheus server. Histograms cost more cardinality because each bucket becomes its own series. Cap labels at design time and use logs for per-user detail instead. This is the silent killer I watch for on busy Laravel APIs where developers instinctively label everything. Enterprise teams sometimes wire Prometheus into CI load tests to catch cardinality regressions before they hit production. Recording rules can pre-aggregate heavy queries, but they do not fix a fundamentally over-labelled metric design.

Short-lived Artisan commands and cron jobs may finish between scrape cycles, so Prometheus never sees their metrics. Push to Pushgateway before exit, then let Prometheus scrape the gateway. Do not push long-running service metrics that way—the gateway becomes a single point of stale data if misused. Pull-based scraping remains the default for always-on exporters, node_exporter, mysqld_exporter, and Laravel /metrics endpoints. Reserve Pushgateway for ephemeral batch work where a final summary counter or gauge is enough. For scheduled jobs that run longer than one scrape interval, expose metrics from the worker process instead.

Prometheus, Alertmanager, node_exporter, and Grafana carry no licence fees. Your real costs are a modest VPS or EC2 instance for the Prometheus server, disk for TSDB storage, and engineer time to install exporters, write alert rules, and respond to pages. For Nepal clients with tight budgets, this stack often beats paid APM at Rs 15,000–25,000/month (~USD 110–185). Paid APM at Rs 20,000+/month (~USD 150) adds convenience but does not remove the need for someone to tune alerts and write runbooks. Linux system administration and support contracts pay off at the alert-tuning stage, not at install time.

node_exporter on every Linux host covers CPU, memory, disk, and network—the baseline for any Ubuntu server monitoring setup. mysqld_exporter with a read-only database user handles MySQL 9.7 or MariaDB 12.3 slow-query and connection pressure. Blackbox exporter probes HTTPS endpoints and pairs well with Laravel health checks for uptime monitoring. Application-level RED metrics come from instrumented Laravel middleware, not from exporters alone. On Deployer-managed EC2 hosts I maintain, this combination catches PHP-FPM saturation, disk creep, and rising 5xx rates without a heavyweight APM bill. Start with node_exporter everywhere, then add mysqld_exporter and app instrumentation on your busiest routes.

Restrict port 9090 to your admin network or VPN—never expose the Prometheus UI to the public internet. Protect Laravel /metrics routes with IP allowlisting or mTLS; unauthenticated public exposure leaks internal request patterns and label data attackers can probe. When scraping HTTPS app targets, set tls_config correctly and avoid insecure_skip_verify unless you have a documented reason on a private network. Alertmanager notification channels should not echo sensitive label values into Slack or email without review. Security here is network placement and access control, not a built-in auth layer—plan firewall rules and scrape topology before you trust production dashboards.

RED stands for Rate, Errors, and Duration—three signals that describe HTTP service health. Rate maps to a counter like http_requests_total queried with rate() over a five-minute window. Errors map to the same counter filtered by status labels such as 5xx, divided by total rate for an error ratio. Duration maps to a histogram like http_request_duration_seconds with histogram_quantile() for p99 latency. RED fits Prometheus counters and histograms cleanly and appears throughout API monitoring setups paired with Grafana. USE—Utilisation, Saturation, Errors—suits node-level monitoring with gauges from node_exporter instead.

Yes. Export queue depth as a gauge and job duration as a histogram from your queue workers so rising backlog triggers alerts before jobs stall. Short cron tasks that finish between scrapes should push a batch summary metric to Pushgateway on completion, or expose a final counter there, so Prometheus does not miss ephemeral runs. On booking platforms, queue depth and payment callback latency deserve dedicated series alongside standard HTTP RED metrics. Instrument workers the same way you instrument web requests: low-cardinality labels, Redis-backed collector registry for PHP-FPM concurrency, and alert rules that page on user-visible symptoms like QueueBacklog above 500 for ten minutes.

Share this article

0 Comments

Leave a comment

Your email is not published. Comments appear once they have been read. Sign in to have your details filled in.

Quick Contact Options
Choose how you want to connect me: