Log Aggregation with Loki and Grafana

By Kokil Thapa | Last reviewed: September 2026

Production Laravel apps write logs to disk, PHP-FPM, Nginx, and queue workers. When a payment callback fails at 2 a.m., you need one place to search across every server. Log aggregation with Loki and Grafana solves that without running Elasticsearch. Loki stores log streams indexed by labels—similar to Prometheus metrics—and Grafana gives you LogQL search, dashboards, and alerts. On real client projects I maintain on Ubuntu with Linux system administration, this stack has replaced heavy ELK setups that cost more RAM than the app itself.

What Is Log Aggregation with Loki and Grafana?

Loki is Grafana Labs' log aggregation system. It does not index every word in every log line. It indexes labels—host, app, environment, job—and stores compressed log chunks in object storage or local disk. Grafana connects to Loki as a data source and renders results in Explore, dashboards, and alert rules.

That design choice matters for small teams. A three-node ELK cluster can consume 8–16 GB RAM before you ingest a single Laravel stack trace. Loki on the same VPS often runs comfortably with 2–4 GB RAM for moderate traffic. You trade instant full-text search on arbitrary strings for fast label-filtered queries—which is exactly how you debug production incidents anyway.

The mental model mirrors Prometheus and Grafana for metrics. You already label time series by job and instance. Loki uses the same pattern for logs. If your metrics stack exists, adding Loki feels familiar rather than alien.

How Do You Install Loki and Grafana on Ubuntu?

Docker Compose is the fastest path for a first deployment. Production teams on bare metal often prefer systemd units—same binaries, easier integration with existing backup scripts. Both approaches work on Ubuntu 22.04 or 24.04 servers I run for ongoing support and maintenance contracts.

Step 1: Create a Docker Compose stack

Create /opt/observability/docker-compose.yml with Loki, Grafana, and Promtail services. Pin image tags instead of using latest—surprise upgrades break LogQL dashboards.

services:
  loki:
    image: grafana/loki:3.2.1
    ports:
      - "3100:3100"
    volumes:
      - ./loki-config.yaml:/etc/loki/local-config.yaml
      - loki-data:/loki
    command: -config.file=/etc/loki/local-config.yaml

  grafana:
    image: grafana/grafana:11.4.0
    ports:
      - "3000:3000"
    volumes:
      - grafana-data:/var/lib/grafana
    environment:
      - GF_SECURITY_ADMIN_PASSWORD=change-me

  promtail:
    image: grafana/promtail:3.2.1
    volumes:
      - ./promtail-config.yaml:/etc/promtail/config.yml
      - /var/log:/var/log:ro
      - /var/www:/var/www:ro
    command: -config.file=/etc/promtail/config.yml

volumes:
  loki-data:
  grafana-data:

Step 2: Configure Loki storage

For a single-server setup, filesystem storage is fine. For multi-server or longer retention, point Loki at S3-compatible object storage. The official Loki configuration reference documents every block.

auth_enabled: false

server:
  http_listen_port: 3100

common:
  path_prefix: /loki
  storage:
    filesystem:
      chunks_directory: /loki/chunks
      rules_directory: /loki/rules
  replication_factor: 1
  ring:
    kvstore:
      store: inmemory

schema_config:
  configs:
    - from: 2024-01-01
      store: tsdb
      object_store: filesystem
      schema: v13
      index:
        prefix: index_
        period: 24h

limits_config:
  retention_period: 744h
  ingestion_rate_mb: 10
  ingestion_burst_size_mb: 20

Step 3: Start the stack and add the data source

1. Run docker compose up -d from /opt/observability.
2. Open Grafana at port 3000 and log in.
3. Go to Connections → Data sources → Add Loki.
4. Set URL to http://loki:3100 inside Docker, or http://127.0.0.1:3100 on the host.
5. Save and test—the green check confirms connectivity.

Lock Grafana behind HTTPS with Nginx and Let's Encrypt. Never expose port 3000 publicly without authentication. Your logs contain session tokens, SQL fragments, and payment references.

How Do You Configure Promtail for Laravel and Nginx Logs?

Promtail tails files, attaches labels, and pushes log lines to Loki. The label schema you choose here determines whether LogQL queries stay fast or grind to a halt. Treat labels like database indexes—few, stable, high cardinality only when you accept the cost.

For a Laravel 13 app on PHP 8.3, I ship three primary streams on each app server. See also log aggregation for small teams for a lighter-weight starting point.

server:
  http_listen_port: 9080

positions:
  filename: /tmp/positions.yaml

clients:
  - url: http://loki:3100/loki/api/v1/push

scrape_configs:
  - job_name: laravel
    static_configs:
      - targets: [localhost]
        labels:
          job: laravel
          app: nepal-gift-card
          env: production
          __path__: /var/www/app/storage/logs/laravel.log

  - job_name: nginx-access
    static_configs:
      - targets: [localhost]
        labels:
          job: nginx-access
          app: nepal-gift-card
          env: production
          __path__: /var/log/nginx/access.log

  - job_name: nginx-error
    static_configs:
      - targets: [localhost]
        labels:
          job: nginx-error
          app: nepal-gift-card
          env: production
          __path__: /var/log/nginx/error.log

Parse Laravel JSON logs with pipeline stages

If your config/logging.php channel uses JSON formatter, add a json pipeline stage. That extracts level and channel as labels or structured metadata.

    pipeline_stages:
      - json:
          expressions:
            level: level
            message: message
            context: context
      - labels:
          level:

Do not label every user_id or request_id unless you enjoy index explosions. Filter those inside LogQL with line filters after narrowing by app and env. This mirrors advice in debugging Laravel in production with logs—structured context belongs in the line, not always in labels.

How Do You Query Logs with LogQL in Grafana?

LogQL combines label selectors—like PromQL—with line filters and parsers. Start every investigation in Grafana Explore. Pick the Loki data source, set a time range, and build queries incrementally.

Common patterns for PHP and Laravel stacks:

• {job="laravel", env="production"} |= "ERROR" — errors only, production Laravel.
• {job="laravel"} | json | level="error" — JSON-formatted Laravel logs.
• {job="nginx-access"} | json | status >= 500 — server errors from access logs.
• rate({job="laravel"}[5m]) — log volume metric for alerting.
• {job="laravel"} |= "eSewa" |= "callback" — payment gateway debugging.

Save useful queries as dashboard panels. A single-row dashboard with error rate, 5xx count, and recent exceptions beats scrolling raw files over SSH. For panel design patterns, see Grafana dashboards: a practical guide.

Build alert rules from log metrics

Loki can emit metrics from log streams. Create a Grafana alert when error rate exceeds a threshold for five minutes.

sum(rate({job="laravel", level="error"}[5m])) by (app) > 0.5

Route alerts to Slack, email, or PagerDuty—the same channels you use for Prometheus Alertmanager rules. Pair metric alerts with log alerts so you catch both slow burns and sudden spikes.

How Does Loki Compare to ELK and Cloud Logging?

Teams often ask whether Loki replaces Elasticsearch or CloudWatch. The honest answer: it depends on query patterns and team size. Loki wins on cost and operational simplicity. ELK wins on ad-hoc full-text analytics across billions of lines.

For a booking platform like Adventure Third Pole Trek, Loki plus existing Prometheus covers 90% of incident response. You still need application-level audit trails—Spatie Activity Log or database events—not raw Nginx lines. Read Laravel activity log with Spatie for that layer.

Choosing between Fluent Bit and Promtail? Fluentd vs Fluent Bit compares agents when you already run Kubernetes or mixed stacks. Grafana Alloy is the long-term unified agent replacing Promtail, but Promtail remains stable and well documented in 2026.

What Production Mistakes Break Loki Deployments?

I've seen the same failures on sister sites sharing a Deployer 7 pipeline. Logs worked locally. Central search failed silently. These fixes prevent weekend fire drills.

Label cardinality explosions

Adding request_id or user_id as a Loki label creates a unique stream per request. Loki's index grows until ingestion slows and queries time out. Keep labels to tens of values, not thousands. Put dynamic IDs in the log line and filter with |= "req-abc123".

Clock skew and missing logs

Loki rejects entries too far in the future or past. Sync NTP on every app server. Promtail's positions.yaml tracks read offsets—deleting it without care re-ships entire files and duplicates history.

Retention without disk planning

Default retention of 30 days on a 40 GB VPS fills disk fast when queue workers log verbosely. Set retention_period in Loki config and monitor /loki mount usage. Rotate Laravel log channels to daily files locally too.

Shipping secrets and PII

Laravel logs often dump request payloads including passwords or card fragments. Add a Promtail drop stage or redact in the app before write. Central logs become a compliance liability if anyone with Grafana access can search them. Use Grafana RBAC and audit who has Explore permissions.

After every deploy, trigger a test log line and confirm it appears in Explore within 30 seconds. I bake this into testing and optimization checklists alongside smoke tests and queue health checks. For deeper incident workflows, AI-powered log analysis covers triage patterns once central logs exist.

Multi-service setups benefit from unified observability. Read multi-cloud observability for metrics, logs, and traces if you run apps across regions. Nginx access logs also feed SEO log file analysis—export subsets or query Loki for crawl anomalies.

Validate JSON log payloads with a JSON formatter before enabling pipeline stages. Broken JSON silently skips parsing stages and leaves you with raw strings.

Key Takeaways

• Log aggregation with Loki and Grafana stores logs by labels, not full text—design a small, stable label schema before shipping.
• Run Promtail on each app server to tail Laravel, Nginx, and PHP-FPM files; use JSON pipeline stages for structured channels.
• Query with LogQL in Grafana Explore: narrow by labels first, then filter lines with |= or JSON parsers.
• Set retention limits and monitor disk—verbose queue logs fill a VPS faster than metrics databases.
• Never label high-cardinality fields like user IDs; never ship unredacted payment or auth payloads.
• Pair Loki log alerts with existing Prometheus metric alerts for complete incident coverage.

People Also Ask

Is Loki free to use?

Yes. Loki, Promtail, and Grafana are open-source with Apache 2.0 licensing. You pay for the server, storage, and your time—not per-gigabyte ingest fees like many cloud log services. Grafana Cloud offers a hosted Loki tier if self-hosting is not viable.

Can Loki replace Elasticsearch for all logging?

No—not if you need heavy ad-hoc full-text search across unstructured data at massive scale. Loki excels when you know your label dimensions and search within filtered streams. Security teams doing broad forensic text search often still need ELK or a SIEM.

What is the difference between Promtail and Grafana Alloy?

Promtail is the mature log-shipping agent built for Loki. Grafana Alloy is the newer unified collector for logs, metrics, and traces. Alloy is the long-term direction from Grafana Labs, but Promtail configs remain valid and widely deployed through 2026.

How do I ship logs from Docker containers to Loki?

Mount the Docker log directory or use Docker logging drivers pointing to Promtail. Alternatively, run Promtail as a sidecar container with shared volumes. Label containers by compose service name so LogQL can filter {container="app"} per service.

Deploy Central Logging on Your Stack

Log aggregation with Loki and Grafana gives you one search bar for every Laravel exception, Nginx 502, and payment callback across your fleet. The setup takes an afternoon on a single VPS. The payoff arrives the first time you trace a production failure without SSH-ing into three servers. If you want this wired into an existing enterprise application or legal-tech portal with proper retention and access controls, contact us for a scoped implementation. For background on the broader monitoring picture, start with Prometheus and Grafana complete setup and extend from metrics to logs.