Centralized Logging with the ELK Stack

By Kokil Thapa | Last reviewed: August 2026

Debugging production issues across multiple servers by SSH-ing into individual boxes and grepping text files is unsustainable for any growing web application. Implementing centralized logging with the ELK Stack solves this by aggregating logs from your Laravel applications, Nginx web servers, and system services into a single searchable interface. For developers managing complex architectures like those described in my modern Laravel architecture best practices guide, moving to a unified observability platform is often the turning point between reactive firefighting and proactive monitoring.

How does centralized logging with the ELK Stack actually work?

The ELK Stack (now officially called the Elastic Stack) consists of four distinct components that form a reliable log shipping pipeline. Understanding the specific role of each component prevents the most common architectural mistake: trying to send logs directly from your application to Elasticsearch. In production, you always want a buffering and parsing layer.

• Beats (Filebeat): Lightweight shippers installed on each source server. They read log files, track registry state to prevent duplicates, and forward data reliably even during network blips.
• Logstash: The processing engine. It receives raw log streams, applies filters (grok patterns, date parsing, geoip), enriches data, and handles backpressure buffering before indexing.
• Elasticsearch: A distributed search and analytics engine optimized for log data. It stores structured JSON documents in time-based indices and provides near real-time search capabilities.
• Kibana: The visualization layer. It connects to Elasticsearch to provide dashboards, discovery interfaces, and alerting configuration without requiring raw query knowledge.

This separation of concerns matters because Elasticsearch is optimized for indexing and searching, not for handling bursty TCP connections from hundreds of application servers. Logstash acts as a shock absorber. On projects where I've managed legal-tech portals with unpredictable traffic spikes during Nepali court filing seasons, this buffering layer has prevented log loss during peak loads when direct-to-Elasticsearch setups would have dropped events.

How do you configure Filebeat for Laravel structured logs?

Laravel 12 writes logs in either single-line or daily rotation mode by default. For centralized logging, switch to the daily channel with structured JSON output. Unstructured text logs require expensive grok parsing at ingest time; JSON logs can be indexed directly with minimal transformation.

Configure Laravel for JSON logging

In your config/logging.php, set up a dedicated channel for production that outputs valid JSON:

<?php
// config/logging.php
'channels' => [
    'production_json' => [
        'driver' => 'daily',
        'path' => storage_path('logs/laravel.json'),
        'formatter' => Monolog\Formatter\JsonFormatter::class,
        'formatter_with' => [
            'includeStacktraces' => true,
        ],
        'level' => 'info',
        'days' => 14,
    ],
],

Set LOG_CHANNEL=production_json in your production .env. The JsonFormatter ensures every log entry is a single valid JSON object with timestamp, level, message, context, and extra fields properly escaped. This eliminates multiline log parsing headaches entirely.

Install and configure Filebeat

On Ubuntu 24.04 servers running PHP 8.4 and Laravel 12, install Filebeat from the official Elastic repository rather than Ubuntu's default packages to get the current 8.x release:

wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo gpg --dearmor -o /usr/share/keyrings/elasticsearch-keyring.gpg
echo "deb [signed-by=/usr/share/keyrings/elasticsearch-keyring.gpg] https://artifacts.elastic.co/packages/8.x/apt stable main" | sudo tee /etc/apt/sources.list.d/elastic-8.x.list
sudo apt update && sudo apt install filebeat

Create a dedicated input configuration at /etc/filebeat/conf.d/laravel.yml:

filebeat.inputs:
- type: filestream
  id: laravel-json-logs
  enabled: true
  paths:
    - /var/www/html/storage/logs/laravel-*.json
  parsers:
    - ndjson:
        target: ""
        add_error_key: true
        overwrite_keys: true
  fields:
    app: laravel
    environment: production
  fields_under_root: true

output.logstash:
  hosts: ["logstash.internal:5044"]
  ssl.certificate_authorities: ["/etc/filebeat/certs/ca.crt"]
  ssl.certificate: "/etc/filebeat/certs/filebeat.crt"
  ssl.key: "/etc/filebeat/certs/filebeat.key"

The ndjson parser tells Filebeat to treat each line as a complete JSON document and merge its fields into the event root. Setting target: "" avoids nesting everything under a json key, which simplifies Kibana queries later. Always enable TLS between Filebeat and Logstash in production; log data frequently contains PII, session tokens, or user identifiers that must not traverse the network in plaintext. For teams building secure systems, this aligns with the authentication patterns discussed in guides on building secure authentication systems.

What Logstash pipeline configuration handles high-volume ingestion?

Logstash pipelines are defined in /etc/logstash/conf.d/. A common mistake is writing monolithic pipeline configs that become unmaintainable. Split your pipeline into discrete stages and use conditional processing based on the app field set by Filebeat.

# /etc/logstash/conf.d/01-input-beats.conf
input {
  beats {
    port => 5044
    ssl_enabled => true
    ssl_certificate => "/etc/logstash/certs/logstash.crt"
    ssl_key => "/etc/logstash/certs/logstash.key"
    ssl_certificate_authorities => ["/etc/logstash/certs/ca.crt"]
  }
}

# /etc/logstash/conf.d/02-filter-laravel.conf
filter {
  if [app] == "laravel" {
    date {
      match => ["datetime", "ISO8601"]
      target => "@timestamp"
    }
    
    mutate {
      rename => { "message" => "log.message" }
      rename => { "level_name" => "log.level" }
      rename => { "context.exception.class" => "error.type" }
      rename => { "context.exception.message" => "error.message" }
      rename => { "context.exception.trace" => "error.stack_trace" }
    }
    
    if [context][user_id] {
      mutate {
        rename => { "[context][user_id]" => "user.id" }
      }
    }
  }
}

# /etc/logstash/conf.d/99-output-elasticsearch.conf
output {
  elasticsearch {
    hosts => ["https://es-node-1:9200", "https://es-node-2:9200"]
    index => "logs-%{[app]}-%{+YYYY.MM.dd}"
    user => "${ES_USER}"
    password => "${ES_PASSWORD}"
    ssl_certificate_authorities => "/etc/logstash/certs/ca.crt"
  }
}

The index pattern logs-laravel-2026.08.17 creates daily indices that integrate naturally with Elasticsearch's Index Lifecycle Management (ILM). Daily indices allow you to delete old data efficiently without expensive delete-by-query operations. On eCommerce projects handling thousands of orders per day, I've found that keeping 30 days of hot logs on SSD-backed nodes and rolling older data to cheaper storage via ILM policies keeps infrastructure costs predictable.

Always enable the Dead Letter Queue (DLQ) in logstash.yml by setting dead_letter_queue.enable: true. When a malformed event fails to index, Logstash writes it to the DLQ instead of dropping it silently. I've recovered hours of missing payment webhook logs from DLQ after a schema change broke the date parser — without DLQ, those events would have been permanently lost.

How do you optimize Elasticsearch for log workloads on limited hardware?

Elasticsearch defaults are designed for general-purpose search, not log analytics. Logs are append-only, time-series data with predictable access patterns. Tuning for this workload dramatically reduces resource requirements, which matters when you're self-hosting on Nepal-based VPS providers or budget-constrained EC2 instances.

Apply these settings via an index template so every new daily index inherits them automatically:

PUT _index_template/logs-laravel
{
  "index_patterns": ["logs-laravel-*"],
  "template": {
    "settings": {
      "number_of_replicas": 0,
      "refresh_interval": "30s",
      "translog.durability": "async",
      "codec": "best_compression",
      "mapping.total_fields.limit": 2000
    },
    "mappings": {
      "properties": {
        "@timestamp": { "type": "date" },
        "log.level": { "type": "keyword" },
        "log.message": { "type": "text" },
        "user.id": { "type": "keyword" },
        "error.type": { "type": "keyword" },
        "app": { "type": "keyword" }
      }
    }
  },
  "priority": 200
}

Explicit mappings prevent dynamic mapping explosions. If Laravel logs a new context field like order.metadata.items[].sku, Elasticsearch will dynamically map it. Over months, this creates thousands of fields that slow cluster state updates and consume heap. Define known fields explicitly and set dynamic: false on nested objects you don't need to query.

When should you choose OpenSearch over Elasticsearch in 2026?

The licensing divergence between Elasticsearch and OpenSearch continues to influence infrastructure decisions. Both are viable for centralized logging, but they serve different organizational constraints.

Choose Elasticsearch if you need Elastic Cloud managed hosting, proprietary features like ES|QL or advanced ML anomaly detection, or official commercial support contracts. The SSPL license restricts offering Elasticsearch as a managed service, but self-hosted internal use remains unrestricted.

Choose OpenSearch if you're already on AWS (Amazon OpenSearch Service is fully managed), require Apache 2.0 licensing for compliance reasons, or want to avoid vendor lock-in. OpenSearch forked from Elasticsearch 7.10 and maintains API compatibility for most logging use cases. Filebeat and Logstash work identically with both backends.

For Nepal-based deployments where budget sensitivity is high and AWS isn't mandatory, I typically recommend self-hosted OpenSearch on local VPS infrastructure. The Apache 2.0 license removes any future licensing uncertainty, and the community-maintained Docker images simplify deployment on modest hardware. Teams already invested in the Elastic ecosystem or requiring specific proprietary features should stick with Elasticsearch — the migration cost between the two is non-trivial once you've built custom dashboards and alerts.

Practical Next Steps for Production Deployment

Centralized logging with the ELK Stack transforms how you diagnose production issues, but only if implemented correctly from the start. Begin with structured JSON logging in your application before touching infrastructure — unstructured logs create technical debt that compounds over time. Deploy Filebeat with TLS, configure Logstash with DLQ enabled, and apply log-optimized index templates before ingesting your first production event. Start with a single-node Elasticsearch or OpenSearch instance; premature clustering adds operational complexity without proportional benefit until you exceed ~50GB of daily log volume.

If you're evaluating observability infrastructure for a Laravel application or need help designing a logging architecture that scales with your business, reach out to discuss your specific requirements. Proper logging setup pays for itself the first time you resolve a production incident in minutes instead of hours.