September 12, 2026
12 min read
By Kokil Thapa | Last reviewed: September 2026
Your production database changed five minutes ago. A customer updated an order in your Laravel app. Your search index still shows the old address. Your analytics warehouse runs a nightly batch job. That gap is exactly why teams adopt Change Data Capture with Debezium. Debezium reads database transaction logs and publishes every insert, update, and delete as structured events. You stop polling tables every few minutes. Downstream systems react in near real time instead. If you already run data pipelines for DevOps and analytics, CDC is the missing link between OLTP and everything else.
Quick answer: Change Data Capture with Debezium uses Kafka Connect connectors to tail database transaction logs—MySQL binlog, PostgreSQL WAL—and emit row-level change events to Kafka topics. Consumers sync search indexes, warehouses, caches, and microservices without heavy polling or dual writes.
What is Change Data Capture with Debezium and how does it work?
Change Data Capture (CDC) captures every row-level change from a source database as it happens. Debezium is an open-source CDC platform built on Apache Kafka Connect. It does not query your tables on a schedule. It reads the database's own replication log.
MySQL uses the binary log (binlog). PostgreSQL uses logical decoding of the write-ahead log (WAL). Debezium connectors attach to those logs, parse transactions, and serialize each change into a Kafka message. The message includes the operation type, the affected table, and before/after row values.
On a production Laravel application backed by MySQL 8.4 LTS or PostgreSQL 18, this pattern keeps auxiliary systems honest. I've seen booking platforms where order status lived in MySQL but reporting ran on stale snapshots. CDC fixed the lag without rewriting the core app.
Each Debezium event follows a predictable envelope. The op field marks create, update, delete, or read (during snapshot). The before and after blocks carry column values. Metadata includes source database, table, transaction timestamp, and log position. That structure makes it easy to route events in Airflow-orchestrated pipelines or custom consumers.
Why log-based CDC beats polling
Polling runs SELECT * FROM orders WHERE updated_at > ? on a timer. It misses hard deletes. It adds read load to production. It races when two rows change in the same second. Log-based CDC avoids all three problems because the database already records every mutation.
Dual writes—your app writes to MySQL and Elasticsearch in the same request—fail silently when one side errors. CDC keeps the source of truth in one place. Everything else catches up from the log.
How do you set up Change Data Capture with Debezium for MySQL or PostgreSQL?
Debezium runs as a Kafka Connect connector. You need Kafka, a Connect worker, and a properly configured source database. For a first deployment, Docker Compose is fine. Production setups use managed Kafka or self-hosted clusters with monitoring.
MySQL prerequisites
Enable row-based binlog and create a dedicated replication user. On MySQL 8.4 LTS, add this to my.cnf:
[mysqld]
server-id=1
log_bin=mysql-bin
binlog_format=ROW
binlog_row_image=FULL
gtid_mode=ON
enforce_gtid_consistency=ON
Create the Debezium user with replication privileges:
CREATE USER 'debezium'@'%' IDENTIFIED BY 'strong_password_here';
GRANT SELECT, RELOAD, SHOW DATABASES, REPLICATION SLAVE, REPLICATION CLIENT
ON *.* TO 'debezium'@'%';
FLUSH PRIVILEGES;
PostgreSQL prerequisites
PostgreSQL 18 supports logical replication natively. Set wal_level=logical in postgresql.conf. Create a publication and a user with replication rights:
ALTER SYSTEM SET wal_level = logical;
CREATE USER debezium WITH REPLICATION LOGIN PASSWORD 'strong_password_here';
GRANT SELECT ON ALL TABLES IN SCHEMA public TO debezium;
Restart PostgreSQL after changing WAL level. Logical slots consume disk if consumers lag, so monitor slot lag daily.
Register the Debezium connector
Post a connector configuration to the Kafka Connect REST API. This example captures a Laravel app's orders and customers tables:
curl -X POST http://connect:8083/connectors \
-H "Content-Type: application/json" \
-d '{
"name": "inventory-connector",
"config": {
"connector.class": "io.debezium.connector.mysql.MySqlConnector",
"database.hostname": "db.example.com",
"database.port": "3306",
"database.user": "debezium",
"database.password": "strong_password_here",
"database.server.id": "184054",
"topic.prefix": "shop",
"database.include.list": "shop_db",
"table.include.list": "shop_db.orders,shop_db.customers",
"schema.history.internal.kafka.bootstrap.servers": "kafka:9092",
"schema.history.internal.kafka.topic": "schema-changes.shop",
"snapshot.mode": "initial"
}
}'
Kafka topics appear as shop.shop_db.orders and shop.shop_db.customers. Each message is a JSON or Avro-encoded change event. Use the JSON formatter tool to inspect payloads during development.
- Verify binlog or WAL settings and restart the database if needed.
- Create a least-privilege replication user scoped to required schemas.
- Deploy Kafka and Kafka Connect with the Debezium connector JAR installed.
- Register the connector via REST API and confirm
RUNNINGstatus. - Build a consumer that handles snapshot READ events, then live changes.
- Set alerts on consumer lag and replication slot age.
Official setup details live in the Debezium MySQL connector documentation and the PostgreSQL connector guide. Cross-check Kafka Connect settings against the Apache Kafka Connect documentation.
How does Debezium compare to polling, triggers, and batch ETL?
Teams evaluate four common sync patterns. Each trades complexity, latency, and operational risk differently. The table below reflects what I recommend on client projects where MySQL or PostgreSQL is the system of record.
| Approach | Latency | Source DB Load | Captures Deletes | Ops Complexity |
|---|---|---|---|---|
| Timestamp polling | Minutes to hours | High (repeated SELECTs) | No | Low |
| DB triggers → queue table | Seconds | Medium (write amplification) | Yes | Medium |
| Nightly batch ETL | Hours | Low during window | Depends on job | Low |
| Debezium log-based CDC | Sub-second to seconds | Low (log tailing) | Yes | High |
Polling works for low-stakes reporting where ten-minute staleness is acceptable. Triggers pollute application schemas and slow writes. Batch ETL is fine for finance close workflows. Debezium earns its place when search, cache, and analytics must track production within seconds.
On legal-tech portals I've maintained, document metadata lives in MySQL while full-text search runs elsewhere. Polling updated_at columns broke every time a row was deleted. CDC kept the search index aligned without touching Laravel controllers. Similar patterns appear in client portal projects with document workflows.
How do you handle schema changes and production gotchas with Debezium?
Schema evolution is the top production headache. A developer adds a column in a Laravel migration. Debezium must learn the new shape before consumers break. Debezium stores schema history in a dedicated Kafka topic. When DDL runs, the connector emits a schema-change event.
Snapshot modes explained
The snapshot.mode setting controls initial sync behaviour:
initial— full snapshot, then stream. Default for new connectors.initial_only— snapshot without streaming. Useful for one-off exports.never— streaming only. Requires existing offset and no backfill.when_needed— snapshot if no offset exists or schema history is incomplete.
On large tables, initial snapshots lock resources. Schedule them off-peak. Use snapshot.locking.mode set to none on MySQL when brief inconsistency is acceptable during backfill.
Idempotent consumers are non-negotiable
Kafka delivers at-least-once by default. Your consumer will see duplicates after restarts. Design handlers that upsert by primary key. Track the last processed binlog position or LSN if you need strict ordering within a partition.
For Redis cache invalidation, compare event timestamps before overwriting. Patterns from Redis caching beyond simple key-value storage apply directly here. Invalidate on update and delete. Warm cache on create if needed.
Operational risks to plan for
PostgreSQL logical replication slots prevent WAL recycling. A stopped consumer can fill your disk. MySQL binlog retention must exceed maximum consumer downtime. Monitor both.
Never point Debezium at a read replica unless the connector explicitly supports it for your database version. MySQL replicas may lag, producing out-of-order events relative to the primary.
Filter sensitive columns with Debezium's column.exclude.list or column.mask.with.N.chars. This matters for data privacy compliance on Nepali web apps where PAN numbers and phone fields must not land in analytics topics.
When should Laravel and PHP teams adopt Change Data Capture with Debezium?
Not every Laravel 13 app needs Kafka on day one. CDC pays off when you outgrow simple caching and nightly exports. Ask three questions before you invest.
First: does stale data hurt revenue or operations? An eCommerce store losing search results after stock changes qualifies. A brochure site with weekly analytics does not.
Second: how many downstream systems read the same tables? When search, BI, and a mobile API all need orders data, CDC beats writing three sync jobs.
Third: can your team operate Kafka Connect? If you run Ubuntu servers and GitLab CI already, the jump is manageable with Linux system administration support. If nobody owns infrastructure, start with managed Confluent Cloud or Redpanda before self-hosting.
Practical Laravel integration stays thin. Keep Eloquent as the write path. Let Debezium propagate changes. Invalidate Laravel cache layers from a small Kafka consumer rather than from model observers scattered across controllers.
For warehouse sync, pipe Debezium topics into dbt transform pipelines. Raw change events land in staging tables. dbt models build slowly changing dimensions from the event stream.
On trek booking platforms with Livewire dashboards, availability updates hit MySQL constantly. CDC pushed those changes to a read-optimised reporting database without adding query load to the booking path.
Cost reality for Nepal-based teams: a minimal self-hosted stack on a Rs 15,000/month VPS (~USD 112) can run Kafka, Connect, and Debezium for moderate traffic. Managed services start higher but reduce on-call burden. Budget for monitoring and backup from the start.
Key Takeaways
- Change Data Capture with Debezium tails MySQL binlog or PostgreSQL WAL and publishes ordered row events to Kafka—no polling required.
- Enable row-based binlog or logical WAL, create a replication user, register a Kafka Connect connector, then build idempotent consumers.
- Initial snapshots backfill existing rows; streaming phase captures live inserts, updates, and deletes including hard deletes.
- Monitor PostgreSQL slot lag and MySQL binlog retention to prevent disk exhaustion when consumers fall behind.
- Filter sensitive columns at the connector level and pair CDC with schema-history topics to survive Laravel migrations safely.
- Adopt CDC when multiple downstream systems need near-real-time data—not for simple sites with tolerant reporting delays.
People Also Ask
Does Debezium require Apache Kafka?
Yes. Debezium is built on Kafka Connect and publishes change events to Kafka topics by default. Redpanda and other Kafka-compatible platforms work as drop-in replacements. Some teams route Debezium output through additional sinks, but Kafka remains the standard transport layer.
Can Debezium capture changes from a Laravel application without code changes?
Yes. Debezium reads the database transaction log, not application code. Your Laravel 12 or 13 app continues using Eloquent and migrations normally. The connector picks up every committed change regardless of whether it came from HTTP requests, Artisan commands, or queued jobs.
What happens to Debezium when the database restarts?
The connector stores its offset in Kafka Connect's internal topics. After a restart, it resumes from the last committed binlog position or LSN. Brief gaps are possible if binlog files were purged during a long outage. Size retention policies to cover your maximum recovery time.
Is Debezium suitable for multi-cloud replication?
Debezium handles capture at the source. You still need a secure pipeline to move Kafka topics across regions or clouds. Pair it with mirror-maker tools or cloud-native replication described in guides on data replication across clouds. CDC replaces batch dumps, not network design.
Ship reliable sync without rewriting your Laravel app
Change Data Capture with Debezium gives you a production-grade path from MySQL or PostgreSQL to search indexes, caches, warehouses, and microservices. You keep one source of truth. Everything downstream listens to the log. Start with one non-critical table, prove the consumer logic, then expand table coverage and add monitoring.
If you want help designing CDC pipelines around an existing Laravel or enterprise application stack, or migrating from brittle polling jobs, contact us for a architecture review. You can also explore related guides on test data management for pipelines, zero-data-loss server migration, and data residency for Nepali companies before you go live.
Frequently Asked Questions
Change Data Capture records every row-level insert, update, and delete as it happens. Debezium is an open-source CDC platform built on Apache Kafka Connect. It does not poll tables on a schedule—it reads the database transaction log. MySQL 8.4 LTS uses the binary log; PostgreSQL 18 uses logical decoding of the write-ahead log. Each event includes an operation type, before and after row values, source table metadata, and log position, then lands on Kafka topics for downstream consumers.
Yes. Debezium is built on Kafka Connect and publishes change events to Kafka topics by default. Redpanda and other Kafka-compatible platforms work as drop-in replacements.
A minimal self-hosted stack on a Rs 15,000/month VPS (~USD 112) can run Kafka, Connect, and Debezium for moderate traffic. Managed services cost more but reduce on-call burden.
Yes. Debezium reads the MySQL binlog or PostgreSQL WAL, not application code. Laravel 12 or 13 continues using Eloquent and migrations; the connector picks up every committed change.
Enable row-based binlog in my.cnf with server-id, log_bin, binlog_format=ROW, binlog_row_image=FULL, gtid_mode=ON, and enforce_gtid_consistency=ON. Create a dedicated debezium user with SELECT, RELOAD, SHOW DATABASES, REPLICATION SLAVE, and REPLICATION CLIENT privileges. Deploy Kafka Connect with the Debezium MySQL connector JAR installed. Register the connector via the REST API with hostname, database.include.list, table.include.list, topic.prefix, and schema history Kafka settings. Confirm RUNNING status, then monitor initial snapshot and streaming phases.
Set wal_level=logical in postgresql.conf and restart PostgreSQL because the change requires a restart. Create a debezium user with REPLICATION login and GRANT SELECT on required tables. Register a PostgreSQL connector through Kafka Connect REST API with database hostname, credentials, and topic prefix. Logical replication slots consume disk if consumers lag, so monitor slot lag daily. Never let a stopped connector sit unattended—WAL accumulation can fill the server disk.
Polling runs repeated SELECT queries on a timer, misses hard deletes, adds read load to production, and races when rows change in the same second. Debezium delivers sub-second to seconds latency by tailing the log with low source load and full delete capture. Polling remains acceptable for low-stakes reporting tolerating ten-minute staleness. Log-based CDC wins when search indexes, caches, and analytics must track MySQL or PostgreSQL within seconds—I've seen this fix broken search sync on legal-tech portals after deletes.
snapshot.mode controls initial sync behaviour. initial takes a full snapshot then streams—default for new connectors. initial_only snapshots without streaming for one-off exports. never streams only and requires an existing offset with no backfill. when_needed snapshots if no offset exists or schema history is incomplete. On large tables, schedule initial snapshots off-peak. Set snapshot.locking.mode to none on MySQL when brief inconsistency during backfill is acceptable. Always build consumers that handle snapshot READ events before live create, update, and delete events.
Schema evolution is the top production headache. Debezium stores schema history in a dedicated Kafka topic. When a Laravel migration adds or alters a column, the connector emits a schema-change event so consumers learn the new shape before breaking. Pair CDC with idempotent handlers that upsert by primary key. After every deploy, verify consumers tolerate new columns and schema-change events. On large tables, plan DDL during off-peak windows because snapshot and streaming behaviour depends on complete schema history.
Ask three questions first. Does stale data hurt revenue or operations—eCommerce search missing stock changes qualifies; a brochure site with weekly analytics does not. Do multiple downstream systems—search, BI, mobile API—read the same tables? CDC beats maintaining three separate sync jobs. Can your team operate Kafka Connect on Ubuntu with GitLab CI, or will you use managed Confluent Cloud or Redpanda? Keep Eloquent as the write path and let Debezium propagate changes to caches, warehouses, and read-optimised reporting databases.
Yes. Log-based CDC reads every committed mutation from the binlog or WAL, including DELETE operations. Timestamp polling only finds rows still present with a matching updated_at value, so hard deletes vanish from downstream search indexes and warehouses. Dual writes—app writes to MySQL and Elasticsearch in one request—also fail silently when one side errors. CDC keeps one source of truth and lets everything else catch up from the log, which is why it replaced brittle polling on document-heavy portals I've maintained.
Kafka delivers at-least-once by default. After restarts or network issues, consumers see duplicate events. Design handlers that upsert by primary key instead of blind inserts. Track the last processed binlog position or LSN when strict ordering within a partition matters. For Redis cache invalidation, compare event timestamps before overwriting—invalidate on update and delete, warm on create if needed. Idempotent consumers are non-negotiable; without them, duplicate events corrupt search indexes, caches, and warehouse staging tables.
Logical replication slots prevent WAL recycling while consumers lag. A stopped Debezium connector or crashed consumer lets WAL accumulate until disk fills. Monitor slot lag and replication slot age daily with alerts configured from day one. On MySQL, size binlog retention to exceed maximum consumer downtime for the same reason. Budget for monitoring and backup alongside the Rs 15,000/month minimal VPS stack. Both databases need retention policies sized to your maximum recovery time, not just your happy-path latency targets.
Kafka Connect stores connector offsets in internal topics. After a MySQL or PostgreSQL restart, Debezium resumes from the last committed binlog position or LSN. Brief gaps are possible if binlog files were purged during a long outage while the connector was down. Size retention policies to cover your maximum recovery time. Verify connector RUNNING status and consumer lag after every database maintenance window before declaring the pipeline healthy.
Do not point Debezium at a read replica unless the connector explicitly supports it for your database version. MySQL replicas can lag behind the primary, producing out-of-order events relative to where Laravel commits writes. Capture from the primary where Eloquent transactions land. If you need read scaling, keep writes on primary and let CDC fan out row events to search, cache, and reporting consumers—substituting a lagging replica as the CDC source creates silent data inconsistency downstream.
0 Comments
Leave a comment
Your email is not published. Comments appear once they have been read. Sign in to have your details filled in.
