Laravel Job Batching for Bulk Operations

By Kokil Thapa | Last reviewed: September 2026

When you need to import ten thousand rows, regenerate invoices for every customer, or resize product images after a catalog upload, a single synchronous request will time out and a lone queued job can fail halfway with no clean recovery path. Laravel Job Batching for Bulk Operations solves that by grouping many queue jobs into one logical batch you can track, cancel, and react to when it finishes or breaks. On production Laravel queue workloads I maintain for clients in Nepal and abroad, batching is the pattern I reach for whenever work must be split across workers but still reported as one business action.

What is Laravel Job Batching for Bulk Operations?

Job batching arrived in Laravel 8 and matured through Laravel 12 and 13. Instead of firing five hundred independent jobs with no shared context, you wrap them in a Batch object. Laravel persists batch metadata—total jobs, pending count, failed count—in a job_batches table and updates it as workers finish each job.

That gives you three things a plain queue cannot:

• Aggregate completion logic — run code only after every job in the batch succeeds.
• Partial failure visibility — see how many jobs failed without guessing from scattered log lines.
• User-facing progress — poll batch progress from a dashboard or admin panel during long imports.

Batching is not the same as job chaining. Chains run jobs sequentially; batches run jobs in parallel (subject to worker count). For bulk operations—CSV imports, email blasts, image processing, search reindexing—parallel batch jobs with a single completion callback is usually the right model.

On a legal-tech portal where staff upload hundreds of client documents, batching lets each file get its own virus scan and thumbnail job while the UI shows one progress bar tied to the batch ID. That pattern maps cleanly to client portal document workflows and similar systems where operators need certainty, not hope.

How do you set up Laravel job batching in production?

Batching requires three infrastructure pieces: a queue driver, the batch database table, and jobs that opt into batching. Here is a production-ready setup for Laravel 13 on PHP 8.5 with Redis 8.10 and MySQL 9.7.

Step 1: Publish the batch migration

Laravel ships a migration for the job_batches table. Run it once per environment:

php artisan queue:batches-table
php artisan migrate

The table stores batch name, total and pending job counts, failure flags, options JSON, cancellation timestamp, and finish time. Never skip this on staging; batches silently misbehave if the table is missing.

Step 2: Configure the queue driver

Set Redis as your default queue connection in .env:

QUEUE_CONNECTION=redis
REDIS_CLIENT=phpredis

CACHE_STORE=redis

Redis is the practical choice for batch throughput. The database queue driver works for low-volume batches but becomes a bottleneck above a few hundred jobs per minute. On shared hosting without Redis, database batching is acceptable for nightly imports triggered by cron-scheduled Artisan commands.

Step 3: Make jobs batchable

Each job in the batch must use the Batchable trait:

<?php

namespace App\Jobs;

use Illuminate\Bus\Batchable;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Queue\Queueable;

class ProcessOrderRow implements ShouldQueue
{
    use Batchable, Queueable;

    public function __construct(
        public int $orderId,
    ) {}

    public function handle(): void
    {
        if ($this->batch()?->cancelled()) {
            return;
        }

        // Process single order row
    }
}

The cancelled() check is easy to forget. Without it, workers keep processing jobs after an operator cancels a batch, wasting CPU and potentially writing stale data.

Step 4: Dispatch the batch

use App\Jobs\ProcessOrderRow;
use Illuminate\Bus\Batch;
use Illuminate\Support\Facades\Bus;
use Throwable;

$orderIds = Order::where('status', 'pending_export')->pluck('id');

$batch = Bus::batch(
    $orderIds->map(fn ($id) => new ProcessOrderRow($id))
)->then(function (Batch $batch) {
    // All jobs completed successfully
    ExportRun::where('batch_id', $batch->id)->update(['status' => 'completed']);
})->catch(function (Batch $batch, Throwable $e) {
    ExportRun::where('batch_id', $batch->id)->update(['status' => 'failed']);
})->finally(function (Batch $batch) {
    Cache::forget('export_in_progress');
})->name('Order Export '.now()->toDateTimeString())
  ->dispatch();

Store $batch->id in your application table so you can poll progress from the frontend. On digital gift card platforms and other eCommerce backends, I persist the batch ID on an imports or bulk_actions record for exactly this reason.

Step 5: Chunk large datasets before dispatch

A common mistake is creating one job per database row for a million-row table and passing all job instances to Bus::batch() at once. That loads enormous arrays into memory during dispatch. Chunk the work:

Order::where('status', 'pending_export')
    ->chunkById(500, function ($orders) use (&$jobs) {
        $jobs[] = new ProcessOrderChunk(
            $orders->pluck('id')->all()
        );
    });

Bus::batch($jobs)
    ->name('Order Export')
    ->dispatch();

Each chunk job processes up to five hundred records internally. You get parallel workers without dispatching a million queue payloads. For JSON-heavy imports, validate payloads with a JSON formatter in admin tooling before the batch starts—bad input at row 9,842 should not be discovered after nine thousand successful jobs.

When should you use job batching instead of a single long-running job?

Not every bulk task needs batching. Pick the pattern based on failure isolation, parallelism, and observability requirements.

Use batching when stakeholders ask "is it done yet?" or "how many failed?" For a trekking booking system like Adventure Third Pole Trek, regenerating availability for hundreds of departure dates across seasons fits batching perfectly—each date range is a job, but operations staff see one batch status in admin.

Skip batching when the work must be one atomic database transaction. Batching spreads work across processes; it cannot roll back job 400 if job 401 succeeds unless you design compensating logic yourself.

How do you handle failures and retries in Laravel batches?

By default, a failed job inside a batch marks the batch as having failures, triggers catch() on the first failure, and still allows other pending jobs to run unless you configure otherwise. Understanding that behaviour prevents surprise duplicate emails or double charges.

Per-job retry configuration

Set retries and backoff on each batch job class:

class ProcessOrderRow implements ShouldQueue
{
    use Batchable, Queueable;

    public int $tries = 3;
    public array $backoff = [10, 60, 300];

    public function failed(?Throwable $exception): void
    {
        Log::error('Order row failed', [
            'order_id' => $this->orderId,
            'batch_id' => $this->batch()?->id,
        ]);
    }
}

Allow failures without stopping the batch

For imports where partial success is acceptable, use allowFailures():

Bus::batch($jobs)
    ->allowFailures()
    ->then(function (Batch $batch) {
        // Runs even if some jobs failed
        NotifyAdminOfImportResults::dispatch($batch->id);
    })
    ->dispatch();

Without allowFailures(), the first permanently failed job fires catch() and sets the batch failed flag, even while remaining jobs still execute. Design your then() and catch() handlers accordingly—do not assume mutual exclusivity with pending work.

Cancelling batches mid-flight

$batch = Bus::findBatch($batchId);

if ($batch && ! $batch->finished()) {
    $batch->cancel();
}

Cancelled batches stop decrementing meaningfully for new work only if each job checks $this->batch()?->cancelled() at the start of handle(). Jobs already running will finish their current unit of work.

On payment reconciliation batches—similar to patterns in Laravel payment integration workflows—I set $tries = 1 for idempotency-sensitive jobs and push failures to a dead-letter table rather than blind retries that could double-settle transactions.

How do you monitor progress and test batches locally?

Production monitoring splits into application-level progress UI and infrastructure-level queue metrics.

Polling batch progress in the admin UI

public function show(string $batchId)
{
    $batch = Bus::findBatch($batchId);

    abort_unless($batch, 404);

    return response()->json([
        'id' => $batch->id,
        'name' => $batch->name,
        'total' => $batch->totalJobs,
        'pending' => $batch->pendingJobs,
        'failed' => $batch->failedJobs,
        'progress' => $batch->progress(),
        'finished' => $batch->finished(),
        'cancelled' => $batch->cancelled(),
    ]);
}

Pair this with Laravel Horizon when running Redis queues. Horizon shows throughput, wait times, and failed job details per queue. For teams without Horizon, failed jobs still land in the failed_jobs table when using the database failed-driver configuration documented in the official Laravel queue batching documentation.

Testing batches in PHPUnit

Laravel fakes batches in tests:

use Illuminate\Support\Facades\Bus;

Bus::fake();

// Trigger controller action that dispatches batch

Bus::assertBatched(function ($batch) {
    return $batch->jobs->count() === 10
        && $batch->name === 'Order Export';
});

For integration tests, run the queue synchronously with QUEUE_CONNECTION=sync in phpunit.xml to execute batch callbacks in-process. Switch to Redis in CI only when you need to test race conditions or worker concurrency—most batch logic does not require it.

Production deployment checklist

Batches fail in production for boring operational reasons more often than code bugs. Before enabling bulk batching on a live site:

1. Run php artisan queue:batches-table and migrate on production—not just locally.
2. Ensure queue:work or Horizon supervisord configs survive reboots; batches stall silently if no workers run.
3. Match PHP versions between deploy runner and workers (PHP 8.3 minimum for Laravel 13, PHP 8.5 recommended).
4. Reload PHP-FPM after deploy so opcache serves updated job classes—I've seen this break batches after otherwise clean GitLab CI deploys.
5. Set sensible --timeout on workers longer than your slowest chunk job.
6. Configure Redis maxmemory and eviction policy; queue payloads for large batches consume memory fast.

For PostgreSQL-backed apps—see PostgreSQL with Laravel—the same batching rules apply; only the queue driver and connection pooling differ. Long-running batch imports benefit from raising statement timeouts on chunk jobs, not on the web request that dispatches them.

What are advanced patterns for Laravel Job Batching for Bulk Operations?

Once basics work, three patterns cover most enterprise bulk scenarios I implement through enterprise Laravel application development.

Adding jobs to an existing batch

Dynamic batches can grow while workers process earlier jobs:

$batch = Bus::findBatch($batchId);
$batch->add([
    new ProcessOrderRow($newOrderId),
]);

Use this when upstream webhooks arrive while a bulk sync is already running. Guard against unbounded growth with a max job count stored in batch options or application state.

Batch pruning and housekeeping

The job_batches table grows indefinitely. Schedule pruning:

// routes/console.php
Schedule::command('queue:prune-batches --hours=168')->daily();

Keep seven days of batch history for debugging; prune older records unless compliance requires longer retention.

Combining batches with rate-limited APIs

When each job hits an external API with strict rate limits, use Laravel's RateLimited middleware on the job:

public function middleware(): array
{
    return [new RateLimited('external-api')];
}

Define the limiter in AppServiceProvider. Parallel batch workers respect the shared Redis rate limit while still processing faster than a single serial job. This pattern appears frequently in API integration projects where bulk sync must not trigger provider bans.

Redis 8.10 handles both queue and rate-limit counters cleanly on a single instance for moderate traffic. At higher scale, split queue Redis from cache Redis to avoid contention—an ops trade-off covered in Linux server administration engagements rather than application code alone.

Key Takeaways

• Run php artisan queue:batches-table and use the Batchable trait on every job before calling Bus::batch().
• Chunk large datasets into jobs of 200–500 records instead of one job per row or one giant job for the entire dataset.
• Always check $this->batch()?->cancelled() at the start of handle() and persist the batch ID on your domain model for progress polling.
• Use allowFailures() for imports where partial success is valid; use strict failure handling for payment or inventory mutations.
• Monitor with Horizon plus a simple JSON progress endpoint, and reload PHP-FPM after deploys so workers execute current job code.
• Schedule queue:prune-batches weekly so the job_batches table does not grow without bound.

People Also Ask

Does Laravel job batching require Redis?

No. Batching works with the database, Redis, Amazon SQS, and other supported queue drivers. Redis is strongly recommended for production bulk workloads because it handles high job throughput and pairs with Horizon for monitoring. Database queues are fine for low-volume nightly batches on budget hosting.

What is the difference between Bus::batch and Bus::chain?

Bus::batch() runs jobs in parallel and tracks group progress. Bus::chain() runs jobs one after another, stopping if any job fails. Use chains for pipelines with strict ordering; use batches for bulk operations where jobs are independent.

Can you retry an entire failed batch?

Laravel does not retry whole batches with one command. Re-dispatch a new batch for failed items by querying your domain tables or inspecting the failed_jobs table. Store enough context on each job—primary keys, import row numbers—to rebuild a failure-only batch without reprocessing successes.

How many jobs can you put in one Laravel batch?

There is no hard framework limit, but practical limits come from dispatch memory, Redis payload size, and worker throughput. Chunk dispatch into jobs that each handle hundreds of records, and split very large operations into multiple named batches rather than one batch with tens of thousands of queue entries.

Ship bulk operations you can trust

Laravel Job Batching for Bulk Operations turns fragile "run it in a loop" scripts into observable, cancellable, production-grade background work. The setup cost is one migration, one trait, and disciplined chunking—far less than recovering from a half-finished import on a live store or client portal. If you are planning bulk imports, media pipelines, or sync jobs on Laravel 13, the batch API is the correct default for anything that outgrows a single queue job.

Need help designing queue architecture, Horizon setup, or a bulk import pipeline for your Laravel application? Review relevant work in the project portfolio, read modern Laravel architecture patterns, or contact us to discuss your bulk processing requirements. For ongoing queue monitoring and worker hardening after launch, support and maintenance covers the operational side so batches keep running after the first deploy.