Kokil Thapa - Professional Web Developer in Nepal
Freelancer Web Developer in Nepal with 15+ Years of Experience

Kokil Thapa is an experienced full-stack web developer focused on building fast, secure, and scalable web applications. He helps businesses and individuals create SEO-friendly, user-focused digital platforms designed for long-term growth.

Test Your Disaster Recovery Plan

By Kokil Thapa | Last reviewed: September 2026

You wrote a backup and disaster recovery strategy months ago. Backups ran nightly. The runbook lived in a shared doc. Then a disk filled on a Friday evening, and nobody could restore a Laravel app within the promised window. That gap is exactly why you must test your disaster recovery plan on a schedule, not treat it as paperwork. A DR plan that never gets exercised is a guess. Testing turns assumptions into measured recovery times your team can trust.

Why should you test your disaster recovery plan before a real outage?

Backups prove storage works. DR testing proves your business can come back online. On production Laravel stacks I maintain, the first full restore drill often exposes three problems: wrong file permissions on storage/, a stale cron path after Deployer symlink swaps, or a database dump that restores but misses Redis session data.

None of those show up in a backup success email. They appear only when someone follows the runbook under mild pressure. Testing also trains the people who will execute recovery at 2 a.m. If only one engineer knows the steps, your plan is fragile regardless of how good the infrastructure looks.

DR Testing LifecycleDefine RTOand RPO targetsWrite Runbookroles and stepsRun DR Drillrestore or failoverMeasure Timeactual vs targetFix Gaps and Retestupdate scripts, permissions, DNS, credentialsProduction Confidenceteam knows the plan works under pressure
Disaster recovery testing lifecycle — define targets, drill, measure, fix, and repeat until recovery times meet business requirements.

Regulatory and client expectations push in the same direction. Law-firm portals and booking platforms I have worked on store sensitive documents and payment records. A DR test report shows due diligence. It is cheaper than explaining a multi-day outage to angry users.

If you host on Ubuntu with Apache and PHP-FPM, pair DR testing with your existing Linux system administration routines. The same person who patches PHP should verify restores quarterly.

What are the main types of disaster recovery tests?

Not every test requires pulling production offline. Match the test type to risk, budget, and team size. Small Nepal agencies often start with tabletop plus isolated restore before attempting full failover.

Test typeWhat you simulateProduction impactBest for
TabletopWalk through the runbook verballyNoneNew teams, role clarity, first draft plans
Backup restoreRestore DB and files to stagingNone if isolatedProving backups are usable
Partial failoverMove read traffic or one serviceLow to mediumAPI or static asset tiers
Full failoverSwitch DNS to standby stackHigh during cutoverValidating true RTO under load
Chaos injectionKill a process or block a portControlled in stagingFinding hidden dependencies

Tabletop exercises cost almost nothing. Schedule 90 minutes. Assign roles: incident commander, communications, technical lead, scribe. Present a scenario such as "primary MySQL server is unrecoverable." Each person states what they would do in the first hour. Gaps surface fast when nobody owns DNS or payment webhook URLs.

Backup restore tests are the highest-value step for most Laravel shops. Spin up a clean Ubuntu 24 staging VM. Copy last night's mysqldump or use point-in-time recovery if you run PostgreSQL 18 or MySQL 9.7. Restore code from Git, run composer install, point .env at the restored database, and load the homepage.

Full failover is heavier. Sister legal-tech sites I deploy with Deployer 7 and GitLab CI share one EC2 pattern: symlinked releases, shared storage/, nightly DB dumps to object storage. A failover test means standing up a second server, restoring the latest dump, updating DNS TTL beforehand, and timing the cutover.

DR Test Types by ImpactTabletopNo downtimeRestoreStaging onlyPartialLow riskFull FailoverHigh impactChaosControlledStart left, move right as maturity growsQuarterly minimumrestore test on stagingvalidate checksumsAnnual full drillfor mission-critical appsdocument actual RTO
Disaster recovery test types ranked by production impact — most teams progress from tabletop exercises to full failover drills over time.

For deeper resilience work, read about chaos engineering to test resilience before outages. You do not need Kubernetes to benefit. Staging tests that stop Redis 8.10 or block outbound SMTP reveal real failure modes.

How often should you test your disaster recovery plan?

Frequency depends on how much data you can afford to lose and how fast you must recover. A brochure WordPress 7.1 site might tolerate four hours of downtime. A Laravel 13 booking platform with live payments needs tighter windows.

  • Monthly: Verify backup jobs completed, spot-check one random file restore, confirm off-site copy exists.
  • Quarterly: Full database restore to staging, run smoke tests, rotate credentials used only in DR.
  • Semi-annually: Tabletop with all stakeholders including non-technical owners.
  • Annually: Full failover or simulated region loss for revenue-critical systems.
  • After major changes: New payment gateway, database migration, or PHP 8.5 upgrade on production.

Calendar reminders beat good intentions. Tie the quarterly restore to an existing support and maintenance retainer task so it actually ships. Dashain and Tihar busy seasons in Nepal are a bad time to schedule disruptive failover tests. Plan drills for quieter weeks.

Document each test date, participants, scenario, results, and follow-up tickets. Auditors and enterprise clients ask for this history. A simple markdown log in your internal wiki beats a forgotten spreadsheet.

How do you measure RTO and RPO during a DR test?

RPO (Recovery Point Objective) is how much data you may lose, measured in time. If your last good backup is six hours old at failure, your achieved RPO is six hours. RTO (Recovery Time Objective) is how long until service is usable again, from incident start to verified recovery.

During a test, write down timestamps at each milestone. Do not estimate afterward. Use a shared clock or UTC in logs.

  1. T0 — Incident declared: Start the stopwatch when you pretend the primary site is lost.
  2. T1 — Decision to restore: Team agrees on restore vs failover path.
  3. T2 — Infrastructure ready: VM provisioned, DNS lowered if needed, disk mounted.
  4. T3 — Data restored: Database import finished, storage/ synced.
  5. T4 — App healthy: HTTP 200 on health route, queue worker running, cron verified.
  6. T5 — Business sign-off: Test login, place test order, or upload a document.

RTO equals T5 minus T0. Compare against your SLA promise. RPO equals T0 minus the timestamp of the backup you restored from.

RTO and RPO During a DR TestFailureT0Last backupRPO windowData loss windowDB restoredT3Service liveT5RTO = T5 minus T0 (total downtime budget)Target RTO: 2 hoursActual: 3h 40m = FAILTarget RPO: 1 hourActual: 45m = PASS
Measuring RTO and RPO during disaster recovery testing — timestamp each milestone and compare results against documented targets.

On a client portal like Mijar Law Associates, RPO includes uploaded PDFs in object storage, not just SQL rows. Your test must restore Spatie Media Library paths or equivalent. Missing files mean a passed database restore and a failed business recovery.

PostgreSQL teams should follow a dedicated PostgreSQL point-in-time recovery playbook during tests. WAL archiving gaps show up only when you replay to a specific timestamp.

How do you run a Laravel production DR restore test step by step?

This workflow mirrors stacks I run on Ubuntu with PHP 8.3 or 8.5, MySQL 9.7, Redis 8.10, and Deployer 7 releases. Adapt paths to your host layout.

Prepare an isolated staging target

Never restore unknown backup bytes onto production. Use a fresh VM or container network segment. Install matching PHP extensions. Copy the production .env.example and fill DR-specific values.

Restore the database

# Example: restore nightly MySQL dump on staging
mysql -u dr_user -p staging_dr < /backups/app_2026-09-10.sql

# Verify row counts against production metadata (not prod connection)
mysql -u dr_user -p staging_dr -e "SELECT COUNT(*) FROM orders;"

For encrypted backups, confirm your restore operator has the key before the drill starts. A locked backup fails the test exactly like a real incident would.

Deploy application code and shared storage

# Clone release tag used in production
git clone --branch v2026.09.10 https://gitlab.example.com/app.git /var/www/dr-test
cd /var/www/dr-test
composer install --no-dev --optimize-autoloader

# Sync user uploads from off-site backup
rsync -avz backup-server:/backups/storage/ /var/www/dr-test/storage/app/
chown -R www-data:www-data storage bootstrap/cache

Run Laravel-specific checks after deploy:

php artisan config:clear
php artisan migrate --force
php artisan queue:restart
php artisan route:list | head
curl -I https://dr-staging.example.com/health

Validate integrations

Payment callbacks, SMS gateways, and eSewa or Khalti credentials often differ between environments. During DR testing, use sandbox keys where possible. Confirm webhook URLs in the gateway dashboard match the DR hostname if you test cutover.

E-commerce restores need order state checks. On a WooCommerce 11.1 or custom Laravel cart, verify pending orders, stock counts, and coupon tables. Use the JSON formatter to inspect API health responses if you expose a status endpoint.

Record results and open fix tickets

Common failures I see: opcache serving old code until PHP-FPM reload, APP_KEY mismatch breaking encrypted sessions, queue workers pointing at old Redis DB index, and cron still referencing a previous Deployer release path. Each becomes a runbook patch and often a one-line automation fix.

Laravel DR Restore Test FlowPull BackupDB + storageRestore SQLverify countsDeploy Codecomposer installSync Filesstorage and mediaArtisan Checksmigrate and queueSmoke Testlogin and payLog timestamps at each step for RTO
Laravel disaster recovery restore test sequence — pull backups, restore data, deploy code, run smoke tests, and log timestamps for RTO measurement.

Multi-cloud or hybrid setups add DNS and load-balancer steps. See multi-cloud disaster recovery strategy if you split primary and standby across providers. The test script grows, but the measurement method stays the same.

What should a disaster recovery test checklist include?

Use a checklist every time. It prevents skipped steps when adrenaline rises during a real event. Below is a practical baseline for a small team running a quarterly restore drill.

  • Scope signed off: Which apps, databases, and file paths are in scope.
  • Roles assigned: Commander, restore operator, verifier, communications.
  • Backup verified: Checksum or size compared to prior day before restore starts.
  • Secrets available: Backup encryption keys, DB passwords, API tokens in a vault.
  • Isolation confirmed: Staging cannot reach production databases or webhooks.
  • Runbook followed verbatim: Note every deviation and why it happened.
  • Smoke tests defined: Login, checkout, document upload, admin report, email send.
  • RTO/RPO recorded: Timestamps written during the drill, not reconstructed later.
  • Post-mortem filed: Pass/fail per criterion, tickets for gaps, retest date set.

Store credentials in a password manager. During onboarding, confirm two people can access it. A password generator helps rotate DR-only accounts after each test.

Align checklist items with your hosting setup. If you use managed domain registration and hosting, include registrar login and DNS TTL in the runbook. Low TTL a day before a failover test makes rollback safer.

Enterprise Laravel apps benefit from formal testing and optimization practices applied to DR. Treat the drill like a release gate. No pass, no confidence.

Reference frameworks help structure expectations. The U.S. National Institute of Standards and Technology publishes SP 800-34 on contingency planning, which separates plan development from plan testing. The ISO 22301 business continuity standard also expects exercised procedures, not shelfware. You do not need certification to borrow the discipline.

For database-specific guidance, the MySQL 9.7 Backup and Recovery documentation and PostgreSQL 18 backup chapter document restore commands your runbook should copy verbatim. Test those commands quarterly so version drift in flags does not surprise you after an upgrade.

On trekking booking platforms such as Adventure Third Pole Trek, peak season traffic makes downtime costly. DR tests should include queue backlog replay and supplier notification templates. Technical recovery without customer communication still counts as a business failure.

GitLab CI pipelines should be part of the drill if deploy automation is your recovery path. Can you redeploy the last green pipeline to a fresh server without manual SSH edits? If not, add that scenario to the next test. Several sites I maintain share the same Deployer 7 pattern documented across our Notary Kathmandu and related legal-tech portfolio work.

Budget-conscious teams in Nepal often ask what DR testing costs. A quarterly staging restore might take four to eight engineer-hours — roughly Rs 20,000–40,000 (~USD 150–300) at typical freelance rates. That is far less than a day of lost e-commerce revenue or emergency consultant fees during an actual outage.

Security matters during tests too. DR staging copies contain production PII. Restrict SSH access, destroy the environment after the test, and scrub logs that capture real user emails. A leaked staging clone is its own incident.

Finally, communicate results to stakeholders in plain language. "We recovered in three hours, target was two" is clearer than jargon-heavy reports. Non-technical owners use that data to decide whether to invest in a hot standby or accept risk.

Key Takeaways

  • Test your disaster recovery plan on a fixed schedule — quarterly restore drills at minimum, annual failover for critical apps.
  • Measure actual RTO and RPO with written timestamps; compare against business promises, not gut feel.
  • Start with tabletop and isolated backup restores before attempting production DNS cutover.
  • Include files, queues, cron, and third-party webhooks — not just the database dump.
  • Document gaps, patch the runbook, and retest until recovery times consistently meet targets.
  • Treat DR staging data as sensitive production PII and tear down test environments after verification.

People Also Ask

What is the difference between a backup test and a disaster recovery test?

A backup test confirms a single file or database dump can be read and restored. A disaster recovery test validates the entire recovery path: infrastructure, application deploy, integrations, DNS, and user-facing smoke tests within your RTO and RPO targets.

Who should be involved in a disaster recovery drill?

Include whoever would respond during a real outage: lead developer, sysadmin or DevOps, database owner, and a business decision-maker. Add communications if clients or regulators must be notified during prolonged downtime.

Can you test disaster recovery without downtime?

Yes. Tabletop exercises and restores to isolated staging have zero production impact. Partial and full failover tests affect live traffic, so schedule them in maintenance windows with lowered DNS TTL and rollback steps ready.

How do you know if your disaster recovery test passed?

The test passes when restored services meet defined smoke-test criteria and both RTO and RPO come in at or better than documented targets. Partial success still counts as a fail if checkout, uploads, or payments do not work.

Make your disaster recovery plan provably real

A documented plan is the starting point. Scheduled drills that test your disaster recovery plan against real backups turn it into operational truth. Start with next quarter's restore to staging, log your timestamps, and close the gaps the first run always reveals. If you want help building runbooks, automating backups, or running a structured drill on Laravel, WordPress, or custom eCommerce stacks, review our enterprise application development and web development services, browse the portfolio for production examples, or contact us to plan a DR test on your infrastructure.

Frequently Asked Questions

A backup test confirms a single file or database dump can be read and restored. A disaster recovery test validates the entire recovery path: infrastructure, application deploy, integrations, DNS, and user-facing smoke tests within your RTO and RPO targets. Backups prove storage works; DR testing proves your business can come back online.

On production Laravel stacks I maintain, the first full restore drill often exposes wrong file permissions on storage/, a stale cron path after Deployer symlink swaps, or a database dump that restores but misses Redis session data. None of those appear in a backup success email. Testing trains the people who will execute recovery at 2 a.m., and if only one engineer knows the steps, your plan is fragile. For law-firm portals and booking platforms storing sensitive documents and payment records, a DR test report demonstrates due diligence cheaper than explaining a multi-day outage to angry users.

Match test type to risk, budget, and team size. Tabletop exercises walk through the runbook verbally with no production impact—ideal for new teams. Backup restore tests pull DB and files to an isolated staging environment and are the highest-value step for most Laravel shops. Partial failover moves read traffic or one service with low to medium impact. Full failover switches DNS to a standby stack to validate true RTO under load. Chaos injection kills a process or blocks a port in staging to find hidden dependencies. Small Nepal agencies often start with tabletop plus isolated restore before attempting full failover.

Frequency depends on how much data you can afford to lose and how fast you must recover. Monthly: verify backup jobs completed, spot-check one random file restore, confirm off-site copy exists. Quarterly: full database restore to staging, run smoke tests, rotate DR-only credentials. Semi-annually: tabletop with all stakeholders including non-technical owners. Annually: full failover or simulated region loss for revenue-critical systems. After major changes—new payment gateway, database migration, or PHP upgrade—run an extra drill. Tie quarterly restores to an existing support retainer task so they actually ship, and document each test date, participants, scenario, results, and follow-up tickets.

RTO (Recovery Time Objective) is how long until service is usable again, from incident start to verified recovery. During a test, record T0 when the incident is declared and T5 when business sign-off completes; RTO equals T5 minus T0.

RPO (Recovery Point Objective) is how much data you may lose, measured in time. If your last good backup is six hours old at failure, your achieved RPO is six hours. During testing, RPO equals T0 minus the timestamp of the backup you restored from.

A quarterly staging restore typically takes four to eight engineer-hours—roughly Rs 20,000–40,000 (~USD 150–300) at typical freelance rates. That is far less than a day of lost e-commerce revenue or emergency consultant fees during an actual outage.

Write down timestamps at each milestone during the drill; do not estimate afterward. Use a shared clock or UTC in logs. T0 is incident declared, T1 is decision to restore, T2 is infrastructure ready, T3 is data restored, T4 is app healthy with HTTP 200 on a health route and queue worker running, T5 is business sign-off such as test login or placing a test order. RTO equals T5 minus T0 compared against your SLA promise. RPO equals T0 minus the backup timestamp you restored from. On client portals, RPO must include uploaded PDFs in object storage and Spatie Media Library paths, not just SQL rows.

Prepare an isolated staging target with matching PHP extensions and DR-specific .env values—never restore unknown backup bytes onto production. Restore the database from last night's mysqldump, verify row counts, then clone the production release tag, run composer install, and rsync user uploads from off-site backup with correct www-data ownership. Run php artisan config:clear, migrate --force, queue:restart, and curl the health route. Validate payment callbacks, SMS gateways, and sandbox keys for eSewa or Khalti, confirming webhook URLs match the DR hostname. Record timestamps at each milestone and open fix tickets for common failures like opcache serving old code, APP_KEY mismatch, queue workers on wrong Redis DB index, or cron referencing a previous Deployer release path.

Every drill should cover: scope signed off for which apps, databases, and file paths are in scope; roles assigned for commander, restore operator, verifier, and communications; backup verified with checksum or size compared to prior day; secrets available including encryption keys and API tokens in a vault; isolation confirmed so staging cannot reach production databases or webhooks; runbook followed verbatim with deviations noted; smoke tests defined for login, checkout, document upload, admin report, and email send; RTO and RPO recorded with live timestamps; and a post-mortem filed with pass/fail per criterion, tickets for gaps, and a retest date. Include registrar login and DNS TTL if you use managed hosting.

Yes. Tabletop exercises walk through the runbook verbally with zero production impact and cost almost nothing—schedule 90 minutes, assign roles, and present a scenario such as primary MySQL server is unrecoverable. Backup restore tests spin up a clean Ubuntu staging VM, restore database and files, and load the homepage without touching production if properly isolated. Chaos injection in staging stops Redis or blocks outbound SMTP to reveal failure modes without pulling production offline. Most teams progress from tabletop and isolated restore drills to partial and full failover over time as confidence grows.

Problems I see repeatedly on Ubuntu stacks with PHP-FPM, MySQL, Redis, and Deployer 7 releases include opcache serving old code until PHP-FPM reload, APP_KEY mismatch breaking encrypted sessions, queue workers pointing at an old Redis DB index, and cron still referencing a previous Deployer release path after symlink swaps. Wrong file permissions on storage/ and bootstrap/cache also block writes. A database dump may restore cleanly while Redis session data or uploaded files in storage/app/ are missing, producing a passed technical restore and a failed business recovery. Each failure becomes a runbook patch and often a one-line automation fix.

A database restore alone is not business recovery. Include shared storage/ user uploads synced from off-site backup, Redis session and cache data, queue worker backlog replay, cron jobs with current Deployer release paths, and third-party integrations such as payment webhooks, SMS gateways, and email SMTP. On e-commerce restores with WooCommerce or custom Laravel carts, verify pending orders, stock counts, and coupon tables. On client portals, restore Spatie Media Library file paths alongside SQL rows. GitLab CI pipelines should be part of the drill if deploy automation is your recovery path—confirm you can redeploy the last green pipeline to a fresh server without manual SSH edits.

DR staging copies contain production PII, so treat them as sensitive. Restrict SSH access to the test environment, destroy the staging VM after verification, and scrub logs that capture real user emails. A leaked staging clone is its own security incident. Store DR credentials in a password manager and confirm at least two people can access it during onboarding. Rotate DR-only accounts after each quarterly test. Never point staging webhooks at production payment gateways without sandbox keys, and confirm isolation so the test environment cannot reach production databases or trigger live callbacks.

Plan disruptive failover drills for quieter operational weeks, not peak business periods. In Nepal, Dashain and Tihar busy seasons are a bad time to schedule tests that risk customer-facing disruption or pull engineers away from live support. Revenue-critical systems such as trekking booking platforms face costly downtime during peak season traffic, so schedule annual full failover tests during off-peak months. Lower DNS TTL a day before a planned failover test to make rollback safer, but coordinate timing with stakeholders who understand seasonal load patterns. Calendar reminders tied to maintenance retainers help ensure quarterly restores happen during appropriate windows rather than being deferred indefinitely.

Share this article

0 Comments

Leave a comment

Your email is not published. Comments appear once they have been read. Sign in to have your details filled in.

Quick Contact Options
Choose how you want to connect me: