RAG Explained: Retrieval-Augmented Generation

By Kokil Thapa | Last reviewed: September 2026

RAG Explained: Retrieval-Augmented Generation bridges the gap between a frozen LLM and your live business data. A base model only knows its training cutoff. It cannot read today's product catalog, court filing templates, or an uploaded contract PDF. RAG retrieves relevant passages first, then generates an answer grounded in those sources. For teams shipping AI into web apps, our AI integration and automation service usually starts with this architecture before fine-tuning or custom training.

Retrieval-augmented generation is not magic. It is a pipeline: chunk documents, embed them, store vectors, search at query time, and prompt the model with the top matches. I've integrated LLM APIs on legal-tech portals and eCommerce backends where auditability matters. RAG wins because you can swap the knowledge base without touching model weights. The rest of this guide walks through architecture, tooling choices, Laravel-friendly implementation, and the failure modes I see in production.

What is RAG (Retrieval-Augmented Generation) and how does it work?

The term comes from the 2020 paper by Lewis et al. on retrieval-augmented generation for knowledge-intensive NLP tasks. The core idea is unchanged in 2026: treat the LLM as a reasoning engine, not a database.

A typical RAG system has two phases. Indexing runs offline or on a schedule. You ingest documents, split them into chunks, convert each chunk to a vector embedding, and store those vectors with metadata. Querying runs at request time. You embed the user's question, find the nearest vectors, fetch the original text, and pass that text into the prompt.

The five moving parts every RAG stack needs

1. Document loader — reads PDFs, HTML pages, database rows, or CMS content.
2. Chunker — splits text into passages sized for embedding models (often 300–800 tokens).
3. Embedding model — converts text to dense vectors. OpenAI's text-embedding-3-small is a common default; see the official embeddings guide.
4. Vector store — persists vectors and runs similarity search.
5. Generator — the LLM that reads retrieved context plus the user question.

Chunk quality drives retrieval quality. A common mistake is splitting mid-sentence or mixing unrelated topics in one chunk. On a legal-tech portal I built, we chunked by section heading first. That alone improved answer relevance more than swapping embedding models.

Metadata matters too. Store source URL, document ID, page number, and last-updated timestamp with each vector. You need those fields for citations, cache invalidation, and access control. A user asking about divorce procedures should only retrieve documents their role can read.

When should you use RAG instead of fine-tuning a language model?

RAG and fine-tuning solve different problems. Fine-tuning changes model behaviour—tone, format, domain vocabulary baked into weights. RAG injects fresh facts at inference time. For most business apps in 2026, RAG is the faster path to production.

Choose RAG when answers must reference specific, changing content: product specs, legal guides, internal SOPs, or API documentation. Choose fine-tuning when you need consistent output structure that prompting alone cannot enforce. Our dedicated write-up on RAG vs fine-tuning walks through decision criteria with cost examples.

On client projects with budget limits—common in Nepal—I recommend RAG first. Re-indexing a PostgreSQL table costs far less than a fine-tuning run on proprietary data. You can always add fine-tuning later if the model's tone or reasoning style is the bottleneck, not its knowledge.

How do you build a RAG pipeline for a Laravel or PHP application?

Laravel 13 on PHP 8.3+ is a practical host for RAG backends. You already have queues, scheduling, Eloquent, and Sanctum for API auth. The pattern I use: index documents via Artisan commands or queued jobs; serve queries through a controller or Livewire component.

Step 1: Store chunks and vectors in PostgreSQL 18 with pgvector

Running vectors inside your existing database avoids another vendor and bill. Enable the extension, then create a table for chunks:

CREATE EXTENSION IF NOT EXISTS vector;

CREATE TABLE document_chunks (
    id BIGSERIAL PRIMARY KEY,
    document_id BIGINT NOT NULL,
    content TEXT NOT NULL,
    embedding vector(1536),
    metadata JSONB DEFAULT '{}',
    created_at TIMESTAMPTZ DEFAULT NOW()
);

CREATE INDEX ON document_chunks
USING ivfflat (embedding vector_cosine_ops)
WITH (lists = 100);

See the pgvector project documentation for index tuning on larger datasets. Our hands-on guide on RAG with pgvector and Laravel covers migrations, indexing jobs, and query code end to end.

Step 2: Index documents on a schedule or webhook

When a CMS page or product record changes, dispatch a job. The job chunks text, calls the embedding API, and upserts vectors. Pseudocode for a Laravel job:

public function handle(EmbeddingClient $embedder): void
{
    $chunks = $this->chunker->split($this->document->body);

    foreach ($chunks as $index => $text) {
        $vector = $embedder->embed($text);

        DocumentChunk::updateOrCreate(
            ['document_id' => $this->document->id, 'chunk_index' => $index],
            ['content' => $text, 'embedding' => $vector]
        );
    }
}

Run indexing through Laravel queues backed by Redis 8.10. Never block HTTP requests with embedding calls. A 200-page PDF can take minutes to process.

Step 3: Query with similarity search and assemble the prompt

$queryVector = $embedder->embed($request->question);

$chunks = DocumentChunk::query()
    ->selectRaw('content, metadata, embedding <=> ? AS distance', [$queryVector])
    ->orderBy('distance')
    ->limit(5)
    ->get();

$context = $chunks->pluck('content')->implode("\n---\n");

$prompt = "Answer using only the context below.\n\n{$context}\n\nQuestion: {$request->question}";

Validate the JSON payload with your existing Form Request rules. Log retrieved chunk IDs for debugging. Use the JSON formatter tool when prototyping API responses during development.

For product search beyond plain Q&A, read our post on AI-powered search for Laravel products. It extends the same embedding pattern with filters and faceted metadata.

Step 4: Add caching, rate limits, and cost guards

Embedding every query costs money. Cache query embeddings in Redis when users repeat similar questions. Apply rate limiting per user or IP—the same patterns we document for API rate limiting in modern web apps. Track token usage per tenant for billing on multi-user platforms like Mijar Law Associates style client portals.

If you prefer a managed vector service over self-hosted pgvector, our comparison of pgvector vs Pinecone for RAG covers latency, ops overhead, and NPR-friendly hosting costs.

What vector database should you choose for production RAG?

The right store depends on scale, ops capacity, and whether vectors live beside relational data. Small teams on Ubuntu servers often win with PostgreSQL plus pgvector. High-volume semantic search across millions of chunks may justify a dedicated engine.

• pgvector on PostgreSQL 18 — best when you already run Postgres, need JOINs with business tables, and stay under roughly a few million vectors.
• Redis 8.10 with vector search — good for low-latency caches and small corpora co-located with session data.
• Pinecone / Weaviate / Qdrant — managed or self-hosted options when dedicated ANN performance and horizontal scaling matter.
• OpenSearch / Elasticsearch — hybrid keyword plus vector search when BM25 and embeddings must combine.

Hybrid search is underrated. Pure vector search misses exact SKU codes, case numbers, and statute references. A pattern I've used on legal-tech content: run BM25 for exact terms, vector search for paraphrases, then merge with reciprocal rank fusion. That cut hallucinated citations on a notary information site similar to Notary Nepal.

For greenfield custom software development projects, start with pgvector. Migrate only when query latency or index size forces it. Premature infrastructure splits ops attention you cannot afford on a Rs 5,000–15,000/month (~USD 37–110) VPS budget.

What are the common RAG failure modes and how do you fix them?

RAG systems fail quietly. The model still returns fluent text—it is just wrong. Treat retrieval quality as a first-class metric, not an afterthought.

Retrieval misses the right document

Symptoms: correct answers exist in your corpus but never surface. Fixes include smaller chunks with overlap, hybrid search, query expansion (ask the LLM to rewrite the user question), and metadata filters (restrict by product line, practice area, or language).

The model ignores retrieved context

Symptoms: answers sound plausible but contradict the provided passages. Tighten the system prompt: instruct the model to say "I don't know" when context is insufficient. Lower temperature. Reduce chunk count so the prompt stays focused.

Stale or duplicate index entries

Symptoms: outdated prices, repealed legal guidance, or two versions of the same page. Delete vectors when source documents are removed. Store content_hash on each chunk and re-embed only when the hash changes. Schedule nightly reconciliation jobs through Laravel's task scheduler.

Security and governance gaps

RAG over internal documents is an exfiltration risk if every user searches the same index. Scope retrieval by tenant ID, user role, or document ACL stored in metadata. Log prompts and retrieved chunk IDs. Read AI governance and responsible AI basics before exposing RAG to end users on a public site.

Evaluate retrieval with a held-out question set. For each test question, mark the correct source document. Measure recall@5 weekly. A drop after a CMS migration is easier to catch than a flood of user complaints.

Ready-made chatbot scaffolding is covered in build a RAG chatbot for your product documentation. Pair that with AI rate limits and cost optimization so a viral traffic spike does not drain your API budget.

Key Takeaways

• RAG grounds LLM answers in your documents by retrieving relevant chunks before generation—no model retraining required.
• Chunk quality, metadata, and hybrid search matter more than chasing the newest embedding model.
• Laravel 13 + PostgreSQL 18 + pgvector is a production-ready stack for most SME and agency projects.
• Cache embeddings, rate-limit queries, and log retrieval sets to control cost and debug misses.
• Measure recall@k on a test set; fluent wrong answers are the default failure mode.
• Apply document-level access control in the vector metadata before launching to customers.

People Also Ask

What is the difference between RAG and a standard ChatGPT prompt?

A standard prompt relies entirely on the model's training data and whatever fits in the context window. RAG fetches your private or live documents first, then passes them into the prompt. That makes answers traceable to specific sources and updatable by re-indexing—not by retraining.

Do you need a vector database for RAG?

You need vector storage and similarity search, but not necessarily a standalone vector DB. PostgreSQL with pgvector, Redis vector indexes, or even in-memory stores work for small corpora. Dedicated vector databases help at scale or when you need advanced filtering and sharding.

How much does running RAG cost in 2026?

Costs split into embedding (one-time per chunk), search (mostly infrastructure), and generation (per query). A 10,000-chunk index might cost a few dollars to embed once. Query costs depend on LLM token pricing. Self-hosting pgvector on an existing VPS adds little beyond disk and CPU.

Can RAG work with Nepali-language content?

Yes. Multilingual embedding models handle Nepali text, though quality varies by model and domain. Chunk Nepali content by paragraph or section, store language in metadata, and filter searches by locale when the site serves both English and Nepali pages. Test with real user questions—not only English paraphrases.

Ship RAG with clear boundaries and measurable retrieval

RAG Explained: Retrieval-Augmented Generation is the pattern I recommend first when a client needs AI that cites real business content. Index well, retrieve with hybrid search, prompt strictly, and measure recall before you polish the chat UI. Whether you run a law-firm portal, an eCommerce catalog, or internal API docs, the architecture stays the same.

Need help wiring RAG into a Laravel app, WordPress site, or client portal? Explore our API development services, browse the Court Marriage in Nepal portfolio for content-heavy examples, or contact us to scope an AI integration that fits your budget and compliance needs.