RAG with pgvector and Laravel Practical Setup

By Kokil Thapa | Last reviewed: August 2026

Building a RAG with pgvector and Laravel practical setup requires moving beyond basic full-text search to implement true semantic understanding within your application. While traditional MySQL or PostgreSQL text search matches keywords, Retrieval Augmented Generation (RAG) matches meaning by converting content into vectors and finding the closest mathematical neighbors in high-dimensional space. This guide provides the exact configuration, migration code, and query patterns needed to deploy this architecture on Laravel 12 with PostgreSQL 17 and pgvector 0.8+.

How do you install and configure pgvector for a RAG with pgvector and Laravel practical setup?

Before writing any PHP code, your database server must support vector operations natively. In my experience deploying database-driven web applications, skipping this infrastructure verification is the most common cause of deployment failure. You cannot simply add a package via Composer and expect vector math to work; the database engine itself must be patched.

For production environments running Ubuntu 22.04 or 24.04 LTS with PostgreSQL 17, install the official PGDG repository packages rather than compiling from source. Compiling introduces build-time dependencies that complicate future security patching and upgrades.

# Install pgvector for PostgreSQL 17 on Ubuntu
sudo apt install postgresql-17-pgvector

# Verify installation inside psql
CREATE EXTENSION IF NOT EXISTS vector;
SELECT extversion FROM pg_extension WHERE extname = 'vector';

Once the extension is active, create the Laravel migration. The critical detail here is dimension matching. If you plan to use OpenAI’s text-embedding-3-small, set dimensions to 1536. For nomic-embed-text via Ollama, use 768. Mismatched dimensions will cause silent failures or runtime exceptions during insertion.

// database/migrations/xxxx_xx_xx_create_document_chunks_table.php
use Illuminate\Database\Migrations\Migration;
use Illuminate\Database\Schema\Blueprint;
use Illuminate\Support\Facades\Schema;

return new class extends Migration
{
    public function up(): void
    {
        Schema::create('document_chunks', function (Blueprint $table) {
            $table->id();
            $table->foreignId('document_id')->constrained()->cascadeOnDelete();
            $table->text('content');
            $table->json('metadata')->nullable();
            // 1536 dimensions for text-embedding-3-small
            $table->rawColumn('embedding', 'vector(1536)'); 
            $table->timestamps();
            
            // Index for cosine similarity search
            $table->rawIndex('USING hnsw (embedding vector_cosine_ops)', 'idx_chunks_embedding');
        });
    }
};

Note the raw index definition. Laravel’s schema builder does not natively support HNSW indexes as of version 12.x. You must use rawIndex or a post-migration DB statement. Without this index, similarity searches on tables exceeding 10,000 rows will degrade from milliseconds to seconds, making real-time RAG impossible.

How do you generate and store embeddings efficiently in Laravel?

Embedding generation is the bottleneck in any RAG system. Never perform this synchronously during a user request. On a legal-tech portal I built for document analysis, synchronous embedding added 800ms–2s per upload, destroying UX. Always offload to Laravel Queues.

Create a dedicated job that handles chunking and embedding atomically. Use the OpenAI PHP client or direct HTTP calls to your local Ollama instance. Store the resulting vector as a string representation that pgvector accepts.

// app/Jobs/GenerateDocumentEmbeddings.php
use App\Models\DocumentChunk;
use Illuminate\Bus\Queueable;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Queue\SerializesModels;
use OpenAI\Laravel\Facades\OpenAI;

class GenerateDocumentEmbeddings implements ShouldQueue
{
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;

    public function __construct(private DocumentChunk $chunk) {}

    public function handle(): void
    {
        $response = OpenAI::embeddings()->create([
            'model' => 'text-embedding-3-small',
            'input' => $this->chunk->content,
        ]);

        // Convert float array to pgvector string format "[0.001, -0.023, ...]"
        $vectorString = '[' . implode(',', $response->embeddings[0]->embedding) . ']';

        $this->chunk->update([
            'embedding' => $vectorString,
        ]);
    }
}

A common mistake is storing embeddings in a separate table linked by foreign key. This forces JOINs during similarity search, which prevents pgvector from using the HNSW index efficiently. Keep content and embedding in the same row unless you have a specific multi-modal reason to separate them.

For cost-sensitive projects in Nepal where API budgets are tight, consider running nomic-embed-text or mxbai-embed-large locally via Ollama. These models produce 768-dimensional vectors with quality approaching proprietary APIs for domain-specific legal and technical content. Adjust your migration dimension accordingly and update the job to call http://localhost:11434/api/embeddings.

How do you execute semantic similarity queries for RAG with pgvector and Laravel practical setup?

Retrieval is where the theoretical meets the practical. Eloquent does not understand vector operators, so you must drop to raw expressions for the similarity calculation while keeping the rest of the query builder intact for filtering and pagination.

The three primary distance operators in pgvector are:

• <-> — Euclidean (L2) distance. Good for normalized vectors.
• <#> — Inner product. Best when vectors are already normalized to unit length.
• <=> — Cosine distance. Most versatile for text embeddings; measures angular similarity regardless of magnitude.

For text-based RAG, cosine distance is almost always correct. Here is the pattern I use repeatedly in production:

// app/Services/SemanticSearchService.php
namespace App\Services;

use App\Models\DocumentChunk;
use OpenAI\Laravel\Facades\OpenAI;

class SemanticSearchService
{
    public function search(string $query, int $limit = 5): array
    {
        // 1. Embed the query first
        $response = OpenAI::embeddings()->create([
            'model' => 'text-embedding-3-small',
            'input' => $query,
        ]);
        
        $queryVector = '[' . implode(',', $response->embeddings[0]->embedding) . ']';

        // 2. Retrieve nearest neighbors with metadata filtering
        return DocumentChunk::query()
            ->selectRaw("*, embedding <=> ? as distance", [$queryVector])
            ->whereNotNull('embedding')
            ->orderByRaw("embedding <=> ?", [$queryVector])
            ->limit($limit)
            ->get()
            ->map(fn ($chunk) => [
                'content'   => $chunk->content,
                'score'     => 1 - $chunk->distance, // Convert distance to similarity
                'metadata'  => $chunk->metadata,
            ])
            ->toArray();
    }
}

Always filter out NULL embeddings before ordering. Rows inserted before backfill jobs complete will have NULL vectors and cause sorting errors or misleading zero-distance results. The whereNotNull guard is non-negotiable in systems with asynchronous embedding pipelines.

What are the performance trade-offs between pgvector and dedicated vector databases?

This question arises on nearly every project where I consult on Laravel API architecture. Dedicated vector databases like Pinecone, Weaviate, or Qdrant offer managed scaling and specialized indexing, but they introduce operational complexity that many teams underestimate.

For most Laravel applications serving SMEs, legal portals, or internal tools, pgvector wins decisively. The ability to join vector search results against users, permissions, documents, and audit logs in a single transactional query eliminates an entire class of synchronization bugs. Only move to a dedicated vector database when you exceed 10 million vectors or require sub-10ms latency at massive concurrent query volumes.

In Nepal specifically, where cloud infrastructure costs are paid in USD but revenue often comes in NPR, avoiding a $200/month vector database bill for a project earning Rs 150,000/month (~USD 1,125) is a meaningful margin difference. pgvector lets you ship semantic search on existing infrastructure without adding another vendor dependency.

How do you tune HNSW indexes and avoid common production pitfalls?

HNSW (Hierarchical Navigable Small World) indexes are powerful but misconfigured defaults cause real problems. Two parameters matter most:

• m — Number of connections per layer. Default 16. Higher = better recall, more memory, slower builds.
• ef_construction — Size of dynamic candidate list during index build. Default 64. Higher = better quality, slower builds.

For text RAG workloads, I’ve found m=16, ef_construction=128 provides excellent recall without excessive build times. Create the index with explicit parameters:

-- Run after initial data load, not during migration
CREATE INDEX CONCURRENTLY idx_chunks_embedding_hnsw 
ON document_chunks 
USING hnsw (embedding vector_cosine_ops) 
WITH (m = 16, ef_construction = 128);

Always use CONCURRENTLY in production. Building an HNSW index on a million-row table can take hours; without this flag, the table is locked and your application goes down. Schedule index creation during low-traffic windows and monitor progress via pg_stat_progress_create_index.

Another pitfall: forgetting to set hnsw.ef_search at query time. This session-level parameter controls search accuracy vs. speed trade-off. Set it per connection or per query:

// In your service provider or before search queries
DB::statement("SET LOCAL hnsw.ef_search = 100");

Values between 50–200 cover most RAG use cases. Below 50, recall drops noticeably. Above 200, latency increases without meaningful quality gains for text embeddings. Benchmark with your actual dataset rather than trusting generic advice.

Implementing Your RAG with pgvector and Laravel Practical Setup Today

A successful RAG with pgvector and Laravel practical setup combines correct infrastructure, disciplined async processing, precise query construction, and informed index tuning. Start with PostgreSQL 17 and pgvector installed via system packages, create migrations with explicit vector dimensions and HNSW indexes, offload embedding generation to queued jobs, and use cosine distance with raw expressions for retrieval. Resist the urge to adopt a dedicated vector database until your scale genuinely demands it.

If you’re planning a semantic search feature, legal document analysis tool, or AI-powered knowledge base and need hands-on implementation support, reach out through my contact page. I’ve shipped multiple production RAG systems on Laravel and can help you avoid the pitfalls that only surface under real user load.