Ray: Distributed Python and ML

By Kokil Thapa | Last reviewed: September 2026

Ray: Distributed Python and ML solves a problem every data team hits eventually. Your notebook trains fine on one GPU, then production needs ten. Your hyperparameter sweep takes days on a single machine. Your Flask API cannot keep up with inference load. Ray turns ordinary Python functions into distributed tasks without forcing you into Java-style boilerplate or a full Spark rewrite. If you already run Python on Ubuntu and ship web systems, Ray is the bridge between local Python scripts and the cluster patterns covered in GPU workloads on Kubernetes.

What is Ray and why use Ray: Distributed Python and ML?

Ray started at UC Berkeley's RISELab and is now maintained under the PyTorch Foundation umbrella. At its core, Ray provides a distributed runtime for Python. You decorate functions with @ray.remote. Ray schedules them across CPU and GPU workers. Results come back through object refs.

That simple model supports much larger workloads. Ray Data handles petabyte-scale ETL. Ray Train wraps PyTorch, TensorFlow, and XGBoost for multi-node training. Ray Tune runs parallel hyperparameter search. Ray Serve deploys models as HTTP microservices. You pick the layer you need.

On real client projects, I integrate LLM APIs and build the web layer in Laravel or WordPress. I do not train foundation models from scratch. Ray still matters because many teams I advise run Python sidecars for batch inference, embedding generation, or fine-tuning pipelines. Ray keeps that Python code portable from a dev laptop to a production cluster.

Ray fits teams that already speak Python and want incremental scale. You do not need to adopt Kubernetes on day one. You can start with ray.init() on one machine. Later you point the same code at a multi-node cluster. That progression mirrors how many Nepal startups grow: prototype locally, then rent GPU servers or cloud VMs when revenue justifies it.

When Ray is the right choice

• You have Python training or inference code and need horizontal scale without rewriting in Scala or Java.
• You want hyperparameter search, distributed training, and model serving under one ecosystem.
• Your team is small and cannot operate a full Kubeflow pipeline yet.
• You need to batch-process embeddings or scores and feed results into a Laravel or WordPress app via API.

When Ray is not the right choice

Ray adds operational complexity. A single cron job on one VM is simpler. If your workload is pure SQL analytics, use your database or a warehouse. If you only call third-party LLM APIs with no custom models, Ray is overkill. For that pattern, see AI integration and automation services instead of building a cluster.

How do you install Ray and start a local or remote cluster?

Install Ray with pip on Python 3.9 or newer. Most production ML stacks today run Python 3.10 or 3.11. Ubuntu 22.04 or 24.04 LTS is a solid host. Match the guide in install Python on Ubuntu before you add Ray packages.

python3 -m venv .venv
source .venv/bin/activate
pip install -U pip
pip install "ray[default]"
ray --version
python -c "import ray; ray.init(); print(ray.cluster_resources()); ray.shutdown()"

The smoke test above starts a local Ray cluster in-process. You should see CPU and memory entries in the resource dict. If that fails, fix Python paths before touching GPUs.

Start a multi-node cluster manually

On the head node, bind the dashboard and GCS port explicitly. Worker nodes join with the address printed by the head.

# Head node (example: 10.0.1.10)
ray start --head --port=6379 --dashboard-host=0.0.0.0 --dashboard-port=8265

# Worker node
ray start --address='10.0.1.10:6379'

# From your laptop, connect remotely
python -c "import ray; ray.init('ray://10.0.1.10:10001')"

Open port 8265 only on trusted networks. The Ray dashboard exposes cluster metrics and job history. On production servers, put it behind a VPN or SSH tunnel. Treat it like any admin panel on a Linux-managed host.

Cluster sizing for Nepal and global budgets

A single GPU cloud instance (Rs 15,000–40,000/month, ~USD 110–295) often beats a four-CPU cluster for deep learning. Ray shines when you need many CPUs for preprocessing, tuning trials, or ensemble inference. Split budget between head node stability and worker throughput. Never run the head node as a heavy worker unless you accept scheduling risk.

How does Ray Core schedule tasks and actors across workers?

Ray Core is the foundation of Ray: Distributed Python and ML. Tasks are stateless remote functions. Actors are stateful workers that persist across calls. Both return object refs immediately. You fetch results with ray.get() when ready.

import ray
import time

ray.init()

@ray.remote
def preprocess_batch(batch_id: int) -> dict:
    time.sleep(1)
    return {"batch_id": batch_id, "rows": 1000}

@ray.remote
class ModelScorer:
    def __init__(self):
        self.count = 0

    def score(self, features: list[float]) -> float:
        self.count += 1
        return sum(features) / len(features)

refs = [preprocess_batch.remote(i) for i in range(8)]
results = ray.get(refs)

scorer = ModelScorer.remote()
scores = ray.get([scorer.score.remote([1.0, 2.0, 3.0]) for _ in range(4)])
print(len(results), scores)
ray.shutdown()

Ray schedules those eight preprocess tasks in parallel up to available CPUs. The actor keeps self.count on one worker process. That pattern suits model warm-up: load weights once, score many requests.

Resource labels: CPU, GPU, and custom tags

Request GPUs per task or actor so Ray does not pile GPU jobs onto one worker.

@ray.remote(num_gpus=1)
def train_fold(fold_id: int):
    import torch
    device = "cuda" if torch.cuda.is_available() else "cpu"
    return {"fold_id": fold_id, "device": device}

Custom resources help when you license software per seat or reserve nodes for batch jobs. Label workers at start time with --resources='{"batch_slot": 4}'. Then request @ray.remote(resources={"batch_slot": 1}) in code.

Object refs and the distributed object store

Large numpy arrays or pandas frames pass by reference inside the cluster. Ray copies data only when a worker needs a local copy. That design cuts network traffic during map-reduce style ETL. It is conceptually similar to shared memory, but works across machines. For debugging payload shapes, pipe sample JSON through a JSON formatter before you ship tensors blindly to production.

Fault tolerance is partial by default. If a worker dies, Ray retries tasks whose lineage it can reconstruct. Stateful actors need restart policies you configure explicitly. Plan for idempotent tasks in production pipelines tracked with MLflow experiment logging.

How do Ray Data, Train, Tune, and Serve work in a full ML pipeline?

Most teams adopt Ray libraries one at a time. Ray Data ingests Parquet, CSV, JSON, and images from local disk or cloud storage. Ray Train adds distributed backends for common ML frameworks. Ray Tune wraps any training function and searches hyperparameters in parallel. Ray Serve exposes models as HTTP endpoints with batching and autoscaling hooks.

Ray Data example

import ray

ds = ray.data.read_parquet("s3://my-bucket/training/")
ds = ds.map_batches(lambda df: df.assign(label=df["label"].astype(int)))
train, test = ds.train_test_split(test_size=0.2)
train.write_parquet("/data/train/")

Ray Data lazy-evaluates transformations. It streams batches instead of loading full tables into driver memory. That matters when your feature store outgrows RAM on a Kathmandu office workstation.

Ray Train and Tune together

from ray import train, tune

def train_loop(config):
    import torch
    for epoch in range(config["epochs"]):
        loss = 1.0 / (epoch + 1)
        train.report({"loss": loss, "epoch": epoch})

 tuner = tune.Tuner(
    train_loop,
    param_space={"epochs": tune.choice([3, 5, 10])},
    tune_config=tune.TuneConfig(num_samples=4, metric="loss", mode="min"),
)
results = tuner.fit()
print(results.get_best_result(metric="loss", mode="min").metrics)

Tune launches multiple trials across the cluster. Train reports metrics each epoch. Integrate the same run IDs into MLflow or your internal dashboard. After deployment, watch for drift using the practices in monitor ML models in production.

Ray Serve deployment sketch

from ray import serve
from fastapi import FastAPI

app = FastAPI()

@serve.deployment(num_replicas=2, ray_actor_options={"num_cpus": 1})
@serve.ingress(app)
class SentimentAPI:
    @app.post("/predict")
    async def predict(self, payload: dict):
        text = payload.get("text", "")
        return {"label": "positive" if "good" in text else "neutral"}

serve.run(SentimentAPI.bind(), route_prefix="/")

Your Laravel app calls this service over HTTP from a queued job. Keep auth at the API gateway layer. Rate-limit upstream like any third-party API. The patterns in API rate limiting and abuse prevention apply directly.

Ray Serve can colocate preprocessing and model inference in one deployment graph. That cuts latency versus chaining three microservices on cold containers. For GPU inference at scale, compare with serving ML models with GPU on Kubernetes before you commit.

How does Ray compare to Dask, Spark, and Kubernetes-native ML stacks?

Pick tooling based on workload shape, team skills, and ops budget. Ray optimises for Python-native ML loops. Dask extends pandas and numpy parallelism. Spark dominates JVM-centric big data batches. Kubeflow orchestrates containers for teams already deep in Kubernetes.

Ray wins when you want one Python runtime from experiment to serving. Dask wins when you only need bigger pandas. Spark wins when your data platform is already JVM-heavy. Kubeflow wins when platform engineering is a dedicated function.

For web agencies shipping Laravel storefronts or legal-tech portals, Ray often lives in a sidecar tier. The public site stays on PHP. Batch scoring runs on Ray overnight. Results land in MySQL via a sync job. That split keeps PCI and PII boundaries cleaner than embedding Python inside PHP-FPM workers.

Production gotchas I watch for

1. Version skew: Pin Ray, Python, and CUDA versions in a lock file. Mixed worker images cause silent import failures.
2. Memory pressure: Object store fills when you return huge arrays from many tasks. Spill to disk or write to Parquet instead.
3. Head node overload: Keep heavy compute off the head. Schedule drivers on a workstation or CI runner.
4. Security: Ray ports are not authenticated by default. Isolate clusters in private subnets.
5. Cost spikes: Tune num_samples carefully. Parallel trials burn GPU hours fast. Track spend like AI rate limits and cost optimization for API workloads.

Official docs at docs.ray.io stay current with API changes. Cross-check cluster install steps against the Ray cluster deployment guide there before you automate with Ansible or Terraform.

Connecting Ray to enterprise web systems

Most businesses I work with do not want ML infrastructure touching their customer-facing CMS. The stable pattern: Ray Serve behind an internal load balancer, Laravel queues for async calls, Redis for result caching, and structured logging exported to your existing stack. That mirrors distributed build agents—specialised workers, central orchestration.

If you need custom scoring for an eCommerce catalog, batch nightly embeddings with Ray Data. Store vectors in PostgreSQL pgvector or a dedicated search engine. The WooCommerce or Laravel frontend reads precomputed scores at request time. See Quick And Easy Nepalese Grocery for the kind of operational eCommerce site where offline batch jobs beat inline inference.

For roles and hiring context, read AI engineer vs ML engineer vs data scientist. Ray skills sit closest to the ML engineer lane. Your PHP developer should integrate APIs, not rewrite Tune configs.

Key Takeaways

• Ray: Distributed Python and ML scales tasks, actors, data, training, tuning, and serving with one Python-centric runtime.
• Start local with ray.init(), then grow to a head-plus-worker cluster without rewriting application code.
• Use Ray Serve for HTTP inference; keep your Laravel or WordPress app as the public edge with queue-based calls.
• Pick Ray over Dask when you need training, tuning, and serving together; pick Kubeflow when Kubernetes is already your platform standard.
• Pin versions, lock down network access, and monitor object store memory before you put Ray on production GPU bills.
• Pair Ray batch jobs with MLflow tracking and drift monitoring so models do not silently rot after launch.

People Also Ask

Is Ray free to use in production?

Yes. Ray is open source under the Apache 2.0 license. You pay for compute, storage, and engineering time. Anyscale offers managed Ray hosting, but self-hosted clusters on Ubuntu VMs or cloud instances are common and cost-controlled.

Does Ray replace Kubernetes?

No. Ray is an application runtime. Kubernetes is an container orchestrator. Many teams run Ray on bare VMs first, then move to Kubernetes with the KubeRay operator when pod scheduling and autoscaling matter. Both layers solve different problems.

Can Ray run LLM fine-tuning workloads?

Yes. Ray Train integrates with PyTorch and Hugging Face trainers for distributed fine-tuning. You still need sufficient GPU memory and a data pipeline that respects token limits. Ray orchestrates workers; it does not remove hardware constraints.

How is Ray different from Celery for Python tasks?

Celery excels at async job queues for web apps—email, PDF generation, webhooks. Ray targets numerical and ML workloads with a distributed object store, GPU scheduling, and ML libraries. Use Celery inside Laravel; use Ray for model training and batch inference.

Ship distributed Python without losing your web stack

Ray: Distributed Python and ML earns its place when Python workloads outgrow one machine but your product still lives on PHP, WordPress, or a mixed stack. Start with Ray Core on a single GPU box. Add Tune when manual hyperparameter sweeps waste weeks. Add Serve when batch scripts cannot meet API latency targets. Keep experiment tracking and drift checks in your existing ops rhythm.

If you want help wiring Ray inference into a Laravel portal, eCommerce backend, or legal-tech workflow, contact us or explore enterprise application development and API development services. You can also browse the portfolio for production systems that combine web apps with background processing, or read more on the blog about distributed tracing and DVC for ML data versioning.