Concourse CI Fundamentals

By Kokil Thapa | Last reviewed: September 2026

Concourse CI fundamentals start with a simple idea: your pipeline is the source of truth, not a UI you click together. Concourse runs every step inside containers on workers you control. Jobs pull inputs from versioned resources, execute tasks, and push outputs forward. If you already run GitLab CI for Laravel deployments, Concourse feels different—but the payoff is strict reproducibility and a clear audit trail. This guide walks through installation, pipeline YAML, the fly CLI, and the production patterns I use alongside Linux server administration work.

What is Concourse CI and how does its architecture work?

Concourse is an open-source continuous integration system built around pipelines-as-code. You declare everything in YAML. The web UI renders that YAML; it does not store hidden state.

Three core components matter on day one:

• ATC — the web UI, scheduler, and API server.
• Workers — hosts that run job containers via Garden and Baggageclaim.
• fly — the CLI you use to log in, set pipelines, and trigger builds.

Every build step runs in a fresh container. That immutability is the main reason teams pick Concourse over click-to-configure tools. A failed step leaves artifacts you can inspect. A passed step means the exact same inputs produced the exact same container image or binary.

On production servers I maintain, Concourse often sits beside GitLab CI on sister sites. GitLab handles repo hosting and review. Concourse handles long-running integration flows that need strict resource pinning. Both can coexist if you draw clear boundaries.

How do you install Concourse CI on a Linux server?

The fastest path for learning is docker-compose on Ubuntu 22 or 24. Production teams usually move to a proper cluster with external PostgreSQL and multiple workers. Start small, then scale workers before you scale ATC replicas.

Quick-start with Docker Compose

Download the official compose file from the Concourse documentation. A minimal local stack needs PostgreSQL, the ATC, and at least one worker.

# docker-compose.yml (minimal local stack)
services:
  db:
    image: postgres:16
    environment:
      POSTGRES_USER: concourse
      POSTGRES_PASSWORD: concourse
      POSTGRES_DB: concourse

  web:
    image: concourse/concourse:7.13
    command: web
    ports:
      - "8080:8080"
    environment:
      CONCOURSE_POSTGRES_HOST: db
      CONCOURSE_POSTGRES_USER: concourse
      CONCOURSE_POSTGRES_PASSWORD: concourse
      CONCOURSE_POSTGRES_DATABASE: concourse
      CONCOURSE_EXTERNAL_URL: http://localhost:8080
      CONCOURSE_ADD_LOCAL_USER: test:test
      CONCOURSE_MAIN_TEAM_LOCAL_USER: test
    depends_on:
      - db

  worker:
    image: concourse/concourse:7.13
    command: worker
    privileged: true
    environment:
      CONCOURSE_TSA_HOST: web:2222
      CONCOURSE_TSA_PUBLIC_KEY: /concourse-keys/host_key.pub
      CONCOURSE_TSA_WORKER_PRIVATE_KEY: /concourse-keys/worker_key
    volumes:
      - ./keys:/concourse-keys
    depends_on:
      - web

Generate TSA keys before first boot:

mkdir keys && cd keys
wget https://concourse-ci.org/download-fly.html
docker run --rm -v "$PWD:/keys" concourse/concourse:7.13 \
  concourse generate-key -t rsa -f /keys/tsa_host_key
docker run --rm -v "$PWD:/keys" concourse/concourse:7.13 \
  concourse generate-key -t rsa -f /keys/worker_key
cp worker_key.pub authorized_worker_keys
cp tsa_host_key.pub host_key.pub

Bring the stack up, then install fly on your laptop. Log in and confirm the main team responds:

fly -t local login -c http://localhost:8080 -u test -p test
fly -t local sync
fly -t local workers

For production, put TLS in front of the ATC with Nginx or Caddy. Store secrets in a vault, not in plain compose files. That aligns with broader CI/CD secrets management practices you should already follow.

What are resources, jobs, and tasks in a Concourse pipeline?

These three primitives define every Concourse pipeline. Mix them correctly and builds stay predictable. Mix them poorly and you chase phantom failures for weeks.

Resources — versioned external state

A resource represents something Concourse can check for new versions. Common types include git, docker-image, s3, time, and registry-image. Resource checks run on an interval. When a new version appears, dependent jobs trigger automatically.

Jobs — ordered plans

A job is a named workflow. It defines inputs (resources), outputs (resources), and a plan—a list of steps. Jobs can run serially or in parallel within the plan. Public jobs expose build history; private jobs hide it from unauthenticated viewers.

Tasks — reusable scripts in containers

A task is the smallest executable unit. It references a task.yml file with an image, inputs, outputs, params, and a shell script or command. Tasks run inside the worker's Garden container runtime. Each task gets a clean filesystem unless you explicitly pass cached volumes.

Here is a minimal pipeline that tests a PHP application on every git push:

# pipeline.yml
resource_types:
  - name: git
    type: registry-image
    source:
      repository: concourse/git-resource

resources:
  - name: repo
    type: git
    source:
      uri: https://github.com/your-org/your-app.git
      branch: main

jobs:
  - name: unit-tests
    plan:
      - get: repo
        trigger: true
      - task: run-tests
        config:
          platform: linux
          image_resource:
            type: registry-image
            source:
              repository: php
              tag: "8.3-cli"
          inputs:
            - name: repo
              path: app
          run:
            path: sh
            args:
              - -c
              - |
                cd app
                curl -sS https://getcomposer.org/installer | php
                php composer.phar install --no-interaction
                vendor/bin/phpunit

Validate YAML before you push. A typo in resource_types fails silently until runtime. Use a JSON or YAML formatter locally, then run fly set-pipeline with --check-creds when credentials are involved.

How do you manage pipelines with the fly CLI?

Everything operational flows through fly. The target flag -t names your Concourse instance. I keep targets per environment: staging, production, local.

1. Log in once per target: fly -t prod login -c https://ci.example.com -u admin
2. Push pipeline config: fly -t prod set-pipeline -p my-app -c pipeline.yml
3. Preview diff before apply: add --preview to see job and resource changes.
4. Unpause after first push: new pipelines start paused; run fly -t prod unpause-pipeline -p my-app.
5. Trigger manually when debugging: fly -t prod trigger-job -j my-app/unit-tests -w
6. Inspect build logs: fly -t prod builds -j my-app/unit-tests then fly -t prod watch.

Store pipeline YAML in git beside your application code. Point a git resource at that repo. Use a self-update job that runs fly set-pipeline when the pipeline directory changes. That closes the loop: the pipeline updates itself, just like application code.

For credential injection, prefer Concourse credential managers. Built-in options integrate with Vault, CredHub, or custom backends. Never commit secrets into pipeline YAML. The ATC redacts them in the UI, but git history keeps everything forever. Scan repos with tools covered in secrets scanning with Gitleaks before you go live.

Task config versus inline task config

Inline task configs suit quick experiments. Production pipelines should reference external task.yml files inside the repo:

jobs:
  - name: deploy
    plan:
      - get: repo
        passed: [unit-tests]
        trigger: true
      - task: deploy-app
        file: repo/ci/tasks/deploy.yml

External tasks are easier to test locally with fly execute. That command runs a task on your workers using the same config the pipeline will use. It saves hours compared to push-and-pray debugging.

How does Concourse CI compare with GitLab CI, Drone, and Tekton?

Pick Concourse when you want strict pipeline immutability and resource-driven triggers. Pick GitLab CI when you already live inside GitLab and want tight merge-request integration. There is no universal winner—only fit for your team size and hosting model.

I've run GitLab CI on shared EC2 infrastructure for legal-tech sister sites with Deployer 7. Concourse enters the picture when a client wants isolated build workers or cross-repo orchestration without buying all of GitLab. Read the GitHub Actions vs GitLab CI comparison for another angle on hosted versus self-managed trade-offs.

Drone CI feels closest to Concourse in spirit—container-first, YAML-native—but Drone leans on Git webhooks while Concourse polls and checks resources. Tekton wins if you already operate Kubernetes at scale. TeamCity still suits .NET-heavy shops that want a rich GUI.

What production patterns and pitfalls should you know?

Concourse rewards discipline. Sloppy resource design creates circular triggers or jobs that never fire. These patterns keep pipelines maintainable on real projects.

Pin and re-run with confidence

The UI lets you pin a resource to a specific version. Re-run downstream jobs against that pin to reproduce a production build exactly. This is Concourse's killer feature for incident response. GitLab and GitHub Actions can approximate it, but Concourse makes version pins first-class citizens in the graph.

Use passed constraints for DAG ordering

The passed: constraint ensures job B only runs after job A succeeds on the same resource version. Without it, jobs race independently and you deploy untested commits.

jobs:
  - name: integration-tests
    plan:
      - get: repo
        passed: [unit-tests]
        trigger: true
      - task: run-integration
        file: repo/ci/tasks/integration.yml

Cache carefully — Concourse has no magic cache

Unlike GitLab's cache keyword, Concourse expects explicit cache resources or S3-backed volumes. For Composer and npm, many teams use a shared S3 resource or a custom cache resource type. Read build caching strategies for CI and adapt them to explicit resource outputs.

Database migrations in deploy jobs

On Laravel apps I ship, migration tasks run as a dedicated job after tests pass and before image promotion. Wrap migrations in a task with database credentials from Vault. See database migrations in CI/CD pipelines for ordering rules that apply regardless of CI engine.

Worker sizing and privileged tasks

Docker-in-Docker builds need privileged workers. That increases attack surface. Isolate privileged workers on a separate pool with network restrictions. Monitor disk usage—Baggageclaim volumes fill up when jobs fail before cleanup.

For Laravel specifically, mirror the test stages from Laravel testing with Pest in CI/CD. Swap the runner syntax for Concourse tasks. The testing philosophy stays identical.

Projects like Adventure Third Pole Trek use Laravel plus Livewire with GitLab CI today. Concourse would slot in if the team needed fan-in from multiple repos—app, infra, and content—or if compliance required pinned artifact replay. The application code would not change; only the pipeline layer would.

Operational costs on a Nepal VPS land around Rs 8,000–15,000/month (~USD 60–110) for a modest ATC plus two workers with PostgreSQL on the same host. That beats many SaaS CI bills once you run more than a handful of private repositories. Factor in your own maintenance time though—Concourse is not zero-ops.

When evaluating Argo Workflows or Bitbucket Pipelines, ask one question: do you need resource-native orchestration or tight SCM integration? Your answer picks the tool. Concourse sits firmly in the orchestration camp.

For broader platform context, see the Concourse GitHub repository and release notes before upgrading ATC images. Breaking changes in resource types appear in changelogs, not in your pipeline UI.

Key Takeaways

• Define Concourse pipelines in YAML with resources, jobs, and tasks—treat the UI as read-only confirmation, not the source of truth.
• Install ATC plus workers with external PostgreSQL and TLS before you call any setup production-ready.
• Use fly set-pipeline, --preview, and external task files so pipeline changes stay reviewable in git.
• Pin resource versions to reproduce builds exactly—a core Concourse CI fundamental that speeds up incident debugging.
• Segment privileged Docker-build workers from general test workers to reduce security exposure on shared infrastructure.
• Pair Concourse with Vault for secrets and explicit cache resources instead of expecting automatic dependency caching.

People Also Ask

Is Concourse CI still maintained in 2026?

Yes. Concourse remains actively developed on GitHub with regular releases in the 7.x line. The community is smaller than GitLab or GitHub Actions, but the project is stable for self-hosted teams that value pipeline immutability. Check the official release page before upgrading production ATC instances.

Do you need Kubernetes to run Concourse?

No. Many teams run Concourse on plain Ubuntu VMs with Docker or containerd workers. Kubernetes deployments exist via Helm charts, but they are optional. A single VPS with Docker Compose works for staging and small teams.

How is Concourse different from Jenkins?

Jenkins relies on plugins and mutable controller state. Concourse forbids manual UI tweaks—every change goes through YAML and fly. Jenkins offers more plugins and a larger ecosystem. Concourse offers stricter reproducibility and a cleaner pipeline graph with less configuration drift.

Can Concourse deploy Laravel applications?

Absolutely. Typical pipelines run Composer install, PHPUnit or Pest, build a container image, push to a registry, and trigger a deploy task over SSH or kubectl. The Laravel application does not care which CI engine runs the steps. Only the task containers and credentials differ.

Build reproducible pipelines with confidence

Mastering Concourse CI fundamentals means embracing pipelines-as-code, resource-driven triggers, and container isolation on workers you own. Start with a local Docker Compose stack, push a two-job pipeline, and practice pinning versions before you migrate production workloads. If you want help designing CI/CD for a Laravel, WordPress, or custom PHP project—or hardening the Linux hosts underneath—contact us or explore support and maintenance services. You can also browse the portfolio for production systems already running automated deploy pipelines, or read more on the blog about blue-green deployment and integration testing in CI.