Parse JSON on the CLI with jq

By Kokil Thapa | Last reviewed: September 2026

API responses, webhook payloads, and CI logs arrive as JSON blobs long before they reach your application code. When you only need one field, a count, or a quick sanity check, opening an IDE to write a throwaway script wastes time. That is why engineers parse JSON on the CLI with jq—a single-purpose filter language built for pipes, shells, and production debugging. On real client projects I maintain REST APIs and webhook integrations, jq is the first tool I reach for after curl.

What is jq and why parse JSON on the CLI with jq?

jq is a lightweight command-line JSON processor. You feed it JSON and a filter expression; it returns transformed JSON or scalar values. Unlike general-purpose languages, jq speaks JSON natively. Arrays, objects, null, and nested paths are first-class.

The mental model is simple: JSON flows in, a filter transforms it, stdout carries the result. That fits Unix philosophy. You chain jq with curl, grep, awk, and sort without leaving the terminal.

I use jq daily when debugging Laravel API responses, payment gateway callbacks, and GitLab CI job output. PHP handles business logic in the app; jq handles inspection at the edge. For larger in-app parsing patterns, see our guide on PHP JSON handling for large payloads.

Three situations make jq the right choice:

• You need one field from a large API response during incident response.
• A webhook payload landed in a log file and you must confirm a signature or order ID.
• CI prints JSON test reports and you want failure counts without a custom parser.

For browser-based pretty-printing when you already have the raw string copied, the online JSON formatter on this site is handy. jq wins when the data lives on a server, in a pipeline, or inside automation.

How do you install jq on Linux, macOS, and Windows?

Installation takes one command on most systems. Package managers ship stable builds; the official project lives at jqlang.github.io.

Ubuntu and Debian

sudo apt update
sudo apt install jq
jq --version

On Ubuntu 22.04 and 24.04 servers I manage for Linux system administration clients, jq is part of the standard toolkit alongside curl and git.

macOS

brew install jq
jq --version

Windows

Use Chocolatey, Scoop, or WSL. Inside WSL, install with apt as on Linux. Native Windows builds exist, but most developers prefer WSL for pipe compatibility.

Verify with a minimal example

echo '{"name":"Kokil","role":"developer"}' | jq '.name'
# output: "Kokil"

If you see command not found, your PATH is wrong or the package did not install. If you see parse error, the input is not valid JSON—often trailing commas or single quotes from manual edits.

How do you filter and transform JSON with jq filters?

jq filters are expressions. The dot . refers to the current value. Bracket notation reaches keys: .user.email. Pipes chain operations left to right, like shell pipes but inside jq.

Core operators every developer should know

1. Field access: .data.items[0].id — drill into nested structures.
2. Array iteration: .orders[] — emit one output per element.
3. Selection: select(.status == "failed") — keep matching objects.
4. Construction: {id, total: .amount} — build new objects.
5. Strings and math: "\(.first) \(.last)" and .price * .qty.

Example: extract paid order totals from an eCommerce webhook payload.

cat webhook.json | jq '.orders[] | select(.payment_status=="paid") | {id, total: .grand_total}'

That pattern mirrors what I debug on Laravel eCommerce projects when Khalti or Stripe callbacks misbehave. I pull the file from storage, run jq, and compare fields against application logs.

Raw output for shell scripts

By default jq JSON-encodes strings, adding quotes. For shell variables, use raw mode:

TOKEN=$(curl -s https://api.example.com/auth | jq -r '.access_token')
echo "Token length: ${#TOKEN}"

The -r flag strips JSON string quoting. Use it whenever the next step expects plain text.

Compact vs pretty output

jq -c '.items[]' large-response.json > items.ndjson
jq '.' messy.json > pretty.json

-c emits one JSON object per line—ideal for streaming and while read loops. Pretty mode helps human review during debugging.

How do you parse JSON from APIs and log files in production workflows?

Production usage rarely stops at pretty-printing. You combine jq with HTTP clients, log tailing, and exit codes for automation.

curl and REST APIs

curl -s -H "Authorization: Bearer $TOKEN" \
  "https://api.example.com/v1/orders?status=pending" \
  | jq '[.data[] | {id, customer: .customer_name, total: .amount_npr}]'

For paginated APIs, loop with shell until .meta.has_more is false. Store each page, merge with jq -s 'add', or process line by line with -c.

When designing APIs that jq-friendly clients consume, consistent key naming matters. Mixed camelCase and snake_case makes filters ugly. Our API rate limiting guide covers another side of the same integration work.

Webhook and payment debugging

Payment gateways post JSON to Laravel routes. When a callback fails, I copy the raw body from storage/logs or nginx access logs into a temp file:

jq '.transaction_uuid, .status, .total_amount' khalti-callback.json
jq 'has("signature") and (.signature | length > 0)' khalti-callback.json

Boolean checks return true or false as JSON—easy to grep in scripts. For Base64 fields inside JSON, decode in a second step or use the Base64 encoder and decoder for manual inspection.

GitLab CI and JSON reports

Test runners often emit JSON summaries. On sister sites using Deployer and GitLab CI, I parse failure lists without opening the web UI:

jq '[.tests[] | select(.status=="failed") | .name] | length' report.json
jq -r '.tests[] | select(.status=="failed") | .name' report.json

Pipe the name list into a Slack notification script or fail the job when the count exceeds zero. That pairs well with ideas from adding checks to your CI pipeline.

Working with NDJSON and log streams

Application logs sometimes use newline-delimited JSON—one object per line. jq handles this with -c input or explicit slurping:

tail -f /var/log/app/structured.log | jq -c 'select(.level=="error") | {at: .timestamp, msg: .message}'

Each line must be valid JSON. Multi-line stack traces break jq unless you preprocess. Fix logging format upstream rather than fighting jq with regex.

Slurping multiple files

jq -s 'map(.invoice_id) | unique | length' invoices/*.json

The -s flag reads all inputs into one array. Useful for batch counts across exported API pages saved during migration work—similar problems to MySQL vs PostgreSQL JSON handling at the database layer, but at the file level.

How does jq compare to Python, PHP, and other JSON CLI tools?

jq is not a replacement for application code. It complements languages you already run in production.

Python one-liners work for quick tasks:

python3 -c "import sys,json; d=json.load(sys.stdin); print(d['meta']['total'])"

That gets painful with nested arrays and conditionals. jq expresses the same logic in fewer characters and clearer intent. PHP’s json_decode() belongs inside Laravel controllers and queued jobs—not in a five-second terminal check.

For LLM output that must conform to schema, application validation still wins. See structured outputs and JSON mode from LLMs for the app-layer approach. jq remains useful to inspect raw model responses before you trust them in code.

Config format choice is a separate decision. JSON as API wire format differs from JSON as human-edited config—our JSON vs YAML vs TOML comparison covers that angle.

What advanced jq techniques help on real client projects?

Once basics are solid, a handful of advanced patterns cover most remaining cases.

Default values with alternative operator

jq '.discount // 0' order.json
jq '.customer.phone // "N/A"' order.json

The // operator returns the right side when the left is null or false. Essential for optional API fields.

Reusable filters in files

cat orders.json | jq -f extract-paid.jq

Store complex filters in version control next to deploy scripts. Teams on custom software projects can share the same filter across staging and production without copy-paste errors.

Exit codes for automation

if jq -e '.errors | length > 0' response.json >/dev/null; then
  echo "API returned errors"
  exit 1
fi

-e sets exit status 1 when the filter yields null, false, or empty. CI jobs can fail on bad API health checks without custom scripts.

Performance on large JSON files

jq loads entire documents into memory by default. Multi-gigabyte exports need streaming strategies:

• Split NDJSON at export time.
• Use jq -c and process one record per line in a while loop.
• Push heavy aggregation into MySQL 9.7 or PostgreSQL 18 JSON functions when data already lives in the database.

For regex inside string fields, jq’s built-in test() helps. Pair with the regex tester when building patterns offline.

The official jq manual documents every builtin function. JSON syntax rules come from RFC 8259. Bookmark both when filters grow beyond simple path access.

On legal-tech portals such as Notary Nepal, lead webhooks and form payloads are JSON-shaped. jq lets support staff verify submissions without database access. That separation keeps production data safer during triage.

Key Takeaways

• Install jq once on every server and dev machine—you will use it weekly.
• Master .path, [], select(), -r, and // before reaching for Python one-liners.
• Combine curl, jq, and -e exit codes for API health checks in CI and cron.
• Use -c and NDJSON for large log streams; avoid loading gigabyte files whole.
• Keep complex filters in .jq files under version control alongside deploy scripts.
• Parse JSON on the CLI with jq for inspection; validate and enforce business rules in application code.

People Also Ask

Can jq modify JSON files in place?

jq writes to stdout by default. Redirect to a temp file, verify output, then move it into place. Piping directly back into the same file truncates before read completes. Use jq … file.json > file.json.tmp && mv file.json.tmp file.json for safe updates.

How do you parse JSON with jq when the API returns HTML errors?

jq fails with a parse error when input is HTML. Check HTTP status first: curl -sf … or curl -w '%{http_code}'. Store responses with headers during debugging so you can see 502 pages disguised as failures.

Does jq support JSON Schema validation?

jq filters express structural checks—has(), type tests, required keys—but not full JSON Schema. For schema validation in pipelines, pair jq extraction with a dedicated validator or validate inside your Laravel or Node application before persistence.

Is jq available on shared hosting?

Most shared hosts do not install jq. SSH VPS or dedicated servers do. For browser-only workflows without shell access, use a JSON formatter tool or run jq locally against downloaded exports.

Build JSON-ready systems that are easy to debug

Knowing how to parse JSON on the CLI with jq saves hours across API integrations, payment webhooks, and deployment troubleshooting. The skill shines brightest when your applications emit clean, consistent JSON in the first place. If you are shipping Laravel APIs, eCommerce callbacks, or automation around AI integration workflows, structured payloads and sensible logging matter as much as the filters you run against them.

Browse the project portfolio for examples of production integrations, or read more on the technical blog. When you want help designing JSON APIs, hardening webhooks, or setting up CI checks around external services, get in touch through the contact page—a short jq filter often becomes the first proof that the data layer is wired correctly.