jq JSON Processing Expert knowledge for processing, querying, and transforming JSON data using jq, the lightweight and flexible command-line JSON processor. Core Expertise JSON Operations Query and filter JSON with path expressions Transform JSON structure and shape Combine, merge, and split JSON data Validate JSON syntax and structure Data Extraction Extract specific fields from JSON objects Filter arrays based on conditions Navigate nested JSON structures Handle null values and missing keys Essential Commands Basic Querying

Pretty-print JSON

jq '.' file.json

Extract specific field

jq '.fieldName' file.json

Extract nested field

jq '.user.email' file.json

Extract array element

jq '.[0]' file.json jq '.items[2]' file.json Array Operations

Get array length

jq '.items | length' file.json

Map over array

jq '.items | map(.name)' file.json

Filter array

jq '.items[] | select(.active == true)' file.json jq '.users[] | select(.age > 18)' file.json

Sort array

jq '.items | sort_by(.name)' file.json jq '.items | sort_by(.date) | reverse' file.json

Get first/last elements

jq '.items | first' file.json jq '.items | last' file.json

Unique values

jq '.tags | unique' file.json Object Operations

Get all keys

jq 'keys' file.json jq '.config | keys' file.json

Get all values

jq '.[] | values' file.json

Select specific fields

jq '{name, email}' file.json jq '{name: .fullName, id: .userId}' file.json

Add field

jq '. + {newField: "value"}' file.json

Delete field

jq 'del(.password)' file.json

Merge objects

jq '. * {updated: true}' file.json Filtering and Conditions

Select with conditions

jq 'select(.status == "active")' file.json jq '.[] | select(.price < 100)' file.json

Multiple conditions (AND)

jq '.[] | select(.active and .verified)' file.json jq '.[] | select(.age > 18 and .country == "US")' file.json

Multiple conditions (OR)

jq '.[] | select(.type == "admin" or .type == "moderator")' file.json

Exists / has field

jq '.[] | select(has("email"))' file.json jq 'select(.optional != null)' file.json

Not condition

jq '.[] | select(.status != "deleted")' file.json String Operations

String interpolation

jq '"Hello, (.name)"' file.json

Convert to string

jq '.id | tostring' file.json

String contains

jq '.[] | select(.email | contains("@gmail.com"))' file.json

String starts/ends with

jq '.[] | select(.name | startswith("A"))' file.json jq '.[] | select(.file | endswith(".json"))' file.json

Split string

jq '.path | split("/")' file.json

Join array to string

jq '.tags | join(", ")' file.json

Lowercase/uppercase

jq '.name | ascii_downcase' file.json jq '.name | ascii_upcase' file.json Pipes and Composition

Chain operations

jq '.items | map(.name) | sort | unique' file.json

Multiple filters

jq '.users[] | select(.active) | select(.age > 18) | .email' file.json

Group by

jq 'group_by(.category)' file.json jq 'group_by(.status) | map({status: .[0].status, count: length})' file.json Output Formatting

Compact output (no pretty-print)

jq -c '.' file.json

Raw output (no quotes for strings)

jq -r '.message' file.json jq -r '.items[] | .name' file.json

Output as tab-separated values

jq -r '.[] | [.name, .age, .email] | @tsv' file.json

Output as CSV

jq -r '.[] | [.name, .age, .email] | @csv' file.json

Output as JSON array on one line

jq -c '[.items[]]' file.json Input/Output Options

Read from stdin

cat file.json | jq '.items' curl -s https://api.example.com/data | jq '.results'

Multiple input files

jq -s '.' file1.json file2.json

Slurp into array

Write to file

jq '.filtered' input.json

output.json

In-place edit (use sponge from moreutils)

jq '.updated = true' file.json | sponge file.json Advanced Patterns

Recursive descent

jq '.. | .email? // empty' file.json

Reduce (sum, accumulate)

jq '[.items[].price] | add' file.json jq 'reduce .items[] as $item (0; . + $item.price)' file.json

Variable assignment

jq '.items[] | . as $item | $item.name + " - " + ($item.price | tostring)' file.json

Conditional (if-then-else)

jq '.items[] | if .price > 100 then "expensive" else "affordable" end' file.json jq 'if .error then .error else .data end' file.json

Try-catch for error handling

jq '.items[] | try .field catch "not found"' file.json

Flatten nested arrays

jq '.items | flatten' file.json jq '.items | flatten(1)' file.json

Flatten one level

Real-World Examples

Extract all emails from nested structure

jq '.. | .email? // empty' users.json

Get unique list of all tags across items

jq '[.items[].tags[]] | unique' data.json

Count items by status

jq 'group_by(.status) | map({status: .[0].status, count: length})' items.json

Transform API response to simple list

jq '.results[] | {id, name: .full_name, active: .is_active}' response.json

Filter GitHub workflow runs (recent failures)

gh run list --json status,conclusion,name,createdAt | \ jq '.[] | select(.conclusion == "failure") | {name, createdAt}'

Extract package.json dependencies with versions

jq '.dependencies | to_entries | map("(.key)@(.value)")' package.json

Merge two JSON files

jq -s '.[0] * .[1]' base.json override.json

Create summary from log data

jq 'group_by(.level) | map({level: .[0].level, count: length, samples: [.[].message][:3]})' logs.json Best Practices Query Construction Start simple, build complexity incrementally Test filters on small datasets first Use -c flag for compact output in scripts Use -r flag for raw strings (no quotes) Performance Use select() early in pipeline to reduce data Use targeted queries with specific paths for large files Stream large JSON with --stream flag Consider jq -c for faster processing Error Handling Use ? operator for optional access: .field? Use // empty to filter out nulls/errors Use try-catch for graceful error handling Check for field existence with has("field") Readability Break complex queries into multiple steps Use variables with as $var for clarity Add comments in shell scripts Format multi-line jq programs for readability Common Patterns API Response Processing

GitHub API: Get PR titles and authors

gh pr list --json title,author,number | \ jq -r '.[] | "#(.number) - (.title) by @(.author.login)"'

REST API: Extract and flatten pagination

curl -s "https://api.example.com/items" | \ jq '.data.items[] | {id, name, status}' Configuration Files

Extract environment-specific config

jq '.environments.production' config.json

Update configuration value

jq '.settings.timeout = 30' config.json

config.updated.json

Merge base config with environment overrides

jq -s '.[0] * .[1]' base-config.json prod-config.json Log Analysis

Count errors by type

jq 'select(.level == "error") | .type' logs.json | sort | uniq -c

Extract error messages with timestamps

jq -r 'select(.level == "error") | "(.timestamp) - (.message)"' logs.json

Group by hour and count

jq -r '.timestamp | split("T")[1] | split(":")[0]' logs.json | sort | uniq -c Data Transformation

CSV to JSON (with headers)

jq -R -s 'split("\n") | .[1:] | map(split(",")) | map({name: .[0], age: .[1], email: .[2]})' data.csv

JSON to CSV

jq -r '.[] | [.name, .age, .email] | @csv' data.json

Flatten nested structure

jq '[.items[] | {id, name, category: .meta.category}]' nested.json Troubleshooting Invalid JSON

Validate JSON syntax

jq empty file.json

Returns exit code 0 if valid

Find syntax errors

jq '.' file.json 2

&1 | grep "parse error" Empty Results

Debug: Print entire structure

jq '.' file.json

Debug: Check field existence

jq 'keys' file.json jq 'type' file.json

Check if array, object, etc.

Debug: Show all values

jq '.. | scalars' file.json Type Errors

Check field types

jq '.field | type' file.json

Convert types safely

jq '.id | tonumber' file.json jq '.count | tostring' file.json

Handle mixed types

jq '.items[] | if type == "array" then .[] else . end' file.json Performance Issues

Stream large files

jq --stream '.' large-file.json

Process line by line

cat large.json | jq -c '.[]' | while read -r line ; do echo " $line " | jq '.field' done Integration with Other Tools

With curl (API calls)

curl -s "https://api.github.com/users/octocat" | jq '.name, .bio'

With gh CLI (GitHub operations)

gh api repos/owner/repo/issues | jq '.[] | {number, title, state}'

With find (batch processing)

find . -name "package.json" -exec jq '.version' { } \ ;

With xargs (parallel processing)

cat urls.txt | xargs -I { } curl -s { } | jq '.data'

With yq (YAML to JSON conversion)

yq eval -o = json file.yaml | jq '.specific.field' Quick Reference Operators .field - Access field .[] - Iterate array/object | - Pipe (chain operations) , - Multiple outputs ? - Optional (suppress errors) // - Alternative operator (default value) Functions keys , values - Object keys/values length - Array/object/string length map() , select() - Array operations sort , sort_by() - Sorting group_by() - Grouping unique - Remove duplicates add - Sum numbers or concatenate has() - Check field existence type - Get value type Type Conversions tostring , tonumber - Convert types @csv , @tsv , @json - Format output split() , join() - String/array conversion Installation

macOS (Homebrew)

brew install jq

Ubuntu/Debian

sudo apt-get install jq

Verify installation

jq --version Resources Official Manual : https://stedolan.github.io/jq/manual/ jq Playground : https://jqplay.org/ Tutorial : https://stedolan.github.io/jq/tutorial/