jq

jq is a lightweight and flexible command-line JSON processor.

Filter on Object key/value

Select the array in .items, loop through and select the items which have a .name equal to "thisname", then return the .name and .status

cat file.json | jq '.items | .[] | select(.name == "thisname") | {name: .name, status: .status}'

Sort and diff JSON filed

Use the --sort-keys flag to sort the files and then compare using diff

diff <(cat /tmp/A.json | jq --sort-keys) <(cat /tmp/B.json | jq --sort-keys)

Walk an object tree looking for a specific key

Walk through an entire object and return all instances of a key (in the below example .target)

echo '{"target": ["a","b","c"], "nested": {"nothingHere": {"somethingElse": ["z", "y", "x"]}, "checkHere": {"target": ["d","e","f"], "anotherLevel": {"not-target": ["w", "v", "u"], "target": ["g", "h", "i"]}}}}' | jq '..|.target?'

Parse string as JSON

$ echo '{"a": 1, "b": 2, "c": "{\"id\":\"9ee ...\",\"parent\":\"abc...\"}"}' | jq -r '.c | fromjson'
{
  "id": "9ee ...",
  "parent": "abc..."
}

Filter on Object key/value (contains)

Loop through an array and select the items which have a .name which contains "someText", then return the .name and .status

cat file.json | jq '.[] | select( .name | contains("someText") ) | {name: .name, status: .status}'

Filter on Object key existing

Select the array in .items, loop through and select the items which have the specified key, then return the .name

cat file.json | jq -r '.items | .[] | select(.a.path.to.an.array[]?.key != null) | .name'

Filter with multiple conditions

Use or in a select statement, to filter on multiple conditions

cat file.json | jq -r '.items | .[] | select((.status == "Running") or (.status == "Scheduled") or .status == "ConfirmationNeeded") | .name'

Filter on Object key/value (not contains)

Loop through an array and select the items which don’t have a .name which contains "some-text", then return the .name.

cat file.json | jq -r '.items | .[] | select(.name | contains("some-text") | not) | .name'

Filter Key with startswith

echo '{"items": [{"hostname": "aThing"}, {"hostname": "aThing2"}, {"hostname": "bThing"}]}' | jq -r '.items | .[] | select(.hostname | startswith("aT"))'

Use if statements

cat << EOF > data.json
{
  "items": [
    {
      "name": "nameone",
      "nonimportantfields": {
        "foo": "bar",
        "hello": "world"
      },
      "importantfields": {
        "superimportant": "foo"
      }
    },
    {
      "name": "nametwo",
      "nonimportantfields": {
        "foo": "bar",
        "hello": "world"
      },
      "importantfields": {
        "superimportant": "bar"
      }
    },
    {
      "name": "namethree",
      "superimportant": true,
      "nonimportantfields": {
        "foo": "bar",
        "hello": "world"
      }
    }
  ]
}
EOF

cat data.json | jq '.items | .[] | if .importantfields and .nonimportantfields.foo == "bar" then .name else empty end'
(out) "nameone"
(out) "nametwo"

Filter Key with endswith

echo '{"items": [{"hostname": "aThing1"}, {"hostname": "aThing2"}, {"hostname": "bThing1"}]}' | jq -r '.items | .[] | select(.hostname | endswith("ing1"))'

Get length of an array

cat file.json | jq -r '.items | length'

Check if a key or value exists

cat << EOF > data.json
{
  "name": "name-field",
  "skills": {
    "cloud": "aws",
    "devops": "git"
  }
}
EOF

## Check if a key exists
cat data.json | jq 'has("notexists")'                 ## returns false
cat data.json | jq 'has("skills")'                    ## returns true
cat data.json | jq '.skills | has("devops")'          ## returns true

## Check if a value exists
cat data.json | jq '.name | contains("notexists")'    ## returns false
cat data.json | jq '.name | contains("debjeet")'      ## returns true
cat data.json | jq '.skills.cloud | contains("aws")'  ## returns true

Combine has with select to extract the objects which are true

cat << EOF > data.json
{
  "items": [
    {
      "name": "name-one",
      "non-important-fields": {
        "foo": "bar",
        "hello": "world"
      },
      "important-fields": {
        "super-important": "foo"
      }
    },
    {
      "name": "name-two",
      "non-important-fields": {
        "foo": "bar",
        "hello": "world"
      },
      "important-fields": {
        "super-important": "bar"
      }
    },
    {
      "name": "name-three",
      "super-important": true,
      "non-important-fields": {
        "foo": "bar",
        "hello": "world"
      }
    }
  ]
}
EOF

cat data.json | jq -r '.items | .[] | {name: .name, exists: .["important-fields"]? | has("super-important")}' | jq -r 'select(.exists == true)'
(out){
(out)  "name": "name-one",
(out)  "exists": true
(out)}
(out){
(out)  "name": "name-two",
(out)  "exists": true
(out)}

Group by and count

echo '{"items": [ {"Priority": "High"},{"Priority": "High"},{"Priority": "High"},{"Priority": "High"},{"Priority": "Low"},{"Priority": "Low"},{"Priority": "Low"} ]}' | jq -r '.items | group_by(.Priority)[] | {Priority: .[0].Priority, Count: length}'
(out){
(out)  "Priority": "High",
(out)  "Count": 4
(out)}
(out){
(out)  "Priority": "Low",
(out)  "Count": 3
(out)}

Select, Group by and count

echo '{"items": [ {"Priority": "High"},{"Priority": "High"},{"Priority": "High"},{"Priority": "High"},{"Priority": "Medium"},{"Priority": "Medium"},{"Priority": "Low"},{"Priority": "Low"},{"Priority": "Low"} ]}' | jq -r '.items | .[] | select((.Priority == "High") or (.Priority == "Medium"))' | jq -s 'group_by(.Priority)[] | {Priority: .[0].Priority, Count: length}'
(out){
(out)  "Priority": "High",
(out)  "Count": 4
(out)}
(out){
(out)  "Priority": "Medium",
(out)  "Count": 2
(out)}

Select fields starting with

Select all fields that start with prefixA, from within the anotherField object

echo '{"aField": {}, "anotherField": { "prefixAOne": "foo", "prefixATwo": "foo", "prefixAThree": "foo", "prefixBOne": "bar", "prefixBTwo": "bar" }, "lastField": true}' | jq -r '.anotherField | to_entries[] | select(.key|startswith("prefixA"))'
(out){
(out)  "key": "prefixAOne",
(out)  "value": "foo"
(out)}
(out){
(out)  "key": "prefixATwo",
(out)  "value": "foo"
(out)}
(out){
(out)  "key": "prefixAThree",
(out)  "value": "foo"
(out)}

Add a field to an object

Assign bar to argument foo and then add the "foo": "bar" key/value to the object

echo '{"hello": "world"}' | jq --arg foo bar '. + {foo: $foo}'
(out){
(out)  "hello": "world",
(out)  "foo": "bar"
(out)}

Preserve nested objects with *

echo '{"hello": {"value": "world"}}' | jq '. * {"hello": {foo: ("bar")}}'
(out){
(out)  "hello": {
(out)    "value": "world",
(out)    "foo": "bar"
(out)  }
(out)}

Using + will not preserve nested objects

echo '{"hello": {"value": "world"}}' | jq '. + {"hello": {foo: ("bar")}}'
(out){
(out)  "hello": {
(out)    "foo": "bar"
(out)  }
(out)}

Concat strings

echo '{"hello": {"value": "world"}}' | jq --arg foo bar '. * {"hello": {foo: ("not" + $foo)}}'
(out){
(out)  "hello": {
(out)    "value": "world",
(out)    "foo": "notbar"
(out)  }
(out)}