4 Tricks to Effectively Use JSON in Python

By Erik van Baaren, Python 3 Guide

Illustration by author

Python has two data types that, together, form the perfect tool for working with JSON: dictionaries and lists. Let's explore how to:

• load and write JSON
• Pretty-print and validate JSON on the command line
• Do advanced queries on JSON docs by using JMESPath

1. Decoding JSON

 
Python ships with a powerful and elegant JSON library. It can be imported with:

import json

Decoding a string of JSON is as simple as json.loads(…) (short for load string).

It converts:

• objects to dictionaries
• arrays to lists,
• booleans, integers, floats, and strings are recognized for what they are and will be converted into the correct types in Python
• Any null will be converted into Python’s None type

Here’s an example of json.loads in action:

>>> import json
>>> jsonstring = '{"name": "erik", "age": 38, "married": true}'
>>> json.loads(jsonstring)
{'name': 'erik', 'age': 38, 'married': True}

2. Encoding JSON

 
The other way around is just as easy. Use json.dumps(…) (short for ‘dump to string) to convert a Python object consisting of dictionaries, lists, and other native types into a string:

>>> myjson = {'name': 'erik', 'age': 38, 'married': True}
>>> json.dumps(myjson)
'{"name": "erik", "age": 38, "married": true}'

This is the exact same document, converted back to a string! If you want to make your JSON document more readable for humans, use the indent option:

>>> print(json.dumps(myjson, indent=2))
{
  "name": "erik",
  "age": 38,
  "married": true
}

3. Command-line usage

 
The JSON library can also be used from the command-line, to validate and pretty-print your JSON:

$ echo "{ \"name\": \"Monty\", \"age\": 45 }" | \
python3 -m json.tool
{
    "name": "Monty",
    "age": 45
}

As a side note: if you’re on a Mac or Linux and get the chance to install it, look into the jq command-line tool too. It’s easy to remember, colorizes your output, and has loads of extra features as explained in my article on becoming a command-line ninja.

jq will pretty-print your JSON by default

4. Searching through JSON with JMESPath

Screenshot by author

JMESPath is a query language for JSON. It allows you to easily obtain the data you need from a JSON document. If you ever worked with JSON before, you probably know that it’s easy to get a nested value.

For example: doc["person"]["age"] will get you the nested value for age in a document that looks like this:

{
  "persons": {
    "name": "erik",
    "age": "38"
  }
}

But what if you want to extract all the age-fields from an array of persons, in a document like this:

{
  "persons": [
    { "name": "erik", "age": 38 },
    { "name": "john", "age": 45 },
    { "name": "rob", "age": 14 }
  ]
}

We could write a simple loop and loop over all the persons. Easy peasy. But loops are slow and introduce complexity to your code. This is where JMESPath comes in!

This JMESPath expression will get the job done:

persons[*].age

It will return an array with all the ages: [38, 45, 14].

Say you want to filter the list, and only get the ages for people named ‘erik’. You can do so with a filter:

persons[?name=='erik'].age

See how natural and quick this is?

JMESPath is not part of the Python standard library, meaning you’ll need to install it with pip or pipenv. For example, when using pip in in virtual environment:

$ pip3 install jmespath
$ python3
Python 3.8.2 (default, Jul 16 2020, 14:00:26)
>>> import jmespath
>>> j = { "people": [{ "name": "erik", "age": 38 }] }
>>> jmespath.search("people[*].age", j)
[38]
>>>

You’re now ready to start experimenting! Make sure to try the interactive tutorial and view the examples on the JMESPath site!

If you have more JSON tips or tricks, please share them in the comments!

Follow me on Twitter to get my latest articles first and make sure to visit my Python 3 Guide. This article was originally published there as well.

 
The Most Important Python Concept That You Need to Understand
Learn all about the building blocks of the Python language

 
Bio: Erik van Baaren is a software engineer and the webmaster of Python 3 Guide, where you can learn both beginner and advanced Python topics. He's also a writer of tech articles on Medium.com.

Original. Reposted with permission.

Related:

• 3 Advanced Python Features You Should Know
• Why Learn Python? Here Are 8 Data-Driven Reasons
• 10 Things You Didn’t Know About Scikit-Learn