Interacting with JSON data using JMESPath

Have you ever found yourself drowning in a sea of JSON data, desperately trying to extract just the right information? I know I have. In my recent Python project, I encountered this exact scenario all too often. I'd be faced with massive JSON files, and extracting the specific data I needed became a headache, involving endless nested loops and conditional statements.

That's when I stumbled upon JMESPath, a lifesaver in the world of data extraction. JMESPath provides a simple and powerful way to query and manipulate JSON data. It's like having a magic wand that lets you effortlessly pluck out exactly what you need from even the most complex JSON structures.

In this post, I'll walk you through an example of how JMESPath came to my rescue and revolutionized the way I work with JSON data. So buckle up, because once you see what JMESPath can do, you'll wonder how you ever managed without it. Let's dive in!

What is JMESPath?

JMESPath, a query language for JSON serves as your trusty companion in the realm of JSON data manipulation. With JMESPath, you can effortlessly filter, extract, and transform information from JSON datasets, making it an invaluable tool for anyone who regularly deals with JSON data.

Whether you're a seasoned developer or just dipping your toes into the world of JSON, having JMESPath in your toolkit can streamline your workflow and simplify the process of navigating through complex JSON structures.

Lets look at an example. We will use the following JSON data:

products = [
  {
    "id":59,
    "title":"Spring and summershoes",
    "price":20,
    "quantity":3,
    "total":60,
    "discountPercentage":8.71,
    "discountedPrice":55
  },
  {
    "id":88,
    "title":"TC Reusable Silicone Magic Washing Gloves",
    "price":29,
    "quantity":2,
    "total":58,
    "discountPercentage":3.19,
    "discountedPrice":56
  },
  {
    "id":18,
    "title":"Oil Free Moisturizer 100ml",
    "price":40,
    "quantity":2,
    "total":80,
    "discountPercentage":13.1,
    "discountedPrice":70
  },
  {
    "id":95,
    "title":"Wholesale cargo lashing Belt",
    "price":930,
    "quantity":1,
    "total":930,
    "discountPercentage":17.67,
    "discountedPrice":766
  },
  {
    "id":39,
    "title":"Women Sweaters Wool",
    "price":600,
    "quantity":2,
    "total":1200,
    "discountPercentage":17.2,
    "discountedPrice":994
  }
]

Example One

For our first example, lets assume that we want to extract the title and discountedPrice of all the products.

If we were to tackle this task using traditional Python methods, we'd find ourselves lost in a maze of nested loops and conditional statements.

required_info = []
for product in products:
  temp_dict = {
      "title": product["title"],
      "discountedPrice": product["discountedPrice"],
  }
  required_info.append(temp_dict)

However, armed with the simplicity and power of JMESPath, we can achieve the same result with elegance and efficiency.

import jmespath

required_info = jmespath.search("[].{title: title, discountedPrice: discountedPrice}", products)

Example Two

For our second example, lets assume that we want to extract the title and discountedPrice of all the products where the discountedPrice is greater than 100.

To achieve this using python and nested loops, we would have to write something like this:

required_info = []
for product in products:
  if product["discountedPrice"] > 100:
    temp_dict = {
      "title": product["title"],
      "discountedPrice": product["discountedPrice"],
    }
    required_info.append(temp_dict)

The same can be achieved using JMESPath in just one line of code:

import jmespath

required_info = jmespath.search("[?discountedPrice > `100`].{title: title, discountedPrice: discountedPrice}", products)

Example Three

For our third and final example, lets assume that we want to extract the title and discountedPrice and total of all the products were the discountPrice is greater than 50 and and total is less than 1000.

The python code to achieve this would look something like this:

required_info = []
for product in products:
  if product["discountedPrice"] > 50 and product["total"] < 1000:
    temp_dict = {
      "title": product["title"],
      "total": product["total"],
      "discountedPrice": product["discountedPrice"],
    }
    required_info.append(temp_dict)

The JMESPath query to achieve the same would look like this:

import jmespath

required_info = jmespath.search("[?discountedPrice > `50` && total < `1000`].{title: title, total: total, discountedPrice: discountedPrice}", products)

Writing your own JMESPath queries

It took me a while to get used to writing JMESPath queries. I hope the below explanation along with the examples shown above will help you in your journey.

jmespath.search([?expression].{key: value, key: value}, data)

• expression -> is the condition that you want to apply to the data. The expression can be a simple comparison or a complex expression.
• key:value -> is the key and value that you want to extract from the data.
• data -> is the JSON data that you want query.

Resources

Exploring the world of JMESPath queries? Look no further than the JMESPath Playground! This invaluable resource serves as both a learning platform and a testing ground for your queries. Dive in and sharpen your JMESPath skills with ease.

Conclusion

JMESPath stands as a versatile powerhouse in the realm of JSON manipulation. With its capabilities to extract, transform, filter, and even merge multiple JSON datasets, it empowers developers to wield JSON data with precision and ease. I trust this post has provided you with a solid introduction to the endless possibilities offered by JMESPath.