Hacking Terraform

In a previous blog post, I talked about how Terraform's native capabilities don't fully cover comprehensive IP address management, which can make network configurations a bit tricky.

In this post, I’m going to dive into a practical approach for handling IP addresses in Terraform. I'll show you how to leverage an external data source and use a Python script to process IP address operations, then integrate the results back into Terraform.

Introduction to External Data Source

In Terraform, a data source allows you to retrieve information from external systems or services, which you can then use in your configurations. Unlike resources, which are used to manage the lifecycle of infrastructure components, data sources are read-only. They provide a way to fetch data that you might need when setting up or configuring your infrastructure. This is especially useful when you want to incorporate existing information without directly managing the components within your Terraform scripts.

A simple data source in Terraform looks like this:

data "external" "ip" {
  id = "ip"
}

A lot of providers provide external data sources to interact with their systems and get configuration state. A data source in Terraform can range from a simple key-value pair to a complex data structure.

In our case, I’ll be using a Python script.

Using Python in External Data Sources

Let’s consider a scenario where we have a list of IP addresses that includes both public and private IP addresses. Our goal is to split these IP addresses into two separate lists based on their type, so we can process them further in Terraform.

We'll create a variables.tf file to define the input variables:

variable "ip_addresses" {
  description = "List of private and public IP addresses"
  type        = list(string)
  default = [
    "192.168.1.10",   # Private IP address
    "10.0.0.25",      # Private IP address
    "172.16.5.100",   # Private IP address
    "192.168.100.50", # Private IP address
    "8.8.8.8",        # Public IP address (Google DNS)
    "8.8.4.4",        # Public IP address (Google DNS)
    "1.1.1.1",        # Public IP address (Cloudflare DNS)
    "9.9.9.9",        # Public IP address (Quad9 DNS)
    "198.51.100.1",   # Public IP address (Example)
    "93.184.216.34",  # Public IP address (Example)
  ]
}

Next, we'll create an external data source in locals.tf in Terraform that calls a Python script to process the IP addresses. We will pass in the list of IP addresses as input to the Python script:

data "external" "ip" {
  program = ["python", "ip.py"]
  query = {
    ip_addresses = jsonencode(var.ip_addresses)
  }
}

Next, we'll create a Python script called ip.py that processes the IP addresses and returns the results back to Terraform. Refer to the inline comments in the Python script for a detailed explanation of each step:

import json
import ipaddress
import sys
import logging

logging.basicConfig(
    filename="ip_classification.log",
    level=logging.INFO,
    format="%(asctime)s - %(message)s",
)


def main():
    # Read the JSON input from Terraform
    input_data = json.load(sys.stdin)

    ip_addresses = json.loads(input_data["ip_addresses"])

    # Initialize empty lists to store private and public IP addresses
    private_ips = []
    public_ips = []

    # Loop through the list of IP addresses
    for ip in ip_addresses:
        # Convert the IP address to an ipaddress object
        ip_address = ipaddress.ip_address(ip)

        # Check if the IP address is public or private
        if ip_address.is_private:
            logging.info(f"Private IP address: {ip_address}")
            private_ips.append(str(ip_address))
        else:
            logging.info(f"Public IP address: {ip_address}")
            public_ips.append(str(ip_address))

    result = {
        "private_ips": json.dumps(private_ips),
        "public_ips": json.dumps(public_ips),
    }

    # Return the result back to Terraform
    json.dump(result, sys.stdout)


# Execute the main function to process the IP addresses
if __name__ == "__main__":
    main()

Next we'll create an output in Terraform to display the results.

output "public_ip_addresses" {
  description = "Public IP addresses"
  value       = data.external.ip.result.public_ips
}

output "private_ip_addresses" {
  description = "Private IP addresses"
  value       = data.external.ip.result.private_ips
}

Now lets initialise and generate a plan by executing terraform init and terraform plan commands.

As you can see, we have processed the list of IP addresses and converted it into two lists one containing Public IP addresses and another containing Private IP addresses.

Conclusion

In this blog post, I explored how to use external data sources in Terraform to process IP addresses with a Python script. This approach allows you to interact with external systems and services, fetching data that can be seamlessly integrated into your Terraform configurations. I hope you found this guide helpful!

You can find the code featured in this blog post in the repository linked below.