Image by johnstocker on Freepik

A while ago, I wrote a tutorial on writing efficient Python code. In it, I talked about using context managers and the with statement to manage resources efficiently.

I used a simple file handling example to show how files are automatically closed when the execution exits the with block—even if there is an exception.

While file handling is a good first example, it can quickly get boring. That is why I'd like to go over other interesting uses of context managers—beyond file handling—in this tutorial. We’ll focus on handling database connections, managing subprocesses, and high-precision floating point arithmetic.

What Are Context Managers in Python?

Context managers in Python allow you to write cleaner code when working with resources. They provide a concise syntax to set up and tear down resources through:

• An enter logic that gets called when the execution enters the context and
• An exit logic the gets called when the execution exits the context 

The simplest example of this is in file handling. Here we use the open() function in the with statement to get a file handler:

with open('filename.txt', 'w') as file:
    file.write('Something random')

This acquires the resource—the file object—that is used (we write to the file) within the code block. The file is closed once the execution exits the context; so there are no resource leaks.

You can write the generic version of this like so:

with some_context() as ctx:
    # do something useful on the resource!

# resource cleanup is automatic

Now let’s proceed to the specific examples.

1. Handling Database Connections 

When you're building Python applications, it's quite common to connect to databases and query the tables they contain. And the workflow to do this will look like so:

• Install the database connector to work with the database (such as psycopg2 for Postgres and the mysql-connector-python for MySQL databases).
• Parse the config file to retrieve the connection parameters. 
• Use the connect() function to establish connection to the database.

Connecting to the db | Image by Author

Once you’ve connected to the database, you can create a database to query the database. Run queries and fetch the results of the query using the run and fetch cursor methods.

Querying the db | Image by Author

In doing so, you create the following resources: a database connection and a database cursor. Now let’s code a simple generic example to see how we can use the connection and the cursor objects as context managers.

Parsing TOML Files in Python

Consider a sample TOML file, say db_config.toml, containing the required info to connect to the database:

# db_config.toml

[database]
host = "localhost"
port = 5432
database_name = "your_database_name"
user = "your_username"
password = "your_password"

Note: You need Python 3.11 or a later version to use tomllib.

Python has a built-in tomllib module (introduced in Python 3.11) that lets you parse TOML files. So you can open the db_config.toml file and parse its contents like so:

import tomllib

with open('db_config.toml','rb') as file:
	credentials = tomllib.load(file)['database']

Notice that we tap into the ‘database’ section of the db_config.toml file. The load() function returns a Python dictionary. You can verify this by printing out the contents of credentials:

print(credentials)

Output >>>
{'host': 'localhost', 'port': 5432, 'database_name': 'your_database_name', 'user': 'your_username', 'password': 'your_password'}

Connecting to the Database

Say you want to connect to a Postgres database. You can install the psycopg2 connector using pip:

pip install psycopg2

You can use both the connection and the cursor objects in with statements as shown:

import psycopg2

# Connect to the database
with psycopg2.connect(**credentials) as conn:
	# Inside this context, the connection is open and managed

	with conn.cursor() as cur:
    	# Inside this context, the cursor is open and managed

    	    cur.execute('SELECT * FROM my_table')
    	    result = cur.fetchall()
            print(result)

In this code:

• We use the with statement to create a context for managing the database connection. 
• Inside this context, we create another context to manage the database cursor. The cursor is automatically closed when exiting this inner context.
• Because the connection is also closed when exiting the outer context, this construct ensures that both the connection and cursor are properly managed—reducing the chance of resource leaks.

You can use a similar construct when working with SQLite and MySQL databases too.

2. Managing Python Subprocesses

Python’s subprocess module provides functionality to run external commands inside a Python script. The subprocess.Popen() constructor creates a new subprocess. Which you can use in a with statement like so:

import subprocess

# Run an external command and capture its output
with subprocess.Popen(['ls', '-l'], stdout=subprocess.PIPE, text=True) as process:
	output, _ = process.communicate()
	print(output)

Here, we run the Bash command ls -l command to long list the files in the current directory:

Output >>>

total 4
-rw-rw-r-- 1 balapriya balapriya   0 Jan  5 18:31 db_info.toml
-rw-rw-r-- 1 balapriya balapriya 267 Jan  5 18:32 main.py

The resources associated with the subprocess are freed once the execution exits the context of the with statement. 

3. High-Precision Floating-Point Arithmetic

The built-in float data type in Python is not suitable for high-precision floating-point arithmetic. But you do need high precision when working with financial data, sensor readings, and the like. For such applications, you can use the decimal module instead. 

The localcontext() function returns a context manager. So you can use the localcontext() function in the with statement, and set the precision for the current context using as shown:

from decimal import Decimal, localcontext

with localcontext() as cur_context:
    cur_context.prec = 40
    a = Decimal(2)
    b = Decimal(3)
    print(a/b)

Here’s the output:

Output >>>
0.6666666666666666666666666666666666666667

Here, the precision is set to 40 decimal places—but only within this with block. When the execution exits the current context, the precision is restored to the default precision (of 28 decimal places).

Wrapping Up

In this tutorial, we learned how context managers can be used for handling database connections, managing subprocesses and contexts in high-precision floating-point arithmetic.

In the next tutorial, we’ll see how we can create custom context managers in Python. Until then, happy coding!
 
 

Bala Priya C is a developer and technical writer from India. She likes working at the intersection of math, programming, data science, and content creation. Her areas of interest and expertise include DevOps, data science, and natural language processing. She enjoys reading, writing, coding, and coffee! Currently, she's working on learning and sharing her knowledge with the developer community by authoring tutorials, how-to guides, opinion pieces, and more. Bala also creates engaging resource overviews and coding tutorials.