How to Use Python Struct Functions

Introduction

Python structs are a practical tool for managing and storing raw byte data. The struct module enables the creation of compact binary data types used in the C programming language.

Structs efficiently store and work with fixed-size data in memory. This tool allows transferring data between systems and languages which use binary data representations.

This article shows how to use the Python struct module and the available functions.

Prerequisites

• Python 3 installed (learn how to check the Python version installed on your system).
• A Python IDE or code editor to write the code.
• Access to an IDE or terminal to run the code.

Python Struct Module Overview

The Python struct module has two functionalities:

• Data exchange with external sources, such as network transfers.
• Data transfer between Python and C applications.

Structs require constructing a format string for packing and unpacking data. The character combinations describe the data type being packed and unpacked.

The table below describes some commonly used characters object types in Python and C, with their corresponding sizes in bytes.

For example, the format string for three numbers (such as 100, 200, 300) is iii or 3i, while the size is 12 bytes (4 for each integer).

Apart from the string format, some struct functions require adding a buffer. The buffer is commonly a bytes() or bytearray() object.

Python Struct Functions

The Python struct module offers several functions to pack and unpack structs and an error catcher for exceptions. The examples below demonstrate how to use each functionality from the struct module through code.

All examples require importing the module:

import struct

The struct library comes with Python by default.

struct.pack()

The struct.pack() function packs values according to the string format. The syntax is:

struct.pack("format", value_1, value_2, etc)

The format string must exactly match the value format. For example:

import struct

packed_struct = struct.pack("3i", 1, 2, 3)
print(packed_struct)

The format string (3i) corresponds to the provided values (three integers). As a result, the code prints the packed struct, where each value is 4 bytes.

b"abc"

struct.unpack()

The struct.unpack() function unpacks a packed struct to the original format. The syntax is:

struct.unpack("format", packed_struct)

The format is identical to the one used to pack a struct. For example:

import struct

packed_struct = struct.pack("3i", 1, 2, 3)
print("Packed struct:")
print(packed_struct)

unpacked_struct = struct.unpack("3i", packed_struct)
print("Unpacked struct:")
print(unpacked_struct)

The code packs a struct with three integers and unpacks it using the same string format. The unpacked struct is in a list.

struct.calcsize()

The struct.calcsize() function helps calculate the string format byte size. To see the size, pass the string format as an input value. For example:

import struct

print(struct.calcsize("3i"))

The code shows the size for 3i as 12. Each integer is four bytes, so 12 is the total size for three integers.

struct.pack_into()

The struct.pack_into() function packs a struct into a buffer. The syntax is:

struct.pack_into("format", buffer, offset, value_1, value_2, etc)

The buffer is either a bytes() object for a single value or a bytesarray() for multiple values. The minimum buffer size is the total bytes of the values. If the buffer size exceeds the data size, the offset value allows shifting the starting data position. All empty bytes are zero-filled.

For example:

import struct

buffer = bytearray(13)
print("Empty buffer:")
print(buffer)

struct.pack_into("3i", buffer, 1, 2, 3, 4)
print("Populated buffer")
print(buffer)

The code creates an empty bytesarray() object with 13 bytes. Initially, the array contains zeros to serve as placeholders.

The struct_packinto() function packs data into the buffer with offset one. The object fits three integer values (four bytes each), with the first value being zero to account for the shift.

The output prints the empty buffer, followed by the populated buffer.

struct.unpack_from()

The struct.unpack_from() function unpacks a populated buffer. The syntax is:

stuct.unpack_from("format", buffer, offset)

Omit the offset parameter if the original offset is zero.

The following code shows how to pack a struct into a buffer and unpack the values:

import struct

buffer = bytearray(13)
print("Empty buffer:")
print(buffer)

struct.pack_into("3i", buffer, 1, 2, 3, 4)
print("Populated buffer")
print(buffer)

unpacked_buffer = struct.unpack_from("3i", buffer, 1)
print("Unpacked buffer")
print(unpacked_buffer)

The result unpacks the original data from the struct in a list.

struct.iter_unpack()

The struct.iter_unpack() function unpacks a struct into an iterative format instead of a ready list. The syntax is:

struct.iter_unpack("format", buffer, offset)

Use a loop to iterate through the values in the unpacked struct. For example:

import struct

buffer = bytearray(12)
print("Empty buffer:")
print(buffer)

struct.pack_into("3i", buffer, 0, 2, 3, 4)
print("Populated buffer")
print(buffer)

iterative_unpack = struct.iter_unpack("3i", buffer)

for value in iterative_unpack:
    print(value)

The for loop goes through all the values in the iterative struct and prints them to the output.

Exception struct.error

The struct library contains a function that stores the error message. Use the struct.error exception in a try except statement to parse the message.

For example:

import struct

buffer = bytearray(2)

try:
    pack_buffer = struct.pack_into("3i", buffer, 0, 2, 3, 4)
except struct.error as e:
    print("I've caught an error! Here it is: ", e)

The code creates a two-byte buffer, which is not enough to store three integers. As a result, the exception handles the error and prints the message as output.

Conclusion

After reading this guide and working through the examples, you know how to pack and unpack structs. Structs help pack and unpack data in a consistent format.