srctools.binformat

The binformat module binformat contains functionality for handling binary formats, esentially expanding on struct’s functionality.

Constants

srctools.binformat.ST_VEC: struct.Struct = Struct('fff')

A struct.Struct with three floats, for unpacking Vec, Angle, etc.

srctools.binformat.SIZES: dict[str, struct.Struct]

A dict mapping each fixed-size number format character (i, L, h, etc) to the size of the data.

srctools.binformat.SIZE_CHAR = 1
srctools.binformat.SIZE_SHORT = 2
srctools.binformat.SIZE_INT = 4
srctools.binformat.SIZE_LONG = 8
srctools.binformat.SIZE_FLOAT = 4
srctools.binformat.SIZE_DOUBLE = 8

The size of each of these standard numeric types.

Structure tools

class srctools.binformat.DeferredWrites(file: IO[bytes])

Several formats require offsets or similar data to be written referring to data later in the file.

Doing this in one pass would be quite complicated, but this class assists in writing such files. Initially null bytes are written in the slots, then the data is filled in at the end.

To use this class, initialise it with the open and seekable file. Call defer() when reaching the relevant parts of the file, passing a format string for the structure and a hashable key used to identify it later. Once the value has been determined, call set_data() to store the data. When the file is written out, call write() which will seek() back and fill in the values.

defer(
key: Hashable,
fmt: str | Struct,
write: bool = False,
) → None

Mark that data of the specified format is going to be written here.

Parameters:

  • key – Any hashable object, used to identify this location later.

  • fmt – The structure of the data, either as an existing struct.Struct instance, or a string in the same format.

  • write – If true, write null bytes to occupy the space in the file. Set to false if you are doing this yourself.

set_data(
key: Hashable,
*data: int | str | bytes | float,
) → None

Specify the data for the given key.

Parameters:

  • key – Any hashable object, used to identify the location and format.

  • data – The values passed to struct.pack() to build the data.

pos_of(key: Hashable) → int

Return the previously marked offset with the given key.

Parameters:

key – Any hashable object, used to match this call to the earlier defer() call.

write() → None

Write out all the data. All values should have been set.

srctools.binformat.struct_read(
fmt: Struct | str,
file: IO[bytes],
) → Tuple[Any, ...]

Read a structure from the file, automatically computing the required number of bytes.

srctools.binformat.str_readvec(
file: IO[bytes],
) → Vec

Shortcut to read a 3-float vector from a file.

srctools.binformat.read_array(
fmt: str | Struct,
data: bytes,
) → List[int]

Read a buffer containing a stream of integers.

The format string should be one of the integer format characters, optionally prefixed by an

endianness indicator. As many integers as possible will then be read from the data.

srctools.binformat.write_array(
fmt: str | Struct,
data: Collection[int],
) → bytes

Build a packed array of integers.

The format string should be one of the integer format characters, optionally prefixed by an endianness indicator. The integers in the data will then be packed into a bytes buffer and returned.

srctools.binformat.read_nullstr(
file: IO[bytes],
pos: int | None = None,
encoding: str = 'ascii',
) → str

Read a null-terminated string from the file.

If the position is 0, this will instead immediately return an empty string. If set to any other value this will first seek to the location.

srctools.binformat.read_nullstr_array(
file: IO[bytes],
count: int,
encoding: str = 'ascii',
) → List[str]

Read the specified number of consecutive null-terminated strings from a file.

If the count is zero, no reading will be performed at all.

srctools.binformat.read_offset_array(
file: IO[bytes],
count: int,
encoding: str = 'ascii',
) → List[str]

Read an array of offsets to null-terminated strings from the file.

This first reads the specified number of signed integers, then seeks to those locations and reads a null-terminated string from each.

Checksumming

srctools.binformat.checksum(data: bytes, prior: int = 0) → int

Compute the VPK checksum for a file (CRC32).

Pass a previous computation to allow continuing a previous checksum.

srctools.binformat.EMPTY_CHECKSUM = 0

The checksum value of an empty bytes buffer (b"").

LZMA (de)compression

srctools.binformat.decompress_lzma(data: bytes) → bytes

Decompress LZMA-encoded data, using the settings Source uses in BSP lumps.

srctools.binformat.compress_lzma(data: bytes) → bytes

Compress data using the LZMA algorithm and the settings Source uses in BSP lumps.