srctools.dmx

Handles DataModel eXchange trees, in both binary and text (keyvalues2) format.

As an extension, optionally all strings may become full UTF-8, marked by a new set of ‘unicode_XXX’ encoding formats.

Special ‘stub’ elements are possible, which represent elements not present in the file. These are represented by StubElement instances. Additionally, NULL elements are possible.

To parse a DMX file, it must be opened in binary mode (the kind will be detected automatically):

with open("path/to/file.dmx", "rb") as f:
    dmx, format_name, format_version = Element.parse(f)

The format name/version are stored in the header, allowing indicating the kind of data stored in the file.

class srctools.dmx.ValueType(
value,
names=None,
*,
module=None,
qualname=None,
type=None,
start=1,
boundary=None,
)

Bases: Enum

The type of value an element attribute has.

ELEMENT = 'element'

A child Element. This may additionally be NULL or a StubElement, representing data stored elsewhere.

INTEGER = 'int'

Alias: INT

A 32-bit signed int value. In text format any precision value can be parsed/exported.

FLOAT = 'float'

A single-precision float point value. In text format the full Python double-precision value can be parsed/exported.

BOOL = 'bool'

A bool true/false value.

STRING = 'string'

Alias: STR

A general text string, though it should probably be kept to ASCII/CP-1252 to ensure Valve’s parsers produce the correct result. See the unicode option in Element.parse(), Element.export_kv2() and export_binary().

BINARY = 'binary'

Aliases: BIN, VOID

A block of raw bytes data. Any buffer object can be converted to this.

TIME = 'time'

Time since the begining of a level, in seconds. This is represented by the Time type.

COLOR = 'color'

Alias: COLOUR

An RGBA 8-bit colour. This can be converted from a 3 or 4-tuple, or a space separated string. In either case alpha defaults to 255 if not specified.

VEC2 = 'vector2'

A generic XY vector, represented by the Vec2 namedtuple. This can be converted from an iterable of 4 floats.

VEC3 = 'vector3'

A generic XYZ vector, represented by srctools.math.FrozenVec classs. This can be converted from an iterable of 3 floats.

VEC4 = 'vector4'

A generic XYZW vector, represented by the Vec4 namedtuple. This can be converted from an iterable of 4 floats.

ANGLE = 'qangle'

An Euler angle, represented by srctools.math.FrozenAngle. This can be converted from an iterable of 3 floats, Angle, FrozenMatrix or Matrix.

QUATERNION = 'quaternion'

A rotational quaternion, represented by the Quaternion namedtuple. This can be converted from a 4-element iterable.

MATRIX = 'vmatrix'

A translation FrozenMatrix. This can be converted from a Angle, FrozenAngle, Matrix or Matrix.

srctools.dmx.NULL = <Null Element>

A special constant Element with an all-zero UUID, representing “null” elements in binary DMX files. This always has no members.

srctools.dmx.STUB = _StubType.STUB
Deprecated:

Use StubElement.stub().

srctools.dmx.Vec3
Deprecated:

Use srctools.math.FrozenVec.

srctools.dmx.AngleTup
Deprecated:

Use srctools.math.FrozenAngle.

class srctools.dmx.Vec2(x: float, y: float)

A 2-dimensional vector.

x: float

Alias for field number 0

y: float

Alias for field number 1

class srctools.dmx.Vec4(x: float, y: float, z: float, w: float)

A 4-dimensional vector.

x: float

Alias for field number 0

y: float

Alias for field number 1

z: float

Alias for field number 2

w: float

Alias for field number 3

class srctools.dmx.Quaternion(x: float, y: float, z: float, w: float)

A quaternion used to represent rotations.

x: float

Alias for field number 0

y: float

Alias for field number 1

z: float

Alias for field number 2

w: float

Alias for field number 3

class srctools.dmx.Color(
r: int | float,
g: int | float,
b: int | float,
a: int | float = 255,
)

An RGB color.

class srctools.dmx.Time(value: float = 0.0)

A relative timestamp.

class srctools.dmx.Attribute(
name: str,
val_type: ValueType,
value: ValueT | List[ValueT],
)

A single attribute of an element.

Attributes store either a single scalar value, or an array of multiple values of the same type. Creation is accomplished mainly via the many classmethods, one for each value type.

To access the value, read/write one of the val_* properties, which will if possible convert the value to the desired type. For arrays either use iter_*() or attr[ind].val_* to fetch by index.

property type: ValueType

Return the current type of the attribute.

property is_array: bool

Check if this is an array, or a singular value.

classmethod array(
name: str,
val_type: ValueType,
values: Iterable[Any] = (),
) → Attribute

Create an attribute with an empty array of a specified type.

classmethod int(
name: str,
value: int | List[int],
) → Attribute[int]

Create an attribute with an integer value.

classmethod float(
name: str,
value: float | List[float],
) → Attribute[float]

Create an attribute with a float value.

classmethod time(
name: str,
value: Time | float,
) → Attribute[Time]

Create an attribute with a ‘time’ value.

This is effectively a float, and only available in binary v3+.

classmethod bool(
name: str,
value: bool | List[bool],
) → Attribute[bool]

Create an attribute with a boolean value.

classmethod string(
name: str,
value: str | List[str],
) → Attribute[str]

Create an attribute with a string value.

classmethod binary(
name: str,
value: bytes | List[bytes],
) → Attribute[bytes]

Create an attribute with binary data.

classmethod vec2(
name: str,
x: float | Iterable[float] = 0.0,
y: float = 0.0,
) → Attribute[Vec2]

Create an attribute with a 2D vector.

classmethod vec3(
name: str,
x: float | Iterable[float] = 0.0,
y: float = 0.0,
z: float = 0.0,
) → Attribute[FrozenVec]

Create an attribute with a 3D vector.

classmethod vec4(
name: str,
x: float | Iterable[float] = 0.0,
y: float = 0.0,
z: float = 0.0,
w: float = 0.0,
) → Attribute[Vec4]

Create an attribute with a 4D vector.

classmethod color(
name: str,
r: float | int | Iterable[float | int] = 0,
g: float | int = 0,
b: float | int = 0,
a: float | int = 255,
) → Attribute[Color]

Create an attribute with a color.

classmethod angle(
name: str,
pitch: float | Iterable[float] = 0.0,
yaw: float = 0.0,
roll: float = 0.0,
) → Attribute[FrozenAngle]

Create an attribute with an Euler angle.

classmethod quaternion(
name: str,
x: float | Iterable[float] = 0.0,
y: float = 0.0,
z: float = 0.0,
w: float = 0.0,
) → Attribute[Quaternion]

Create an attribute with a quaternion rotation.

iter_int() → Iterator[int]

Iterate over the attribute, treating it as an array of integer values.

iter_str() → Iterator[str]

Iterate over the attribute, treating it as an array of string values.

iter_string() → Iterator[str]

Iterate over the attribute, treating it as an array of string values.

iter_bin() → Iterator[bytes]

Iterate over the attribute, treating it as an array of binary values.

iter_binary() → Iterator[bytes]

Iterate over the attribute, treating it as an array of binary values.

iter_bytes() → Iterator[bytes]

Iterate over the attribute, treating it as an array of binary values.

iter_float() → Iterator[float]

Iterate over the attribute, treating it as an array of float values.

iter_time() → Iterator[Time]

Iterate over the attribute, treating it as an array of time values.

iter_bool() → Iterator[bool]

Iterate over the attribute, treating it as an array of bool values.

iter_colour() → Iterator[Color]

Iterate over the attribute, treating it as an array of color values.

iter_color() → Iterator[Color]

Iterate over the attribute, treating it as an array of color values.

iter_vec2() → Iterator[Vec2]

Iterate over the attribute, treating it as an array of vec2 values.

iter_vec3() → Iterator[FrozenVec]

Iterate over the attribute, treating it as an array of vec3 values.

iter_vec4() → Iterator[Vec4]

Iterate over the attribute, treating it as an array of vec4 values.

iter_quat() → Iterator[Quaternion]

Iterate over the attribute, treating it as an array of quaternion values.

iter_quaternion() → Iterator[Quaternion]

Iterate over the attribute, treating it as an array of quaternion values.

iter_ang() → Iterator[FrozenAngle]

Iterate over the attribute, treating it as an array of angle values.

iter_angle() → Iterator[FrozenAngle]

Iterate over the attribute, treating it as an array of angle values.

iter_mat() → Iterator[FrozenMatrix]

Iterate over the attribute, treating it as an array of matrix values.

iter_matrix() → Iterator[FrozenMatrix]

Iterate over the attribute, treating it as an array of matrix values.

iter_compound() → Iterator[Element]

Iterate over the attribute, treating it as an array of element values.

iter_elem() → Iterator[Element]

Iterate over the attribute, treating it as an array of element values.

append(
value: int | float | bool | str | bytes | Color | Time | Vec2 | FrozenVec | Vec4 | FrozenAngle | Quaternion | FrozenMatrix | Element | Vec | Matrix | Angle,
) → None

Append an item to the array.

If not already an array, it is converted to one holding the existing value.

extend(
values: Iterable[int | float | bool | str | bytes | Color | Time | Vec2 | FrozenVec | Vec4 | FrozenAngle | Quaternion | FrozenMatrix | Element | Vec | Matrix | Angle],
) → None

Append multiple values to the array.

If not already an array, it is converted to one holding the existing value.

clear_array() → None

Remove all items in this, if it is an array.

copy() → Attribute[ValueT]

Duplicate this attribute shallowly, retaining references if this is an Element type.

class srctools.dmx.Element(
name: str,
type: str,
uuid: UUID | None = None,
)

An element in a DMX tree.

This is a mapping over Attribute objects, representing each key-value pair in the element. Other than the three special attributes name, type and uuid, keys are accessed by regular mapping methods, producing Attribute objects. As a convenience, values or lists of values can be assigned directly to implicitly create the Attribute.

type: str

In Valve’s formats, this is the name of the C++ class that the element should deserialise into, like DMElement for example. It can be used for any purpose though. In binary files, elements with identical types names are deduplicated in the file.

uuid: UUID

This used in the serialised file to allow elements to be appear multiple times in the tree, including recursively. This does not normally need to be set manually, since a random one will be computed for each new element automatically.

property is_stub: bool

Check if this is a ‘stub’ element, found in binary DMXes.

property is_null: bool

Check if this is a NULL element, found in binary DMXes.

property name: str

The name can be any string, but usually is some sort of item ID.

For elements which are children of another element, the name usually matches the key. This is actually just a key with the name name. If it is missing, "" is returned.

classmethod parse(
file: IO[bytes],
unicode: bool = False,
) → Tuple[Element, str, int]

Parse a DMX file encoded in binary or KV2 (text).

The return value is the tree, format name and version. The format name and version are stored in the header, and allow validating the kind of dat stored in the file. If unicode is set to True, strings will always be treated as UTF8, even if this module’s encoding name is not provided. Otherwise, only ASCII characters are allowed for safety.

classmethod parse_bin(
file: IO[bytes],
version: int,
unicode: bool = False,
) → Element

Parse the core binary data in a DMX file.

The <!-- --> format comment line should have already be read. If unicode is set to True, strings will be treated as UTF8 instead of safe ASCII.

classmethod parse_kv2(
file: IO[str],
version: int,
unicode: bool = False,
) → Element

Parse a DMX file encoded in KeyValues2.

The <!-- --> format comment line should have already been read.

export_binary(
file: IO[bytes],
version: int = 5,
fmt_name: str = 'dmx',
fmt_ver: int = 1,
unicode: Literal['ascii', 'format', 'silent'] = 'ascii',
) → None

Write out a DMX tree, using the binary format.

Parameters:

  • file – A writable binary file to save to.

  • version – Must be a number from 0-5.

  • fmt_name – This can be any string, to indicate what contents the file contains.

  • fmt_ver – This can be a format number, to indicate changes in the format.

  • unicode – This controls how Unicode characters are handled.

For the unicode parameter, the following values are permitted:

  • "ascii" (the default) raises an error if any value is non-ASCII. This ensures no encoding issues occur when read by the game.

  • "format" changes the encoding format to "unicode_binary", allowing the file to be rejected if the game tries to read it and this module’s parser to automatically switch to Unicode.

  • "silent" outputs UTF8 without any marker, meaning it could be parsed incorrectly by the game or other utilties. This must be parsed with unicode=True to succeed.

export_kv2(
file: IO[bytes],
fmt_name: str = 'dmx',
fmt_ver: int = 1,
*,
flat: bool = False,
unicode: Literal['ascii', 'format', 'silent'] = 'ascii',
cull_uuid: bool = False,
) → None

Write out a DMX tree, using the text-based KeyValues2 format.

The format name and version can be anything, to indicate which application should read the file.

  • If flat is enabled, elements will all be placed at the toplevel, so they don’t nest inside each other.

  • If cull_uuid is enabled, UUIDs are only written for self-referential elements. When parsed by this or Valve’s parser, new ones will simply be generated.

  • unicode controls whether Unicode characters are permitted:

    • ascii (the default) raises an error if any value is non-ASCII. This ensures no encoding issues occur when read by the game.

    • format changes the encoding format to unicode_keyvalues2, allowing the file to be rejected if the game tries to read it and this module’s parser to automatically switch to Unicode.

    • silent outputs UTF8 without any marker, meaning it could be parsed incorrectly by the game or other utilties. This must be parsed with unicode=True to succeed.

classmethod from_kv1(
props: Keyvalues,
) → Element

Convert a KeyValues 1 property tree into DMX format, allowing nesting inside a DMX file.

All blocks have a type of DmElement, with children stored in the subkeys array. Leaf properties are stored as regular attributes. The following attributes are prepended with an underscore when converting as they are reserved: name and subkeys.

If multiple leaf properties with the same name or the element has a mix of blocks and leafs all elements will be put in the subkeys array. Leafs will use DmElementLeaf elements with values in the value key.

to_kv1() → Keyvalues

Convert an element tree containing a KeyValues 1 tree back into a Property.

These must satisfy the format from_kv1() produces - all elements have the type DmElement, all attributes are strings, except for the subkeys attribute which is an element array.

keys() → KeysView[str]

Return a view of the valid (casefolded) keys for this element.

values() → ValuesView[Attribute]

Return a view of the attributes for this element.

clear() → None

Remove all attributes from the element.

pop(
name: str,
default: ~srctools.dmx.Attribute | int | float | bool | str | bytes | ~srctools.dmx.Color | ~srctools.dmx.Time | ~srctools.dmx.Vec2 | ~srctools.math.FrozenVec | ~srctools.dmx.Vec4 | ~srctools.math.FrozenAngle | ~srctools.dmx.Quaternion | ~srctools.math.FrozenMatrix | ~srctools.dmx.Element | ~srctools.math.Vec | ~srctools.math.Matrix | ~srctools.math.Angle | ~typing.Sequence[int | float | bool | str | bytes | ~srctools.dmx.Color | ~srctools.dmx.Time | ~srctools.dmx.Vec2 | ~srctools.math.FrozenVec | ~srctools.dmx.Vec4 | ~srctools.math.FrozenAngle | ~srctools.dmx.Quaternion | ~srctools.math.FrozenMatrix | ~srctools.dmx.Element | ~srctools.math.Vec | ~srctools.math.Matrix | ~srctools.math.Angle] = <object object>,
) → Attribute

Remove the specified attribute and return it.

If not found, an attribute is created from the default if specified, or KeyError is raised otherwise.

popitem() → Tuple[str, Attribute]

Remove and return a (name, attr) pair as a 2-tuple, or raise KeyError.

setdefault(
name: str,
default: Attribute | int | float | bool | str | bytes | Color | Time | Vec2 | FrozenVec | Vec4 | FrozenAngle | Quaternion | FrozenMatrix | Element | Vec | Matrix | Angle | Sequence[int | float | bool | str | bytes | Color | Time | Vec2 | FrozenVec | Vec4 | FrozenAngle | Quaternion | FrozenMatrix | Element | Vec | Matrix | Angle],
) → Attribute

Return the specified attribute name.

If it does not exist, set it using the default and return that.

class srctools.dmx.StubElement(
typ: _StubType,
uuid: UUID | None,
)

In binary DMXes, it is possible to have stub elements representing values excluded from the file.

This always has no members - attempting to add members will fail silently.

classmethod stub(
uuid: UUID | None = None,
) → StubElement

Create a stubbed element reference with the specified UUID.