srctools.dmx
Handles DataModel eXchange trees, in both binary and text (keyvalues2) format.
As an extension, optionally all strings may become full UTF-8, marked by a new set of ‘unicode_XXX’ encoding formats.
Special ‘stub’ elements are possible, which represent elements not present in the file. These are represented by StubElement instances. Additionally, NULL elements are possible.
To parse a DMX file, it must be opened in binary mode (the kind will be detected automatically):
with open("path/to/file.dmx", "rb") as f:
dmx, format_name, format_version = Element.parse(f)
The format name/version are stored in the header, allowing indicating the kind of data stored in the file.
-
class
srctools.dmx.
ValueType
(value) Bases:
Enum
The type of value an element attribute has.
-
ELEMENT
= 'element' A child
Element
. This may additionally beNULL
or aStubElement
, representing data stored elsewhere.
-
INTEGER
= 'int' Alias:
INT
A 32-bit signed
int
value. In text format any value can be parsed/exported.
-
FLOAT
= 'float' A single-precision
float
point value. In text format the full Python double-precision value can be parsed/exported.
-
STRING
= 'string' Alias:
STR
A general text string, though it should probably be kept to ASCII/CP-1252 to ensure Valve’s parsers produce the correct result. See the
unicode
option inElement.parse()
,Element.export_kv2()
andexport_bin()
.
-
BINARY
= 'binary' Aliases:
BIN
,VOID
A block of raw
bytes
data. Any buffer object can be converted to this.
-
TIME
= 'time' Time since the begining of a level, in seconds. This is represented by the
Time
typing type, but at runtime is simply afloat
.
-
COLOR
= 'color' Alias:
COLOUR
An RGBA 8-bit colour. This can be converted from a 3 or 4-tuple, or a space separated string. In either case alpha defaults to
255
if not specified.
-
VEC2
= 'vector2' A generic XY vector, represented by the
Vec2
namedtuple. This can be converted from an iterable of 4 floats.
-
VEC3
= 'vector3' A generic XYZ vector, represented by the
srctools.math.Vec
classs. This can be converted from an iterable of 3 floats. The vector will be copied on insertion/extraction to ensure immutability.
-
VEC4
= 'vector4' A generic XYZW vector, represented by the
Vec4
namedtuple. This can be converted from an iterable of 4 floats.
-
ANGLE
= 'qangle' An Euler angle, represented by the
AngleTup
namedtuple. This can be converted from an iterable of 3 floats,srctools.math.Angle
orsrctools.math.Matrix
.
-
QUATERNION
= 'quaternion' A rotational quaternion, represented by the
Quaternion
namedtuple. This can be converted from a 4-element iterable.
-
MATRIX
= 'vmatrix' A translation
srctools.math.Matrix
. Values are copied on insertion/extraction to keep immutability.
-
-
srctools.dmx.
NULL
= <Null Element> A special constant Element with an all-zero UUID, representing “null” elements in binary DMX files. This always has no members.
-
srctools.dmx.
STUB
= _StubType.STUB Deprecated: Use StubElement.stub()
.
-
class
srctools.dmx.
Quaternion
(x: float, y: float, z: float, w: float) A quaternion used to represent rotations.
-
class
srctools.dmx.
Attribute
(name: str, val_type: ~typing.Literal[<ValueType.ELEMENT: 'element'>], value: ~typing.Union[~srctools.dmx.Element, ~typing.List[~srctools.dmx.Element]],) -
class
srctools.dmx.
Attribute
(name: str, val_type: ~typing.Literal[<ValueType.INTEGER: 'int'>], value: ~typing.Union[int, ~typing.List[int]],) -
class
srctools.dmx.
Attribute
(name: str, val_type: ~typing.Literal[<ValueType.FLOAT: 'float'>], value: ~typing.Union[float, ~typing.List[float]],) -
class
srctools.dmx.
Attribute
(name: str, val_type: ~typing.Literal[<ValueType.BOOL: 'bool'>], value: ~typing.Union[bool, ~typing.List[bool]],) -
class
srctools.dmx.
Attribute
(name: str, val_type: ~typing.Literal[<ValueType.STRING: 'string'>], value: ~typing.Union[str, ~typing.List[str]],) -
class
srctools.dmx.
Attribute
(name: str, val_type: ~typing.Literal[<ValueType.BINARY: 'binary'>], value: ~typing.Union[str, ~typing.List[str]],) -
class
srctools.dmx.
Attribute
(name: str, val_type: ~typing.Literal[<ValueType.TIME: 'time'>], value: ~typing.Union[~srctools.dmx.Time, ~typing.List[~srctools.dmx.Time]],) -
class
srctools.dmx.
Attribute
(name: str, val_type: ~typing.Literal[<ValueType.COLOR: 'color'>], value: ~typing.Union[~srctools.dmx.Color, ~typing.List[~srctools.dmx.Color]],) -
class
srctools.dmx.
Attribute
(name: str, val_type: ~typing.Literal[<ValueType.VEC2: 'vector2'>], value: ~typing.Union[~srctools.dmx.Vec2, ~typing.List[~srctools.dmx.Vec2]],) -
class
srctools.dmx.
Attribute
(name: str, val_type: ~typing.Literal[<ValueType.VEC3: 'vector3'>], value: ~typing.Union[~srctools.dmx.Vec3, ~typing.List[~srctools.dmx.Vec3]],) -
class
srctools.dmx.
Attribute
(name: str, val_type: ~typing.Literal[<ValueType.VEC4: 'vector4'>], value: ~typing.Union[~srctools.dmx.Vec4, ~typing.List[~srctools.dmx.Vec4]],) -
class
srctools.dmx.
Attribute
(name: str, val_type: ~typing.Literal[<ValueType.ANGLE: 'qangle'>], value: ~typing.Union[~srctools.dmx.AngleTup, ~typing.List[~srctools.dmx.AngleTup]],) -
class
srctools.dmx.
Attribute
(name: str, val_type: ~typing.Literal[<ValueType.QUATERNION: 'quaternion'>], value: ~typing.Union[~srctools.dmx.Quaternion, ~typing.List[~srctools.dmx.Quaternion]],) -
class
srctools.dmx.
Attribute
(name: str, val_type: ~typing.Literal[<ValueType.MATRIX: 'vmatrix'>], value: ~typing.Union[~srctools.math.Matrix, ~typing.List[~srctools.math.Matrix]],) -
class
srctools.dmx.
Attribute
() -
class
srctools.dmx.
Attribute
(name: str,)
val_type: ValueType,
value: Union[int, float, bool, str, bytes, Color, Time, Vec2, Vec3, Vec4, AngleTup, Quaternion, Matrix, Element, List[int], List[float], List[bool], List[str], List[bytes], List[Color], List[Time], List[Vec2], List[Vec3], List[Vec4], List[AngleTup], List[Quaternion], List[Matrix], List[Element]], A single attribute of an element.
Attributes store either a single scalar value, or an array of multiple values of the same type. Creation is accomplished mainly via the many classmethods, one for each value type.
To access the value, read/write one of the val_* properties, which will if possible convert the value to the desired type. For arrays either use iter_*() or attr[ind].val_* to fetch by index.
-
classmethod
array
(name: str, val_type: ~typing.Literal[<ValueType.ELEMENT: 'element'>], values: ~typing.Iterable[~srctools.dmx.Element] = (),) Attribute[Element] -
classmethod
array
(name: str, val_type: ~typing.Literal[<ValueType.INTEGER: 'int'>], values: ~typing.Iterable[int] = (),) Attribute[int] -
classmethod
array
(name: str, val_type: ~typing.Literal[<ValueType.FLOAT: 'float'>], values: ~typing.Iterable[float] = (),) Attribute[float] -
classmethod
array
(name: str, val_type: ~typing.Literal[<ValueType.BOOL: 'bool'>], values: ~typing.Iterable[bool] = (),) Attribute[bool] -
classmethod
array
(name: str, val_type: ~typing.Literal[<ValueType.STRING: 'string'>], values: ~typing.Iterable[str] = (),) Attribute[str] -
classmethod
array
(name: str, val_type: ~typing.Literal[<ValueType.BINARY: 'binary'>], values: ~typing.Iterable[bytes] = (),) Attribute[bytes] -
classmethod
array
(name: str, val_type: ~typing.Literal[<ValueType.TIME: 'time'>], values: ~typing.Iterable[~srctools.dmx.Time] = (),) Attribute[Time] -
classmethod
array
(name: str, val_type: ~typing.Literal[<ValueType.COLOR: 'color'>], values: ~typing.Iterable[~srctools.dmx.Color] = (),) Attribute[Color] -
classmethod
array
(name: str, val_type: ~typing.Literal[<ValueType.VEC2: 'vector2'>], values: ~typing.Iterable[~srctools.dmx.Vec2] = (),) Attribute[Vec2] -
classmethod
array
(name: str, val_type: ~typing.Literal[<ValueType.VEC3: 'vector3'>], values: ~typing.Iterable[~srctools.dmx.Vec2] = (),) Attribute[Vec3] -
classmethod
array
(name: str, val_type: ~typing.Literal[<ValueType.VEC4: 'vector4'>], values: ~typing.Iterable[~srctools.dmx.Vec2] = (),) Attribute[Vec4] -
classmethod
array
(name: str, val_type: ~typing.Literal[<ValueType.ANGLE: 'qangle'>], values: ~typing.Iterable[~typing.Union[~srctools.math.Matrix, ~typing.Iterable[float]]] = (),) Attribute[AngleTup] -
classmethod
array
(name: str, val_type: ~typing.Literal[<ValueType.QUATERNION: 'quaternion'>], values: ~typing.Iterable[~srctools.dmx.Quaternion] = (),) Attribute[Quaternion] -
classmethod
array
(name: str, val_type: ~typing.Literal[<ValueType.MATRIX: 'vmatrix'>], values: ~typing.Iterable[~typing.Union[~srctools.dmx.AngleTup, ~srctools.math.Angle, ~srctools.math.Matrix]] = (),) Attribute[Matrix] -
classmethod
array
(name: str, val_type: ValueType) Attribute Create an attribute with an empty array of a specified type.
-
classmethod
time
(name: str, value: Union[Time, float]) Attribute[Time] Create an attribute with a ‘time’ value.
This is effectively a float, and only available in binary v3+.
-
classmethod
vec2
(__name: str, __it: Iterable[float]) Attribute[Vec2] -
classmethod
vec2
() Attribute[Vec2] Create an attribute with a 2D vector.
-
classmethod
vec3
(__name: str, __it: Iterable[float]) Attribute[Vec3] -
classmethod
vec3
() Attribute[Vec3] Create an attribute with a 3D vector.
-
classmethod
vec4
(__name: str, __it: Iterable[float]) Attribute[Vec4] -
classmethod
vec4
() Attribute[Vec4] Create an attribute with a 4D vector.
-
classmethod
color
() Attribute[Color] -
classmethod
color
(__name: str,) Attribute[Color]
r: Union[float, int] = 0,
g: Union[float, int] = 0,
b: Union[float, int] = 0,
a: Union[float, int] = 255, Create an attribute with a color.
-
classmethod
angle
(__name: str, __it: Iterable[float]) Attribute[AngleTup] -
classmethod
angle
() Attribute[AngleTup] Create an attribute with an Euler angle.
-
classmethod
quaternion
(__name: str, __it: Iterable[float]) Attribute[Quaternion] -
classmethod
quaternion
() Attribute[Quaternion] Create an attribute with a quaternion rotation.
-
iter_quat
() Iterator[Quaternion] Iterate over quaternion values.
-
iter_quaternion
() Iterator[Quaternion] Iterate over quaternion values.
-
append
(value: Union[int, float, bool, str, bytes, Color, Time, Vec2, Vec3, Vec4, AngleTup, Quaternion, Matrix, Element],) None Append an item to the array.
If not already an array, it is converted to one holding the existing value.
-
classmethod
-
class
srctools.dmx.
Element
() An element in a DMX tree.
This is a mapping over Attribute objects, representing each key-value pair in the element. Other than the three special attributes name, type and uuid, keys are accessed by regular mapping methods, producing Attribute objects. As a convenience, values or lists of values can be assigned directly to implicitly create the Attribute.
-
name
: str The name can be any string, but usually is some sort of item ID. For elements which are children of another element, the name usually matches the key.
-
type
: str In Valve’s formats, this is the name of the C++ class that the element should deserialise into, like DMElement for example. It can be used for any purpose though. In binary files, elements with identical types names are deduplicated in the file.
-
uuid
: UUID This used in the serialised file to allow elements to be appear multiple times in the tree, including recursively. This does not normally need to be set manually, since a random one will be computed for each new element automatically.
-
classmethod
parse
(file: IO[bytes], unicode: bool = False) Tuple[Element, str, int] Parse a DMX file encoded in binary or KV2 (text).
The return value is the tree, format name and version. The format name and version are stored in the header, and allow validating the kind of dat stored in the file. If unicode is set to True, strings will always be treated as UTF8, even if this module’s encoding name is not provided. Otherwise, only ASCII characters are allowed for safety.
-
classmethod
parse_bin
() Element Parse the core binary data in a DMX file.
The <!– –> format comment line should have already be read. If unicode is set to True, strings will be treated as UTF8 instead of safe ASCII.
-
classmethod
parse_kv2
() Element Parse a DMX file encoded in KeyValues2.
The <!– –> format comment line should have already been read.
-
export_binary
(file: IO[bytes],) None
version: int = 5,
fmt_name: str = 'dmx',
fmt_ver: int = 1,
unicode: Literal['ascii', 'format', 'silent'] = 'ascii', Write out a DMX tree, using the binary format.
Parameters: - file – A writable binary file to save to.
- version – Must be a number from 0-5.
- fmt_name – This can be any string, to indicate what contents the file contains.
- fmt_ver – This can be a format number, to indicate changes in the format.
- unicode – This controls how Unicode characters are handled.
For the unicode parameter, the following values are permitted:
"ascii"
(the default) raises an error if any value is non-ASCII. This ensures no encoding issues occur when read by the game."format"
changes the encoding format to"unicode_binary"
, allowing the file to be rejected if the game tries to read it and this module’s parser to automatically switch to Unicode."silent"
outputs UTF8 without any marker, meaning it could be parsed incorrectly by the game or other utilties. This must be parsed withunicode=True
to succeed.
-
export_kv2
(file: IO[bytes],) None
fmt_name: str = 'dmx',
fmt_ver: int = 1,
*,
flat: bool = False,
unicode: Literal['ascii', 'format', 'silent'] = 'ascii',
cull_uuid: bool = False, Write out a DMX tree, using the text-based KeyValues2 format.
The format name and version can be anything, to indicate which application should read the file.
- If flat is enabled, elements will all be placed at the toplevel, so they don’t nest inside each other.
- If cull_uuid is enabled, UUIDs are only written for self-referential elements. When parsed by this or Valve’s parser, new ones will simply be generated.
- unicode controls whether Unicode characters are permitted:
- ‘ascii’ (the default) raises an error if any value is non-ASCII. This ensures no encoding issues occur when read by the game.
- ‘format’ changes the encoding format to ‘unicode_keyvalues2’, allowing the file to be rejected if the game tries to read it and this module’s parser to automatically switch to Unicode.
- ‘silent’ outputs UTF8 without any marker, meaning it could be parsed incorrectly by the game or other utilties. This must be parsed with unicode=True to succeed.
-
classmethod
from_kv1
(props: Keyvalues) Element Convert a KeyValues 1 property tree into DMX format.
All blocks have a type of “DmElement”, with children stored in the “subkeys” array. Leaf properties are stored as regular attributes. The following attributes are prepended with an underscore when converting as they are reserved: “name” and “subkeys”.
If multiple leaf properties with the same name or the element has a mix of blocks and leafs all elements will be put in the subkeys array. Leafs will use “DmElementLeaf” elements with values in the “value” key.
-
to_kv1
() Keyvalues Convert an element tree containing a KeyValues 1 tree back into a Property.
These must satisfy the format from_kv1() produces - all elements have the type DmElement, all attributes are strings except for the “subkeys” attribute which is an element array.
-
values
() ValuesView[Attribute] Return a view of the attributes for this element.
-
pop
(name: str,) Attribute
default: ~typing.Union[~srctools.dmx.Attribute,
int,
float,
bool,
str,
bytes,
~srctools.dmx.Color,
~srctools.dmx.Time,
~srctools.dmx.Vec2,
~srctools.dmx.Vec3,
~srctools.dmx.Vec4,
~srctools.dmx.AngleTup,
~srctools.dmx.Quaternion,
~srctools.math.Matrix,
~srctools.dmx.Element,
~typing.Sequence[~typing.Union[int,
float,
bool,
str,
bytes,
~srctools.dmx.Color,
~srctools.dmx.Time,
~srctools.dmx.Vec2,
~srctools.dmx.Vec3,
~srctools.dmx.Vec4,
~srctools.dmx.AngleTup,
~srctools.dmx.Quaternion,
~srctools.math.Matrix,
~srctools.dmx.Element]]] = <object object>, Remove the specified attribute and return it.
If not found, an attribute is created from the default if specified, or KeyError is raised otherwise.
-
popitem
() Tuple[str, Attribute] Remove and return a (name, attr) pair as a 2-tuple, or raise KeyError.
-
setdefault
(name: str,) Attribute
default: Union[Attribute, int, float, bool, str, bytes, Color, Time, Vec2, Vec3, Vec4, AngleTup, Quaternion, Matrix, Element, Sequence[Union[int, float, bool, str, bytes, Color, Time, Vec2, Vec3, Vec4, AngleTup, Quaternion, Matrix, Element]]], Return the specified attribute name.
If it does not exist, set it using the default and return that.
-
-
class
srctools.dmx.
StubElement
() In binary DMXes, it is possible to have stub elements representing values excluded from the file.
This always has no members - attempting to add members will fail silently.
-
classmethod
stub
(uuid: Optional[UUID] = None) StubElement Create a stubbed element reference with the specified UUID.
-
classmethod