Skip to content

traversal

Relation traversal without a precomputed per-class relation map.

model_dump(), save_related() and select_all(follow=True) need to know which relations to follow so the bidirectional relations of ormar do not loop. Instead of the cached _relation_tree() of the class they use a PathRelationMap: one node of the relation tree, computed from the models on the current path only, while the loaded instances are walked. Each class keeps a bounded number of shared nodes.

The mode is selected with the ORMAR_RELATION_TRAVERSAL environment variable (instance, the default, or the deprecated map) or with set_relation_traversal. The map mode is removed in ormar 0.31.0.

PathRelationMap

Bases: dict

One node of the relation tree of a model, decided from the set of model classes on the path above it: a relation is followed unless its target model is in that set (a direct self relation is followed once) or it points to an unresolved forward reference. It answers what the dump and save code asks a relation map (relation in node, node[relation], bool(node)) without building the tree, and a leaf is ... like in the cached tree.

A node depends only on (model, models above), so nodes are shared: each class keeps at most PATH_NODE_CACHE_SIZE of them (oldest dropped first) and the dump of a row only looks them up::

Text Only
1
2
3
node = PathRelationMap.for_path(model=Author, ancestors=NO_ANCESTORS)
"books" in node  # -> True
node["books"]  # -> node of Book below Author, without "author"
Source code in ormar/models/traversal.py
Python
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
class PathRelationMap(dict):
    """
    One node of the relation tree of a model, decided from the set of model
    classes on the path above it: a relation is followed unless its target
    model is in that set (a direct self relation is followed once) or it
    points to an unresolved forward reference. It answers what the dump and
    save code asks a relation map (``relation in node``, ``node[relation]``,
    ``bool(node)``) without building the tree, and a leaf is ``...`` like in
    the cached tree.

    A node depends only on (model, models above), so nodes are shared: each
    class keeps at most ``PATH_NODE_CACHE_SIZE`` of them (oldest dropped
    first) and the dump of a row only looks them up::

        node = PathRelationMap.for_path(model=Author, ancestors=NO_ANCESTORS)
        "books" in node  # -> True
        node["books"]  # -> node of Book below Author, without "author"
    """

    __slots__ = ("model", "ancestors", "child_ancestors")

    def __init__(self, model: type["RelationMixin"], ancestors: frozenset) -> None:
        # the dict maps a followed relation to the node cache of its target,
        # so ``in``, ``bool`` and ``len`` run at plain dict speed
        super().__init__(
            (relation, relation_index(model=target).nodes)
            for relation, target in relation_index(model=model).targets.items()
            if target not in ancestors
        )
        self.model = model
        self.ancestors = ancestors
        self.child_ancestors = ancestors | {model}

    @classmethod
    def for_path(
        cls, model: type["RelationMixin"], ancestors: frozenset
    ) -> "PathRelationMap":
        """
        Returns the shared node of a model below the given set of models.

        :param model: model class of the node
        :type model: type[RelationMixin]
        :param ancestors: model classes on the path above the node
        :type ancestors: frozenset
        :return: node
        :rtype: PathRelationMap
        """
        nodes = relation_index(model=model).nodes
        node = nodes.get(ancestors)
        if node is None:
            node = cls.store(nodes=nodes, node=cls(model=model, ancestors=ancestors))
        return node

    @staticmethod
    def store(
        nodes: dict[frozenset, "PathRelationMap"], node: "PathRelationMap"
    ) -> "PathRelationMap":
        """
        Puts a node into the bounded node cache of its class, dropping the
        oldest one when the cache is full.

        :param nodes: node cache of the class
        :type nodes: dict[frozenset, PathRelationMap]
        :param node: new node
        :type node: PathRelationMap
        :return: the node
        :rtype: PathRelationMap
        """
        if len(nodes) >= PATH_NODE_CACHE_SIZE:
            del nodes[next(iter(nodes))]
        nodes[node.ancestors] = node
        return node

    def __getitem__(self, relation: str) -> Any:
        nodes = dict.__getitem__(self, relation)
        child = nodes.get(self.child_ancestors)
        if child is None:
            target = relation_index(model=self.model).targets[relation]
            child = self.store(
                nodes=nodes,
                node=PathRelationMap(model=target, ancestors=self.child_ancestors),
            )
        return child if child else Ellipsis

    def get(self, relation: str, default: Any = None) -> Any:
        """
        Returns the child node of a relation or ``default`` when the relation
        is not followed.

        :param relation: relation name
        :type relation: str
        :param default: value returned for a relation that is not followed
        :type default: Any
        :return: child node or default
        :rtype: Any
        """
        return self[relation] if relation in self else default

    def items(self) -> Any:
        """
        :return: followed relation names with their child nodes
        :rtype: Iterator[tuple[str, Any]]
        """
        return ((relation, self[relation]) for relation in self)

    def values(self) -> Any:
        """
        :return: child nodes of the followed relations
        :rtype: Iterator[Any]
        """
        return (self[relation] for relation in self)

    def __eq__(self, other: object) -> bool:
        # the values are node caches, comparing them would walk the caches
        return self is other

    __hash__ = None  # type: ignore[assignment]

    def __repr__(self) -> str:  # pragma: no cover
        above = ",".join(sorted(model.get_name() for model in self.ancestors))
        return f"PathRelationMap({self.model.get_name()} below {{{above}}})"

for_path(model, ancestors) classmethod

Returns the shared node of a model below the given set of models.

Parameters:

Name Type Description Default
model type[RelationMixin]

model class of the node

required
ancestors frozenset

model classes on the path above the node

required

Returns:

Type Description
PathRelationMap

node

Source code in ormar/models/traversal.py
Python
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
@classmethod
def for_path(
    cls, model: type["RelationMixin"], ancestors: frozenset
) -> "PathRelationMap":
    """
    Returns the shared node of a model below the given set of models.

    :param model: model class of the node
    :type model: type[RelationMixin]
    :param ancestors: model classes on the path above the node
    :type ancestors: frozenset
    :return: node
    :rtype: PathRelationMap
    """
    nodes = relation_index(model=model).nodes
    node = nodes.get(ancestors)
    if node is None:
        node = cls.store(nodes=nodes, node=cls(model=model, ancestors=ancestors))
    return node

get(relation, default=None)

Returns the child node of a relation or default when the relation is not followed.

Parameters:

Name Type Description Default
relation str

relation name

required
default Any

value returned for a relation that is not followed

None

Returns:

Type Description
Any

child node or default

Source code in ormar/models/traversal.py
Python
259
260
261
262
263
264
265
266
267
268
269
270
271
def get(self, relation: str, default: Any = None) -> Any:
    """
    Returns the child node of a relation or ``default`` when the relation
    is not followed.

    :param relation: relation name
    :type relation: str
    :param default: value returned for a relation that is not followed
    :type default: Any
    :return: child node or default
    :rtype: Any
    """
    return self[relation] if relation in self else default

items()

Returns:

Type Description
Iterator[tuple[str, Any]]

followed relation names with their child nodes

Source code in ormar/models/traversal.py
Python
273
274
275
276
277
278
def items(self) -> Any:
    """
    :return: followed relation names with their child nodes
    :rtype: Iterator[tuple[str, Any]]
    """
    return ((relation, self[relation]) for relation in self)

store(nodes, node) staticmethod

Puts a node into the bounded node cache of its class, dropping the oldest one when the cache is full.

Parameters:

Name Type Description Default
nodes dict[frozenset, PathRelationMap]

node cache of the class

required
node PathRelationMap

new node

required

Returns:

Type Description
PathRelationMap

the node

Source code in ormar/models/traversal.py
Python
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
@staticmethod
def store(
    nodes: dict[frozenset, "PathRelationMap"], node: "PathRelationMap"
) -> "PathRelationMap":
    """
    Puts a node into the bounded node cache of its class, dropping the
    oldest one when the cache is full.

    :param nodes: node cache of the class
    :type nodes: dict[frozenset, PathRelationMap]
    :param node: new node
    :type node: PathRelationMap
    :return: the node
    :rtype: PathRelationMap
    """
    if len(nodes) >= PATH_NODE_CACHE_SIZE:
        del nodes[next(iter(nodes))]
    nodes[node.ancestors] = node
    return node

values()

Returns:

Type Description
Iterator[Any]

child nodes of the followed relations

Source code in ormar/models/traversal.py
Python
280
281
282
283
284
285
def values(self) -> Any:
    """
    :return: child nodes of the followed relations
    :rtype: Iterator[Any]
    """
    return (self[relation] for relation in self)

RelationIndex dataclass

Followable relations of one model class: the target of every relation by name (in name order), the relation names by target model, and the bounded cache of the class's PathRelationMap nodes by the set of models above.

Source code in ormar/models/traversal.py
Python
124
125
126
127
128
129
130
131
132
133
134
135
@dataclass(frozen=True)
class RelationIndex:
    """
    Followable relations of one model class: the target of every relation by
    name (in name order), the relation names by target model, and the bounded
    cache of the class's ``PathRelationMap`` nodes by the set of models above.
    """

    targets: dict[str, Any]
    names_by_target: dict[Any, tuple[str, ...]]
    nodes: dict[frozenset, "PathRelationMap"]
    forward: frozenset[str]

RelationTraversalSettings

Holds the relation traversal mode of the process::

Text Only
1
RelationTraversalSettings.mode  # -> "instance"
Source code in ormar/models/traversal.py
Python
82
83
84
85
86
87
88
89
class RelationTraversalSettings:
    """
    Holds the relation traversal mode of the process::

        RelationTraversalSettings.mode  # -> "instance"
    """

    mode: str = relation_traversal_from_environment()

instance_traversal_enabled()

Checks if the path based instance traversal is enabled.

Returns:

Type Description
bool

True in instance mode

Source code in ormar/models/traversal.py
Python
110
111
112
113
114
115
116
117
def instance_traversal_enabled() -> bool:
    """
    Checks if the path based instance traversal is enabled.

    :return: True in ``instance`` mode
    :rtype: bool
    """
    return RelationTraversalSettings.mode == "instance"

relation_index(model)

Returns the followable relations of a model, leaving out relations to unresolved forward references. Kept on the class until the relation graph changes::

Text Only
1
relation_index(model=Book).targets  # -> {"author": Author, "tags": Tag}

Parameters:

Name Type Description Default
model type[RelationMixin]

model class

required

Returns:

Type Description
RelationIndex

relation targets of the model

Source code in ormar/models/traversal.py
Python
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
def relation_index(model: type["RelationMixin"]) -> RelationIndex:
    """
    Returns the followable relations of a model, leaving out relations to
    unresolved forward references. Kept on the class until the relation graph
    changes::

        relation_index(model=Book).targets  # -> {"author": Author, "tags": Tag}

    :param model: model class
    :type model: type[RelationMixin]
    :return: relation targets of the model
    :rtype: RelationIndex
    """
    cached = model.__dict__.get("_relation_index")
    if cached is not None and cached[0] == RelationGraph.version:
        return cached[1]
    fields = model.ormar_config.model_fields
    targets = {
        relation: fields[relation].to
        for relation in sorted(model.extract_related_names())
        if not fields[relation].has_unresolved_forward_refs()
    }
    names_by_target: dict[Any, tuple[str, ...]] = {}
    for relation, target in targets.items():
        names_by_target[target] = (*names_by_target.get(target, ()), relation)
    index = RelationIndex(
        targets=targets,
        names_by_target=names_by_target,
        nodes={},
        forward=frozenset(
            relation for relation in targets if fields[relation].is_valid_uni_relation()
        ),
    )
    model._relation_index = (RelationGraph.version, index)  # type: ignore[attr-defined]
    return index

relation_paths(model, max_depth, reverse_depth=0, ancestors=NO_ANCESTORS)

Returns the select_related paths reachable from a model, at most max_depth relations deep, with the same rule as PathRelationMap. Forward foreign keys (one parent per row) are followed at every level, reverse and many to many relations (which multiply rows) only in the first reverse_depth levels. Only the deepest paths are listed, as select_related loads the models along a path anyway::

Text Only
1
2
relation_paths(model=Chapter, max_depth=5)  # -> ["book__author__country"]
relation_paths(model=Book, max_depth=1, reverse_depth=1)  # + "tags"

Parameters:

Name Type Description Default
model type[RelationMixin]

model class to start from

required
max_depth int

number of relations to follow at most

required
reverse_depth int

number of first levels that follow reverse and many to many relations too

0
ancestors frozenset

model classes on the path above the model

NO_ANCESTORS

Returns:

Type Description
list[str]

relation paths in dunder notation

Source code in ormar/models/traversal.py
Python
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
def relation_paths(
    model: type["RelationMixin"],
    max_depth: int,
    reverse_depth: int = 0,
    ancestors: frozenset = NO_ANCESTORS,
) -> list[str]:
    """
    Returns the ``select_related`` paths reachable from a model, at most
    ``max_depth`` relations deep, with the same rule as ``PathRelationMap``.
    Forward foreign keys (one parent per row) are followed at every level,
    reverse and many to many relations (which multiply rows) only in the
    first ``reverse_depth`` levels. Only the deepest paths are listed, as
    ``select_related`` loads the models along a path anyway::

        relation_paths(model=Chapter, max_depth=5)  # -> ["book__author__country"]
        relation_paths(model=Book, max_depth=1, reverse_depth=1)  # + "tags"

    :param model: model class to start from
    :type model: type[RelationMixin]
    :param max_depth: number of relations to follow at most
    :type max_depth: int
    :param reverse_depth: number of first levels that follow reverse and many
        to many relations too
    :type reverse_depth: int
    :param ancestors: model classes on the path above the model
    :type ancestors: frozenset
    :return: relation paths in dunder notation
    :rtype: list[str]
    """
    paths: list[str] = []
    if max_depth <= 0:
        return paths
    index = relation_index(model=model)
    for relation in PathRelationMap.for_path(model=model, ancestors=ancestors):
        if reverse_depth <= 0 and relation not in index.forward:
            continue
        nested = relation_paths(
            model=index.targets[relation],
            max_depth=max_depth - 1,
            reverse_depth=reverse_depth - 1,
            ancestors=ancestors | {model},
        )
        paths.extend(f"{relation}__{path}" for path in nested)
        if not nested:
            paths.append(relation)
    return paths

relation_traversal_from_environment()

Returns the relation traversal mode set in the ORMAR_RELATION_TRAVERSAL environment variable, instance when it is not set.

Returns:

Type Description
str

relation traversal mode

Raises:

Type Description
ValueError

on an unknown mode

Source code in ormar/models/traversal.py
Python
67
68
69
70
71
72
73
74
75
76
77
78
79
def relation_traversal_from_environment() -> str:
    """
    Returns the relation traversal mode set in the ``ORMAR_RELATION_TRAVERSAL``
    environment variable, ``instance`` when it is not set.

    :return: relation traversal mode
    :rtype: str
    :raises ValueError: on an unknown mode
    """
    return validate_relation_traversal(
        mode=os.environ.get(RELATION_TRAVERSAL_ENV, DEFAULT_RELATION_TRAVERSAL),
        stacklevel=2,
    )

set_relation_traversal(mode)

Selects how relations are traversed by model_dump, save_related, get_pydantic and select_all(follow=True). The map mode is deprecated and kept for one release as a transitional switch::

Text Only
1
set_relation_traversal(mode="instance")

Parameters:

Name Type Description Default
mode str

instance for the path based traversal of the loaded instances, map for the deprecated cached relation tree

required

Raises:

Type Description
ValueError

on an unknown mode

Source code in ormar/models/traversal.py
Python
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
def set_relation_traversal(mode: str) -> None:
    """
    Selects how relations are traversed by ``model_dump``, ``save_related``,
    ``get_pydantic`` and ``select_all(follow=True)``. The ``map`` mode is
    deprecated and kept for one release as a transitional switch::

        set_relation_traversal(mode="instance")

    :param mode: ``instance`` for the path based traversal of the loaded
        instances, ``map`` for the deprecated cached relation tree
    :type mode: str
    :raises ValueError: on an unknown mode
    """
    RelationTraversalSettings.mode = validate_relation_traversal(
        mode=mode, stacklevel=2
    )

validate_relation_traversal(mode, stacklevel)

Checks a relation traversal mode and warns when the deprecated map mode is selected::

Text Only
1
validate_relation_traversal(mode="instance", stacklevel=2)  # -> "instance"

Parameters:

Name Type Description Default
mode str

map or instance

required
stacklevel int

stack level of the warning, pointing at the caller

required

Returns:

Type Description
str

the mode

Raises:

Type Description
ValueError

on an unknown mode

Source code in ormar/models/traversal.py
Python
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
def validate_relation_traversal(mode: str, stacklevel: int) -> str:
    """
    Checks a relation traversal mode and warns when the deprecated ``map``
    mode is selected::

        validate_relation_traversal(mode="instance", stacklevel=2)  # -> "instance"

    :param mode: ``map`` or ``instance``
    :type mode: str
    :param stacklevel: stack level of the warning, pointing at the caller
    :type stacklevel: int
    :return: the mode
    :rtype: str
    :raises ValueError: on an unknown mode
    """
    if mode not in RELATION_TRAVERSAL_MODES:
        raise ValueError(
            f"Unknown relation traversal mode: {mode}, "
            f"use one of {', '.join(RELATION_TRAVERSAL_MODES)}"
        )
    if mode == "map":
        warnings.warn(
            OrmarDeprecationWarning(
                "The 'map' relation traversal (precomputed relation tree) is "
                "deprecated, the default 'instance' traversal follows the same "
                "relations without it",
                since=(0, 30),
                expected_removal=(0, 31),
            ),
            stacklevel=stacklevel + 1,
        )
    return mode