Skip to content

Runtime

The tree-walking engine that drives a transform: apply() / apply_children() route each element through the generated _dispatch router. Every output function receives the run's RenderContext as config; XPath is evaluated by the context's XPathEnvironment, which is built once per document and caches the parsed node trees. xpath_extensions loads user-defined tp:* functions — see the XPath extensions guide.

To drive a compiled module yourself, build one environment per document and pass it to each transform:

from opm.runtime import XPathEnvironment
from opm.transform import run_transform

env = XPathEnvironment(base_uri=path.resolve().as_uri(), extensions=['my.ext'])
html = run_transform(mod, root, xpath_env=env)

Render context

opm.runtime.context

The context one transform run passes to every output function as config.

A RenderContext replaces the plain dict the runtime used to thread through apply, the generated _dispatch and every pmf method. It has three kinds of content, kept apart:

  • Run settings — output functions, dispatch, stylesheet, templates, the $parameters dict, the XPathEnvironment. Fixed for the run.
  • Shared run state — footnotes, collected metadata, counters — in one RunState that every view of the run holds by reference, so nothing is lost when a view is derived.
  • Per-call settingsindent, list_type, template and the like, which a behaviour changes for its children only. RenderContext.derive returns a new view for that; a context is never mutated in place.

Backend-private caches (DOCX images and numbering, JSON source positions) live on the output-functions instance, which a run creates for itself.

RunState dataclass

RunState(
    footnotes: list = list(),
    metadata: dict[str, list[str]] = dict(),
    note_counter: int = 0,
    id_counter: int = 0,
)

Mutable state shared by every view of one transform run.

next_note

next_note() -> int

Number the next footnote.

Source code in src/opm/runtime/context.py
def next_note(self) -> int:
    """Number the next footnote."""
    self.note_counter += 1
    return self.note_counter

next_id

next_id() -> int

A fresh number for a synthetic fragment id.

Source code in src/opm/runtime/context.py
def next_id(self) -> int:
    """A fresh number for a synthetic fragment id."""
    self.id_counter += 1
    return self.id_counter

RenderContext dataclass

RenderContext(
    output: str = "web",
    pmf: ProcessingModelFunctions | None = None,
    dispatch: Callable = _hand_on,
    apply: Callable = _apply,
    apply_children: Callable | None = None,
    parameters: dict[str, Any] = dict(),
    xpath: XPathEnvironment | None = None,
    webcomponents: bool = False,
    odd_css: str = "",
    normalize_text: Callable[[str], str] | None = None,
    text_escape: Callable[[str], str] | None = None,
    input_path: str | None = None,
    docx_template: Any = None,
    typst_functions: frozenset[str] = frozenset(),
    models: dict | None = None,
    root: Any = None,
    template: bool = False,
    indent: str = "",
    list_type: str | None = None,
    list_depth: int = -1,
    list_id: int | None = None,
    table_rows: list | None = None,
    state: RunState = RunState(),
)

Everything a behaviour needs; see the module docstring.

derive

derive(**changes) -> RenderContext

A view with changes applied; the RunState stays shared.

Source code in src/opm/runtime/context.py
def derive(self, **changes) -> RenderContext:
    """A view with *changes* applied; the [`RunState`][opm.runtime.context.RunState] stays shared."""
    return replace(self, **changes)

build_context

build_context(
    root,
    options: dict | None = None,
    *,
    mode: str,
    xpath_env: XPathEnvironment | None = None,
    odd_namespaces: dict[str, str] | None = None,
    **settings,
) -> RenderContext

The context a generated module's transform() runs with.

mode names the output mode; its entry in opm.output_modes supplies the output functions and the text handling. options holds the $parameters plus the run options in RUN_OPTIONS. xpath_env supplies everything else XPath can see; an empty environment when omitted. settings are RenderContext fields the module fixes: dispatch, stylesheet and the like.

Source code in src/opm/runtime/context.py
def build_context(
    root,
    options: dict | None = None,
    *,
    mode: str,
    xpath_env: XPathEnvironment | None = None,
    odd_namespaces: dict[str, str] | None = None,
    **settings,
) -> RenderContext:
    """The context a generated module's ``transform()`` runs with.

    *mode* names the output mode; its entry in [`opm.output_modes`][opm.output_modes]
    supplies the output functions and the text handling. *options* holds the
    ``$parameters`` plus the run options in `RUN_OPTIONS`. *xpath_env*
    supplies everything else XPath can see; an empty environment when omitted.
    *settings* are [`RenderContext`][opm.runtime.context.RenderContext] fields the module fixes: dispatch,
    stylesheet and the like.
    """
    from opm.output_modes import output_mode  # noqa: PLC0415

    from .xpath_env import XPathEnvironment  # noqa: PLC0415

    entry = output_mode(mode)
    opts = dict(options or {})
    env = xpath_env if xpath_env is not None else XPathEnvironment()
    parameters = {k: v for k, v in opts.items() if k not in RUN_OPTIONS}
    metadata = opts.get('metadata')
    return RenderContext(
        output=entry.name,
        pmf=entry.output_functions()(),
        parameters=parameters,
        xpath=env.for_odd(odd_namespaces).with_parameters(parameters),
        webcomponents=bool(opts.get('webcomponents')) and entry.webcomponents,
        docx_template=opts.get('docx_template'),
        input_path=opts.get('input_path'),
        root=root,
        # The caller's dict, when it passes one, so it can read what was collected.
        state=RunState(metadata=metadata if isinstance(metadata, dict) else {}),
        **entry.context_settings(),
        **settings,
    )

XPath environment

opm.runtime.xpath_env

Evaluate ODD XPath against a document, with everything it depends on bound once.

An XPathEnvironment holds what an expression may reach beyond the node it is evaluated on: the source document's URI, the documents and collections doc() and collection() can open, project variables and namespace prefixes, the tp: extension modules, the $parameters map, and the node bound as $parameters?root. It is built once per transform run. All of this used to travel inside the $parameters dict under reserved keys and was re-derived for every predicate: extension fingerprints, merged namespaces and the parameters cache key are now computed when the environment is made.

The environment also owns the per-document caches — the elementpath node tree wrapped around each lxml document, the $parameters maps, the index fn:id() answers from — so they are freed with the run instead of accumulating in module globals for the life of the process. XPathEnvironment.with_root and XPathEnvironment.with_parameters return cheap views sharing those caches, which is how a chunked document binds each chunk's source node without rebuilding anything.

Parsed expressions stay in a bounded module-level cache (compiled_xpath): they depend on strings only, never on a document.

DocumentCache

DocumentCache()

Wrapped node trees, $parameters maps and xml:id indexes for one run.

Keyed by the lxml element object, never by id(): lxml recycles proxy objects, so ids collide across nodes (see opm.runtime.source_map). A wrapped tree keeps its document alive, so an entry stays valid for as long as the cache — and with it the run's environment — exists.

Source code in src/opm/runtime/xpath_env.py
def __init__(self) -> None:
    self.trees: dict[tuple, Any] = {}
    self.parameter_maps: dict[tuple, Any] = {}
    # Keyed by id() of the root node, which the entry keeps alive.
    self.id_indexes: dict[int, tuple[XPathNode, IdIndex]] = {}

XPathEnvironment

XPathEnvironment(
    *,
    base_uri: str | None = None,
    documents: dict[str, Any] | None = None,
    collections: dict[str, list] | None = None,
    variables: dict[str, Any] | None = None,
    namespaces: dict[str, str] | None = None,
    extensions=None,
    parameters: dict[str, Any] | None = None,
    root: _Element | None = None,
    cache: DocumentCache | None = None,
)

Everything an ODD expression can see beyond its context node.

Parameters:

Name Type Description Default
base_uri str | None

URI of the source document; document-uri() and relative doc() arguments resolve against it.

None
documents dict[str, Any] | None

doc() targets by absolute URI, already wrapped (see opm.transform.load_xpath_documents).

None
collections dict[str, list] | None

collection() members by URI.

None
variables dict[str, Any] | None

XPath variables, keys in Clark notation ({ns}local).

None
namespaces dict[str, str] | None

Project prefixes ([transform.namespaces]); they win over the ODD's own declarations, see for_odd.

None
extensions

Dotted module paths whose public callables become tp: functions.

None
parameters dict[str, Any] | None

Bound as $parameters.

None
root _Element | None

Bound as $parameters?root. None binds the document element of the context node, unless parameters has a root of its own.

None
cache DocumentCache | None

Shared document caches; a fresh one by default.

None
Source code in src/opm/runtime/xpath_env.py
def __init__(
    self,
    *,
    base_uri: str | None = None,
    documents: dict[str, Any] | None = None,
    collections: dict[str, list] | None = None,
    variables: dict[str, Any] | None = None,
    namespaces: dict[str, str] | None = None,
    extensions=None,
    parameters: dict[str, Any] | None = None,
    root: etree._Element | None = None,
    cache: DocumentCache | None = None,
) -> None:
    self.base_uri = base_uri or None
    self.documents: dict[str, Any] = documents or {}
    self.collections: dict[str, list] = collections or {}
    self.variables: dict[str, Any] = dict(variables) if variables else {}
    self.project_namespaces: dict[str, str] = dict(namespaces) if namespaces else {}
    self.odd_namespaces: dict[str, str] = {}
    self.extensions = normalize_extensions(extensions)
    self.parameters: dict[str, Any] = dict(parameters) if parameters else {}
    self.root = root
    self._cache = cache if cache is not None else DocumentCache()
    self._ext_fp = extension_fingerprint(self.extensions)
    self._ns_key = self._namespaces_key(self.odd_namespaces, self.project_namespaces)
    self._params_key = params_cache_key(self.parameters)

namespaces property

namespaces: dict[str, str]

The prefixes in effect: the ODD's, overridden by the project's.

with_root

with_root(root: _Element | None) -> XPathEnvironment

A view binding root as $parameters?root.

Source code in src/opm/runtime/xpath_env.py
def with_root(self, root: etree._Element | None) -> XPathEnvironment:
    """A view binding *root* as ``$parameters?root``."""
    return self._view(root=root)

with_parameters

with_parameters(
    parameters: dict[str, Any] | None,
) -> XPathEnvironment

A view binding parameters as $parameters.

Source code in src/opm/runtime/xpath_env.py
def with_parameters(self, parameters: dict[str, Any] | None) -> XPathEnvironment:
    """A view binding *parameters* as ``$parameters``."""
    params = dict(parameters) if parameters else {}
    return self._view(parameters=params, _params_key=params_cache_key(params))

for_odd

for_odd(
    odd_namespaces: dict[str, str] | None,
) -> XPathEnvironment

A view adding the prefixes the ODD root declares.

Prefixes in XPath inside ODD attribute values are not XML names, so the ODD cannot be the only place to declare them: project prefixes win on a clash, and bind what the ODD leaves out (variable prefixes above all).

Source code in src/opm/runtime/xpath_env.py
def for_odd(self, odd_namespaces: dict[str, str] | None) -> XPathEnvironment:
    """A view adding the prefixes the ODD root declares.

    Prefixes in XPath inside ODD attribute values are not XML names, so the
    ODD cannot be the only place to declare them: project prefixes win on a
    clash, and bind what the ODD leaves out (variable prefixes above all).
    """
    odd = dict(odd_namespaces) if odd_namespaces else {}
    return self._view(
        odd_namespaces=odd,
        _ns_key=self._namespaces_key(odd, self.project_namespaces),
    )

wrapped

wrapped(node: _Element)

The elementpath document node for node's tree, built once per run.

The base URI is attached to the document node, which is what makes document-uri() and base-uri() return the source file rather than the empty sequence. The parser's static base URI is a separate thing: it resolves relative arguments to doc() and collection() but never reaches the tree, so both are supplied.

Source code in src/opm/runtime/xpath_env.py
def wrapped(self, node: etree._Element):
    """The elementpath document node for *node*'s tree, built once per run.

    The base URI is attached to the document node, which is what makes
    ``document-uri()`` and ``base-uri()`` return the source file rather than
    the empty sequence. The parser's static base URI is a separate thing:
    it resolves relative arguments to ``doc()`` and ``collection()`` but
    never reaches the tree, so both are supplied.
    """
    return self._cache.tree(node.getroottree().getroot(), self.base_uri)

id_index

id_index(root: XPathNode) -> IdIndex

fn:id()'s index of the node tree under root, built once per run.

See build_id_index.

Source code in src/opm/runtime/xpath_env.py
def id_index(self, root: XPathNode) -> IdIndex:
    """``fn:id()``'s index of the node tree under *root*, built once per run.

    See `build_id_index`.
    """
    hit = self._cache.id_indexes.get(id(root))
    if hit is None:
        hit = (root, build_id_index(root))
        self._cache.id_indexes[id(root)] = hit
    return hit[1]

compile

compile(expr: str, node: _Element)

The parsed expression for expr in the static context of node.

Source code in src/opm/runtime/xpath_env.py
def compile(self, expr: str, node: etree._Element):
    """The parsed expression for *expr* in the static context of *node*."""
    return compiled_xpath(
        expr, default_element_namespace_uri(node), self._ext_fp, self._ns_key, self.base_uri,
    )

context

context(node: _Element) -> XPathContext

An XPathContext with node as context item and everything else bound.

$parameters?root is the viewed node in the original document (tei-publisher-lib). An explicit string root in the parameters is left as a string.

Source code in src/opm/runtime/xpath_env.py
def context(self, node: etree._Element) -> XPathContext:
    """An XPathContext with *node* as context item and everything else bound.

    ``$parameters?root`` is the viewed node in the original document
    (tei-publisher-lib). An explicit string ``root`` in the parameters is
    left as a string.
    """
    tree_root = node.getroottree().getroot()
    wrapped = self.wrapped(tree_root)
    view_root = self._view_root(node)
    if view_root is None:
        pmap = _parameters_map(self._params_key)
    else:
        key = (self._params_key, view_root, self.base_uri)
        pmap = self._cache.parameter_maps.get(key)
        if pmap is None:
            pmap = _parameters_map_with_root(
                self._params_key, view_root, self.wrapped(view_root),
            )
            self._cache.parameter_maps[key] = pmap
    # ODD-declared variables (e.g. $global:register-root) sit alongside
    # $parameters; a project variable never shadows the parameters map.
    variables: dict[str, Any] = {'parameters': pmap}
    for name, value in self.variables.items():
        if name != 'parameters':
            variables[name] = value

    documents: dict[str, Any] | None = self.documents or None
    if view_root is not None:
        view_tree_root = view_root.getroottree().getroot()
        if view_tree_root is not tree_root:
            # Chunking transforms a synthetic copy of the page, while
            # $parameters?root still points into the original document. The
            # two are separate lxml trees, and XPathContext.get_root() only
            # searches the context root's tree and `documents` — so without
            # registering the source document, root($parameters?root) is the
            # empty sequence, and every model reaching the teiHeader through
            # it (page titles, facsimile links) degrades on chunk output.
            # The wrapper is the cached one the parameters map took its root
            # from: get_root() compares by identity.
            documents = dict(documents) if documents else {}
            documents.setdefault(self.base_uri or '', self.wrapped(view_tree_root))

    return XPathContext(
        root=wrapped,  # type: ignore[arg-type]
        item=wrapped.elements[node],  # type: ignore[union-attr,index]
        variables=variables,
        documents=documents,
        collections=self.collections or None,
    )

evaluate

evaluate(node: _Element, expr: str) -> list

Raw elementpath results; raises elementpath.ElementPathError.

Source code in src/opm/runtime/xpath_env.py
def evaluate(self, node: etree._Element, expr: str) -> list:
    """Raw elementpath results; raises `elementpath.ElementPathError`."""
    token = self.compile(expr, node)
    context = self.context(node)
    reset = _CURRENT.set(self)
    try:
        return list(token.select(context))
    finally:
        _CURRENT.reset(reset)

test

test(node: _Element, expr: str) -> bool

Boolean test for an ODD @predicate; an error counts as false.

Source code in src/opm/runtime/xpath_env.py
def test(self, node: etree._Element, expr: str) -> bool:
    """Boolean test for an ODD ``@predicate``; an error counts as false."""
    try:
        result = self.evaluate(node, expr)
    except elementpath.ElementPathError as exc:
        record_xpath_error(expr, exc, node, self.base_uri)
        return False
    if not result:
        return False
    if len(result) == 1 and isinstance(result[0], bool):
        return result[0]
    return True

count

count(node: _Element, expr: str) -> int

Length of the sequence expr selects; an error counts as zero.

Source code in src/opm/runtime/xpath_env.py
def count(self, node: etree._Element, expr: str) -> int:
    """Length of the sequence *expr* selects; an error counts as zero."""
    try:
        return len(self.evaluate(node, expr))
    except elementpath.ElementPathError as exc:
        record_xpath_error(expr, exc, node, self.base_uri)
        return 0

select

select(node: _Element, expr: str)

Evaluate a param @value: nodes as a list, a lone item bare.

count(ancestor::div) or string(.) give a single atomic; a path gives lxml elements. An error gives the empty sequence.

Source code in src/opm/runtime/xpath_env.py
def select(self, node: etree._Element, expr: str):
    """Evaluate a param ``@value``: nodes as a list, a lone item bare.

    ``count(ancestor::div)`` or ``string(.)`` give a single atomic; a path
    gives lxml elements. An error gives the empty sequence.
    """
    try:
        return _unwrap_singleton(_pipeline_values(self.evaluate(node, expr)))
    except elementpath.ElementPathError as exc:
        record_xpath_error(expr, exc, node, self.base_uri)
        return []

select_all

select_all(node: _Element, expr: str) -> list

Like select, but always a list.

Source code in src/opm/runtime/xpath_env.py
def select_all(self, node: etree._Element, expr: str) -> list:
    """Like [`select`][opm.runtime.xpath_env.XPathEnvironment.select], but always a list."""
    try:
        return _pipeline_values(self.evaluate(node, expr))
    except elementpath.ElementPathError as exc:
        record_xpath_error(expr, exc, node, self.base_uri)
        return []

select_or_node

select_or_node(node: _Element, expr: str)

Like select, but node itself when expr is unconfigured.

Used for ODD params that read collection() or an external variable ($global:register-root and friends), which resolve only once the project configures them. Without that configuration the expression raises one of UNCONFIGURED_CODES, and the fallback to the context node is what the bundled teipublisher.odd relies on for its in-document listPerson register. Any other error gives the empty sequence, so a broken expression is not replaced by the whole element.

Source code in src/opm/runtime/xpath_env.py
def select_or_node(self, node: etree._Element, expr: str):
    """Like [`select`][opm.runtime.xpath_env.XPathEnvironment.select], but *node* itself when *expr* is unconfigured.

    Used for ODD params that read ``collection()`` or an external variable
    (``$global:register-root`` and friends), which resolve only once the
    project configures them. Without that configuration the expression
    raises one of `UNCONFIGURED_CODES`, and the fallback to the
    context node is what the bundled teipublisher.odd relies on for its
    in-document listPerson register. Any other error gives the empty
    sequence, so a broken expression is not replaced by the whole element.
    """
    try:
        return _unwrap_singleton(_pipeline_values(self.evaluate(node, expr)))
    except elementpath.ElementPathError as exc:
        message = str(exc)
        if 'XPST0081' not in message:
            # An undeclared prefix means the ODD never opted in to project
            # variables (see PythonGenerator._param_tier_ok), and falling
            # back is then intended, not something to report. A declared
            # prefix with no value or collection behind it is.
            record_xpath_error(expr, exc, node, self.base_uri)
        if any(code in message for code in UNCONFIGURED_CODES):
            return node
        return []

resolve_element

resolve_element(
    document_root: _Element, expr: str
) -> _Element

The one element expr selects with document_root as context.

Raises:

Type Description
ValueError

expr is invalid, or does not select exactly one element.

Source code in src/opm/runtime/xpath_env.py
def resolve_element(self, document_root: etree._Element, expr: str) -> etree._Element:
    """The one element *expr* selects with *document_root* as context.

    Raises:
        ValueError: *expr* is invalid, or does not select exactly one element.
    """
    try:
        raw = self.evaluate(document_root, expr)
    except elementpath.ElementPathError as e:
        raise ValueError(f'Invalid XPath: {e}') from e
    elements = [
        item for item in _pipeline_values(raw) if isinstance(item, etree._Element)
    ]
    if len(elements) != 1:
        raise ValueError(
            f'XPath {expr!r} must select exactly one element; '
            f'got {len(elements)} element(s) from {len(raw)} value(s)',
        )
    return elements[0]

current_environment

current_environment() -> XPathEnvironment | None

The environment evaluating the expression that is running right now.

For functions that need the run's caches while XPath runs, such as fn:id() and tp:source-node. None outside an evaluation.

Source code in src/opm/runtime/xpath_env.py
def current_environment() -> XPathEnvironment | None:
    """The environment evaluating the expression that is running right now.

    For functions that need the run's caches while XPath runs, such as
    ``fn:id()`` and ``tp:source-node``. ``None`` outside an evaluation.
    """
    return _CURRENT.get()

params_cache_key

params_cache_key(
    params: dict | None,
) -> tuple[tuple[str, str], ...]

Hashable, order-independent form of a $parameters dict.

Source code in src/opm/runtime/xpath_env.py
def params_cache_key(params: dict | None) -> tuple[tuple[str, str], ...]:
    """Hashable, order-independent form of a ``$parameters`` dict."""
    if not params:
        return ()
    return tuple(sorted((str(k), str(v)) for k, v in params.items()))

default_element_namespace_uri

default_element_namespace_uri(node: _Element) -> str

Namespace URI used for unprefixed element names in XPath.

XPath 3.1 binds unqualified names to this URI; TEI div lives in http://www.tei-c.org/ns/1.0, not the empty namespace, so parent::div only matches after setting this. Empty string means no default.

Source code in src/opm/runtime/xpath_env.py
def default_element_namespace_uri(node: etree._Element) -> str:
    """Namespace URI used for unprefixed element names in XPath.

    XPath 3.1 binds unqualified names to this URI; TEI ``div`` lives in
    ``http://www.tei-c.org/ns/1.0``, not the empty namespace, so ``parent::div``
    only matches after setting this. Empty string means no default.
    """
    if isinstance(node, (etree._Comment, etree._ProcessingInstruction, etree._Entity)):
        return ''
    return etree.QName(node).namespace or ''

compiled_xpath cached

compiled_xpath(
    expr: str,
    default_element_ns: str = "",
    ext_fp: str = "",
    namespaces: frozenset[tuple[str, str]] | None = None,
    base_uri: str | None = None,
)

Parse each distinct expression and static context once.

Source code in src/opm/runtime/xpath_env.py
@lru_cache(maxsize=8192)
def compiled_xpath(
    expr: str,
    default_element_ns: str = '',
    ext_fp: str = '',
    namespaces: frozenset[tuple[str, str]] | None = None,
    base_uri: str | None = None,
):
    """Parse each distinct expression and static context once."""
    callables = dict(_BUILTIN_XPATH_CALLABLES)
    if ext_fp:
        # A project extension of the same name wins over the built-in.
        callables.update(_loaded_extension_callables(ext_fp))
    parser = build_extension_parser(
        default_element_ns,
        callables,
        namespaces=dict(namespaces) if namespaces else {},
        base_uri=base_uri,
    )
    return parser.parse(expr)

normalize_extensions

normalize_extensions(extensions) -> tuple[str, ...]

Extension modules as a tuple of non-empty dotted paths.

Source code in src/opm/runtime/xpath_env.py
def normalize_extensions(extensions) -> tuple[str, ...]:
    """Extension modules as a tuple of non-empty dotted paths."""
    if not extensions:
        return ()
    if isinstance(extensions, str):
        extensions = (extensions,)
    return tuple(m for m in (str(mod).strip() for mod in extensions) if m)

extension_fingerprint

extension_fingerprint(extensions: tuple[str, ...]) -> str

Cache-key fragment for extensions: module paths plus source mtimes.

Source code in src/opm/runtime/xpath_env.py
def extension_fingerprint(extensions: tuple[str, ...]) -> str:
    """Cache-key fragment for *extensions*: module paths plus source mtimes."""
    if not extensions:
        return ''
    # The separator does not appear in module paths.
    return '\x1f'.join(fingerprint_for_module(module) for module in extensions)

clear_compiled_xpath_cache

clear_compiled_xpath_cache() -> None

Drop parsed expressions and loaded extension modules (e.g. between tests).

Source code in src/opm/runtime/xpath_env.py
def clear_compiled_xpath_cache() -> None:
    """Drop parsed expressions and loaded extension modules (e.g. between tests)."""
    compiled_xpath.cache_clear()
    _loaded_extension_callables.cache_clear()
    _parameters_map.cache_clear()

XPath errors

An expression that fails at run time counts as false or empty. Wrap a run in collect_xpath_errors() to see which ones failed.

opm.runtime.xpath_diagnostics

Collect the XPath errors raised while documents are transformed.

A predicate that raises is treated as false and a param as empty: the processing model has to carry on, and an ODD shared with TEI Publisher may hold expressions only one of the two runtimes can evaluate. The ones the compiler can recognise never reach the runtime (opm.odd_compiler.expression_check). What is recorded here is what is left, in two kinds:

  • Hints — the project configuration is missing something the expression needs: an undeclared prefix, an unset variable, an unregistered tp: function, an unknown collection. Each is recorded once.
  • Failures — everything else: a cast on bad data, a type error, a mistake in a config-supplied XPath. Recorded once per expression and error code, with a count and the location of the first occurrence.

Collection is opt-in and scoped. with collect_xpath_errors() as log: records what is evaluated inside the block, in the current thread or task only; with no block active, recording is a no-op. The CLI wraps each command in one.

XPathFailure dataclass

XPathFailure(
    expression: str,
    code: str,
    message: str,
    count: int = 0,
    element: str | None = None,
    document: str | None = None,
    line: int | None = None,
)

One expression that raised at run time, and where it first did.

XPathErrorLog dataclass

XPathErrorLog(
    failures: dict[tuple[str, str], XPathFailure] = dict(),
    hints: dict[str, str] = dict(),
)

Everything collect_xpath_errors saw.

ordered_failures

ordered_failures() -> list[XPathFailure]

Failures, the most frequent first.

Source code in src/opm/runtime/xpath_diagnostics.py
def ordered_failures(self) -> list[XPathFailure]:
    """Failures, the most frequent first."""
    return sorted(self.failures.values(), key=lambda f: (-f.count, f.expression))

collect_xpath_errors

collect_xpath_errors() -> Iterator[XPathErrorLog]

Record the XPath errors raised inside the with block.

Source code in src/opm/runtime/xpath_diagnostics.py
@contextmanager
def collect_xpath_errors() -> Iterator[XPathErrorLog]:
    """Record the XPath errors raised inside the ``with`` block."""
    log = XPathErrorLog()
    token = _ACTIVE.set(log)
    try:
        yield log
    finally:
        _ACTIVE.reset(token)

record_xpath_error

record_xpath_error(
    expression: str,
    exc: Exception,
    node: _Element | None = None,
    base_uri: str | None = None,
) -> None

Hand one error to the active log; does nothing outside a collection block.

Source code in src/opm/runtime/xpath_diagnostics.py
def record_xpath_error(
    expression: str,
    exc: Exception,
    node: etree._Element | None = None,
    base_uri: str | None = None,
) -> None:
    """Hand one error to the active log; does nothing outside a collection block."""
    log = _ACTIVE.get()
    if log is not None:
        log.record(expression, exc, node, base_uri)

Processing-model runtime

opm.runtime.pm_runtime

Processing-model runtime: apply / apply-children and the node helpers.

Used by ODD-generated modules and runtime helpers. A run's settings and state travel in a RenderContext (config), and its XPath is evaluated by the context's XPathEnvironment.

tag

tag(node: _Element) -> str

Local name for node.

lxml comments, PIs, and entities use a Cython factory object as .tag, not a string, so etree.QName cannot be used on them directly.

Source code in src/opm/runtime/pm_runtime.py
def tag(node: etree._Element) -> str:
    """Local name for *node*.

    lxml comments, PIs, and entities use a Cython factory object as ``.tag``, not a
    string, so `etree.QName` cannot be used on them directly.
    """
    if isinstance(node, etree._Comment):
        return 'comment'
    if isinstance(node, etree._ProcessingInstruction):
        return 'processing-instruction'
    if isinstance(node, etree._Entity):
        return 'entity'
    return etree.QName(node).localname

template_config

template_config(config)

config as seen inside a pb:template.

A behaviour combined with a pb:template receives the already-rendered template nodes as its content, so apply and apply_children must hand them straight on instead of dispatching them again. Mirrors map:entry("template", true()) in model.xql, which is what stops tei-publisher-lib from reprocessing template output.

Without it an ODD whose schemaSpec has ns="" (JATS, and any other vocabulary in no namespace) loses every element a template builds: the generated _dispatch passes foreign-namespace nodes through untouched, but for those ODDs the template's <li> looks exactly like a source element and falls through to "apply children", dropping the wrapper.

Source code in src/opm/runtime/pm_runtime.py
def template_config(config):
    """*config* as seen inside a ``pb:template``.

    A behaviour combined with a ``pb:template`` receives the already-rendered
    template nodes as its content, so [`apply`][opm.runtime.pm_runtime.apply] and `apply_children`
    must hand them straight on instead of dispatching them again. Mirrors
    ``map:entry("template", true())`` in ``model.xql``, which is what stops
    tei-publisher-lib from reprocessing template output.

    Without it an ODD whose ``schemaSpec`` has ``ns=""`` (JATS, and any other
    vocabulary in no namespace) loses every element a template builds: the
    generated ``_dispatch`` passes foreign-namespace nodes through untouched,
    but for those ODDs the template's ``<li>`` looks exactly like a source
    element and falls through to "apply children", dropping the wrapper.
    """
    return config.derive(template=True)

apply

apply(config, nodes, dispatch)

Transform nodes via dispatch(config, node, params).

Source code in src/opm/runtime/pm_runtime.py
def apply(config, nodes, dispatch):
    """Transform nodes via *dispatch(config, node, params)*."""
    if config.template:
        # Template output is finished markup — see [`template_config`][opm.runtime.pm_runtime.template_config].
        return list(normalize(nodes))
    params = config.parameters
    norm = config.normalize_text
    text_escape = config.text_escape
    result = []
    for node in nodes:
        if isinstance(node, (str, etree._ElementUnicodeResult)):
            text = maybe_normalize_text(str(node), norm)
            if text_escape and not isinstance(node, TemplateOutput):
                text = text_escape(text)
            result.append(text)
        elif isinstance(node, etree._Element) and not callable(node.tag):
            result.extend(dispatch(config, node, params))
    return result

apply_template_param_value

apply_template_param_value(config, source_node, raw)

Normalize and dispatch raw for pb:template [[param]] substitution.

XPath (or a literal . param) may yield the context element itself. Passing that element through apply would re-dispatch the same TEI node and, in templates, often stringifies it. When an item is source_node, recurse on child_nodes(source_node) instead (same rule as apply_children).

Source code in src/opm/runtime/pm_runtime.py
def apply_template_param_value(config, source_node, raw):
    """Normalize and dispatch *raw* for ``pb:template`` ``[[param]]`` substitution.

    XPath (or a literal ``.`` param) may yield the context element itself. Passing
    that element through [`apply`][opm.runtime.pm_runtime.apply] would re-dispatch the same TEI node and, in
    templates, often stringifies it. When an item **is** *source_node*, recurse on
    ``child_nodes(source_node)`` instead (same rule as `apply_children`).
    """
    dispatch = config.dispatch
    norm = config.normalize_text
    text_escape = config.text_escape
    result = []
    for item in normalize(raw):
        if isinstance(item, (str, etree._ElementUnicodeResult)):
            text = maybe_normalize_text(str(item), norm)
            if text_escape and not isinstance(item, TemplateOutput):
                text = text_escape(text)
            result.append(text)
        elif isinstance(item, etree._Element):
            if item is source_node:
                result.extend(apply(config, child_nodes(source_node), dispatch))
            else:
                result.extend(apply(config, [item], dispatch))
        else:
            result.append(str(item))
    return result

inject_cached_footnotes

inject_cached_footnotes(nodes: list, config) -> list

Append the footnote bodies collected in config.state after the main flow.

HTML: HtmlOutputFunctions stores dl.footnote elements. Markdown: stores reference-definition strings.

Source code in src/opm/runtime/pm_runtime.py
def inject_cached_footnotes(nodes: list, config) -> list:
    """Append the footnote bodies collected in ``config.state`` after the main flow.

    HTML: [`HtmlOutputFunctions`][opm.runtime.html_output_functions.HtmlOutputFunctions] stores
    ``dl.footnote`` elements. Markdown: stores reference-definition strings.
    """
    footnotes = config.state.footnotes
    if not footnotes:
        return nodes
    if isinstance(footnotes[0], str):
        out = list(nodes) + list(footnotes)
        footnotes.clear()
        return out
    roots = [x for x in nodes if isinstance(x, etree._Element)]
    if not roots:
        return nodes
    target = _footnote_injection_target(roots)
    if target is None:
        return nodes
    for dl in footnotes:
        target.append(dl)
    footnotes.clear()
    return nodes

XPath extensions

opm.runtime.xpath_extensions

Load Python callables as XPath 3.1 extension functions (tp: prefix).

expect_element

expect_element(
    value: Any, *, arg_name: str = "argument"
) -> _Element

Unwrap an XPath item and require an lxml element.

Extension functions receive XPath values; node items can arrive as XPathNode wrappers.

Source code in src/opm/runtime/xpath_extensions.py
def expect_element(value: Any, *, arg_name: str = 'argument') -> ET._Element:
    """Unwrap an XPath item and require an lxml element.

    Extension functions receive XPath values; node items can arrive as
    `XPathNode` wrappers.
    """
    if isinstance(value, XPathNode):
        value = value.value
    if not isinstance(value, ET._Element):
        raise ValueError(f'{arg_name} expects an element node')
    return cast(ET._Element, value)

expect_string

expect_string(
    value: Any, *, arg_name: str = "argument"
) -> str

Normalize an XPath argument to a string.

Accepts atomics, single-item sequences, XPathNode wrappers, and elements. Elements are converted from their string value (concatenated descendant text).

Source code in src/opm/runtime/xpath_extensions.py
def expect_string(value: Any, *, arg_name: str = 'argument') -> str:
    """Normalize an XPath argument to a string.

    Accepts atomics, single-item sequences, XPathNode wrappers, and elements.
    Elements are converted from their string value (concatenated descendant text).
    """
    value = _unwrap_xpath_singleton(value)
    if isinstance(value, ET._Element):
        return ''.join(value.itertext())
    if value is None:
        raise ValueError(f'{arg_name} expects a string-compatible value')
    return str(value)

expect_text

expect_text(
    value: Any,
    *,
    arg_name: str = "argument",
    strip: bool = True,
) -> str

Like expect_string but intended for human-facing text.

Source code in src/opm/runtime/xpath_extensions.py
def expect_text(value: Any, *, arg_name: str = 'argument', strip: bool = True) -> str:
    """Like [`expect_string`][opm.runtime.xpath_extensions.expect_string] but intended for human-facing text."""
    text = expect_string(value, arg_name=arg_name)
    return text.strip() if strip else text

fingerprint_for_module

fingerprint_for_module(module_dotted_path: str) -> str

Cache key fragment: import path plus source mtime when available.

Source code in src/opm/runtime/xpath_extensions.py
def fingerprint_for_module(module_dotted_path: str) -> str:
    """Cache key fragment: import path plus source mtime when available."""
    mod = importlib.import_module(module_dotted_path)
    path = getattr(mod, '__file__', None)
    if path:
        try:
            st = os.stat(path)
            return f'{module_dotted_path}\0{st.st_mtime_ns}'
        except OSError:
            pass
    return f'{module_dotted_path}\0'

load_extension_callables

load_extension_callables(
    module_dotted_path: str,
) -> dict[str, Callable[..., Any]]

Import module_dotted_path and collect public callables (name does not start with _).

Skips classes and non-routine callables so tp: functions map to plain functions/methods.

Source code in src/opm/runtime/xpath_extensions.py
def load_extension_callables(module_dotted_path: str) -> dict[str, Callable[..., Any]]:
    """Import *module_dotted_path* and collect public callables (name does not start with ``_``).

    Skips classes and non-routine callables so ``tp:`` functions map to plain functions/methods.
    """
    mod = importlib.import_module(module_dotted_path)
    out: dict[str, Callable[..., Any]] = {}
    for name in dir(mod):
        if name.startswith('_'):
            continue
        obj = getattr(mod, name)
        if inspect.isclass(obj):
            continue
        if not (inspect.isroutine(obj) or callable(obj)):
            continue
        # Unbound methods etc. are routines; avoid modules
        if inspect.ismodule(obj):
            continue
        out[name] = obj
    return out

build_extension_parser

build_extension_parser(
    default_element_ns: str,
    callables: dict[str, Callable[..., Any]],
    namespaces: dict[str, str] | None = None,
    base_uri: str | None = None,
) -> OpmXPathParser

Create an OpmXPathParser with tp: external functions.

Source code in src/opm/runtime/xpath_extensions.py
def build_extension_parser(
    default_element_ns: str,
    callables: dict[str, Callable[..., Any]],
    namespaces: dict[str, str] | None = None,
    base_uri: str | None = None,
) -> OpmXPathParser:
    """Create an `OpmXPathParser` with ``tp:`` external functions."""
    ns = extension_namespace_map()
    if namespaces:
        ns = {**namespaces, **ns}  # ODD namespaces take precedence over tp: prefix
    kwargs: dict[str, Any] = {'namespaces': ns}
    if default_element_ns:
        kwargs['default_namespace'] = default_element_ns
    if base_uri:
        kwargs['base_uri'] = base_uri
    parser = OpmXPathParser(**kwargs)
    for name, fn in sorted(callables.items()):
        try:
            parser.external_function(
                fn,
                name=name,
                prefix=TEI_PUBLISHER_XPATH_EXT_PREFIX,
                sequence_types=(),
            )
        except ElementPathValueError as e:
            raise ElementPathValueError(
                f'XPath extension {name!r} could not be registered: {e}',
            ) from e
    return parser

Built-in XPath helper functions

opm.runtime.common_xpath_functions

Common XPath extension functions shared across projects.

format_date

format_date(when: Any, locale: Any = 'en') -> str

Format a TEI-style xs:date value for display in a popover/title.

Source code in src/opm/runtime/common_xpath_functions.py
def format_date(when: Any, locale: Any = 'en') -> str:
    """Format a TEI-style xs:date value for display in a popover/title."""
    when = expect_string(when, arg_name='format_date(when)')
    locale = expect_string(locale, arg_name='format_date(locale)')

    if re.fullmatch(r'\d{4}', when):
        return when

    if re.fullmatch(r'\d{4}-\d{2}', when):
        year_str, month_str = when.split('-', 1)
        try:
            parsed = date(int(year_str), int(month_str), 1)
        except ValueError:
            return when
        return babel_format_date(parsed, format='MMMM y', locale=locale)

    if re.fullmatch(r'\d{4}-\d{2}-\d{2}', when):
        try:
            parsed = datetime.strptime(when, '%Y-%m-%d').date()
        except ValueError:
            return when
        return babel_format_date(parsed, format='d MMMM y', locale=locale)

    return when

heading_number

heading_number(div: Any) -> str

Port of pmf:heading-number from ext-common.xql (TEI div outline numbering).

Returns a dotted index such as 1.2.3: at each level, the 1-based index among preceding tei:div siblings, joined from outer ancestor div down to div.

Source code in src/opm/runtime/common_xpath_functions.py
def heading_number(div: Any) -> str:
    """Port of ``pmf:heading-number`` from ``ext-common.xql`` (TEI ``div`` outline numbering).

    Returns a dotted index such as ``1.2.3``: at each level, the 1-based index among
    preceding ``tei:div`` siblings, joined from outer ancestor ``div`` down to *div*.
    """
    if isinstance(div, (list, tuple)):
        if len(div) != 1:
            raise ValueError('heading_number() expects a single node')
        div = div[0]
    node = expect_element(div, arg_name='heading_number()')

    parts: list[str] = []
    cur: ET._Element = node
    while True:
        n = 1
        for sib in cur.itersiblings(preceding=True):
            if _is_tei_div(sib):
                n += 1
        parts.append(str(n))
        parent = cur.getparent()
        if not _is_tei_div(parent):
            break
        cur = parent
    parts.reverse()
    return '.'.join(parts)

roman_fn

roman_fn(n: Any) -> str

XPath-accessible version of ec:roman-fn.

Takes an integer (1-based count) and returns a letter (a-z, no j), cycling through the alphabet using mod 25 arithmetic.

Example: 1 -> 'a', 25 -> 'z', 26 -> 'a', etc.

Source code in src/opm/runtime/common_xpath_functions.py
def roman_fn(n: Any) -> str:
    """XPath-accessible version of ec:roman-fn.

    Takes an integer (1-based count) and returns a letter (a-z, no j),
    cycling through the alphabet using mod 25 arithmetic.

    Example: 1 -> 'a', 25 -> 'z', 26 -> 'a', etc.
    """
    if isinstance(n, (list, tuple)):
        if len(n) != 1:
            raise ValueError('roman_fn() expects a single integer')
        n = n[0]
    try:
        num = int(n)
    except (TypeError, ValueError):
        return str(n)
    return _to_app_label(num)

request

request(uri: Any) -> Any

tp:request(uri) — HTTP GET to uri; XML responses become element nodes.

Inspects the response Content-Type: XML media types (application/xml, text/xml, or */*+xml) are parsed and returned as an elementpath node so path expressions such as tp:request($uri)/entry work. All other types are returned as a decoded string.

When the surrounding document uses a default element namespace (e.g. TEI), unprefixed child steps on the fetched tree resolve in that namespace; for namespace-less API XML use *[local-name()='entry'] instead of /entry.

Source code in src/opm/runtime/common_xpath_functions.py
def request(uri: Any) -> Any:
    """``tp:request(uri)`` — HTTP GET to *uri*; XML responses become element nodes.

    Inspects the response ``Content-Type``: XML media types (``application/xml``,
    ``text/xml``, or ``*/*+xml``) are parsed and returned as an elementpath node
    so path expressions such as ``tp:request($uri)/entry`` work.  All other types
    are returned as a decoded string.

    When the surrounding document uses a default element namespace (e.g. TEI),
    unprefixed child steps on the fetched tree resolve in that namespace; for
    namespace-less API XML use ``*[local-name()='entry']`` instead of ``/entry``.
    """
    uri = expect_string(uri, arg_name='request(uri)')
    if not uri:
        raise ValueError('request(uri) expects a non-empty URI')

    http_req = urllib.request.Request(
        uri,
        method='GET',
        headers={'User-Agent': 'opm/1.0 tp:request'},
    )
    try:
        with urllib.request.urlopen(http_req, timeout=_REQUEST_TIMEOUT_SECS) as resp:
            body = resp.read()
            content_type = resp.headers.get_content_type()
            charset = resp.headers.get_content_charset() or 'utf-8'
    except urllib.error.URLError as e:
        raise ValueError(f'request({uri!r}) failed: {e}') from e

    if _is_xml_content_type(content_type):
        if not body.strip():
            # eXist REST returns 200 + application/xml with an empty body when
            # _xpath matches nothing (_wrap=no); treat as an empty result.
            return ''
        try:
            el = ET.fromstring(body)
        except ET.XMLSyntaxError as e:
            raise ValueError(f'request({uri!r}) returned invalid XML: {e}') from e
        return _wrap_fetched_element(el)

    return body.decode(charset, errors='replace')