XML

omniload reads XML files. Like BSON, MessagePack and CBOR it is a read format: it is decoded through the same filesystem readers as CSV, JSONL and Parquet, so any source that reads files can read XML.

There is no XML destination; file:// writes csv, jsonl and parquet only.

Installation

XML support ships in the optional iterable extra, so it is not part of the base install:

pip install 'omniload[iterable]'

If a .xml file is loaded without the extra installed, omniload fails with a clear error naming the exact pip install to run, rather than a bare ImportError.

XML is parsed with a hardened lxml configuration directly, not through the iterabledata bridge, so it can be locked down against XXE / entity-expansion attacks and so a corrupt file raises instead of loading partial data; see File-format routing for how omniload chooses a reader per format.

The #tagname hint is required

XML has no single natural “row” like a JSON array does, so you must tell omniload which repeated element is one row. Append a #tagname=<row-tag> reader hint to the source:

omniload ingest \
    --source-uri 'file://catalog/products.xml#tagname=product' \
    --source-table 'products' \
    --dest-uri duckdb:///local.duckdb \
    --dest-table 'public.products'

Every element whose (local) name matches tagname becomes one row; everything outside those elements (the document root, wrapping containers) is ignored. A namespaced row tag matches on its local name, so #tagname=product matches both <product> and <ns:product>. Without a #tagname hint the load fails with a clear error asking for one.

Where it works

XML is available on every source that goes through the shared file readers:

• Local files: file://

• s3://, gs://, Azure blob storage

• sftp://

Remote reads go through the source’s own fsspec handle, so they reuse its existing authentication (no separate XML storage configuration). A file is read as XML when its extension is .xml (optionally .xml.gz) or when an explicit #xml format hint is appended (the #tagname hint is still required, e.g. #xml&tagname=product). Gzipped files are decompressed automatically. The whole file is read into memory and parsed at once; a corrupt or malformed file raises rather than loading partial data. Character encoding follows the document’s <?xml encoding="..."?> declaration (lxml’s detection); a declaration that disagrees with the actual bytes is rejected with a parse error.

Row shape: elements, attributes and text

Each matched row element is turned into a record:

XML	Record
child element <name>foo</name>	"name": "foo"
repeated child elements <tag>a</tag><tag>b</tag>	"tag": ["a", "b"] (a list)
attribute id="5"	"@id": "5"
element text mixed with attributes/children	"#text": "..."
empty element <empty/>	"empty": null
nested element	a nested object
element with only text (no attributes/children)	the text string directly

Attribute keys are prefixed with @ and mixed text with #text, so neither can collide with a child element’s key. Namespaces are stripped to their local names (<ns:name> becomes name); two children that share a local name across different namespaces collapse into one list. All values are strings (or nested objects/lists), matching XML’s text-only data model; cast them at the destination if you need typed columns.

Parser safety

The parser is configured to neutralize the common XML attack classes without any per-file tuning:

• External entities (XXE) are not resolved, so an &xxe; reference to file:///etc/passwd or a URL never reads a local file or makes a network request: in element text the reference is dropped from the row, and in an attribute value it raises a parse error rather than resolving. (An internal entity defined inline in the document does expand inside an attribute value, because XML normalizes attributes to strings; that is document-local text, not fetched data.)

• Entity-expansion bombs (“billion laughs”) cannot expand, because entities are never resolved; a pathological document is rejected or read without expansion, never materialized.

• External DTDs are not loaded and nothing is fetched over the network.

• Oversized nodes are capped: a single text node larger than lxml’s built-in limit (about 10 MB) is rejected rather than parsed, a denial-of-service guard (huge_tree=False). Tabular XML stays well under this; a document with a single >10 MB value is not supported.

• A malformed document raises rather than being silently half-recovered.

Note

Known limitation: nested XML is flattened. The filesystem readers run with max_table_nesting=0, so a deeply nested element does not become a set of related child tables; nested objects and lists are stored as JSON in a single column (or flattened by the destination’s own rules). For tabular XML with one level of fields per row this is exactly what you want; for deeply hierarchical XML, expect the nested structure to land as JSON rather than normalized tables. Making the nesting depth tunable per reader is a planned follow-up.