44. Object-shape rules generated from the Arlington model

Date: 2026-09-26

Status

Accepted on 2026-09-26, by the maintainer, for M02's third slice. It adds a source of rules to the structural profile of 36 without changing its engine, its identifiers' grammar or its severities.

Amended on 2026-09-28, by the maintainer, as M02's third slice implemented it: the tables are generated by a tool and committed, and a test checks them, rather than generated at build time; an override is made only where ISO 32000-1 does not require what the model says; a hand-written rule that reports a fault silences the generated ones on it (Amended on 2026-09-28, below).

Reviewed on 2026-09-29, by the maintainer: the overrides the slice made beyond the seven settled beforehand, and the two candidates the text did not support, are confirmed; the fifth rule, a key newer than the declared version, moves to M20 (Reviewed on 2026-09-29, below).

Implemented by tools/AdCodicem.Pdf.Arlington — the generator, the vendored model and its lock, overrides.tsv —, src/AdCodicem.Pdf/Validation/Arlington/, ArlingtonWalk, the rules object.key-missing, object.value-type-wrong, object.type-value-wrong and object.key-deprecated, NOTICE, and ArlingtonGeneratorTests, ArlingtonModelTests and ArlingtonRuleTests.

Context

M02's object-graph family must check that each dictionary carries the entries its /Type requires, with the types the specification gives them, and that a version does not use keys it deprecates. ISO 32000-2 defines several hundred dictionary types; writing those checks by hand would take many sessions, drift from the specification, and leave most types unchecked.

The PDF Association publishes the Arlington PDF Model: a machine-readable description of every object in ISO 32000-2 — each key, its types, whether it is required, the versions that introduce or deprecate it, and the conditions between keys — as TSV files under the Apache License 2.0. veraPDF, PDFix and BFO already derive checks from it.

M02's first rule of judgment also applies: a validator that calls a widely read file broken is wrong, not strict. A large share of real files carry keys the model says a type must not have, or lack keys it says a type requires, and every reader opens them.

Decision

We will generate the object-shape rules from the Arlington model at build time (amended on 2026-09-28: by a tool, the output committed and checked by a test — below).

• A generator reads the model's TSV files, pinned to one commit of the model's repository, and emits the rule tables as static data in the core — no file is read at run time, and nothing is interpreted by reflection, so the core stays trimming- and AOT-compatible.
• The rules are version-aware: a key is checked against the version the file declares.
• Every generated finding is a warning at most, under the object family's identifiers; a condition an error would need is written by hand, with its own justification.
• The model's license is Apache-2.0: its notice travels with the generated data, and NOTICE names it.
• A model rule that disagrees with qpdf and with the corpus's well-formed documents is overridden by name, with the reason, rather than dropped in silence.

Consequences

• The structural profile covers every dictionary type the specification defines, from one pinned input.
• Updating to a new commit of the model is a reviewed change: the corpus tests show every finding it adds or removes.
• The generator and the pinned model add a build step that CI must run and test.
• Rejected — hand-written rules for every type (slow, and never complete); reading the TSV files at run time (a file dependency and reflection in the core); leaving shape rules to M20's conformance profiles (they are structural, and ADR 36 puts structural rules in the core).
• What would reopen it — the model's license changing, or its maintenance stopping, after which the generated tables would be frozen and maintained by hand.

Amended on 2026-09-28: generated by a tool, checked by a test

Not at build time. The tables are generated by tools/AdCodicem.Pdf.Arlington, a console project of the solution held to the repository's build rules, and the file it writes, src/AdCodicem.Pdf/Validation/Arlington/ArlingtonModel.g.cs, is committed. A unit test regenerates it in memory from the vendored model and overrides.tsv and fails unless it is the committed file byte for byte; another checks every vendored file against the lock's SHA-256. A source generator or an MSBuild task would have hidden the tables from review — a model update would then show in a pull request only through the corpus tests —, and a source generator would have parsed half a megabyte of TSV on every compilation, design-time builds included, for no gain at run time. Committed, the tables read row by row in a diff: one line per row of the model, under a comment naming its object and key. The core's build compiles them and nothing more; the "build step that CI must run" of the consequences above is that test.

The pin. tsv/latest at commit c48b363e9b78902deea03e958693c09339248a3a, with the model's LICENSE and NOTICE.txt, is vendored byte for byte under tools/AdCodicem.Pdf.Arlington/model/, beside model.lock — the commit, then each file's SHA-256. The tool's update --from <clone> re-vendors it from a clone and rewrites the lock; verify checks the lock and the tables.

The data. Static RVA data only — spans of bytes and integers over constants, no static constructor, no string built at run time —, the names compared as ASCII bytes with PdfName.Value, so that the validator never interns a name read from a file. The encoding of the tables is one source file compiled into both the core and the generator, so that their writer and their reader cannot drift. The generator refuses what it cannot encode rather than dropping it: an object of more than 64 rows, an unknown type, a grammar it does not recognize. The tables' data weighs about 88 KB; with the walk and the rules, the core's assembly grows by about 130 KB.

The subset. Four rules — object.key-missing, object.value-type-wrong and object.type-value-wrong, warnings, and object.key-deprecated, information, since ISO 32000-2 still lets a file hold a deprecated key (ADR 45). The version is the header's, or the catalog's /Version when later, never rounded, and none without a header that names one; the model's predicates are evaluated only where they bear on the version alone. A key newer than the version the file declares is not reported: the noisiest of the candidate rules on sound files, it is left to a debt issue. docs/website/docs/reference/validation-rules.md says what the rules read of the model and what they leave silent.

The bar for an override replaces "disagrees with qpdf and with the corpus's well-formed documents", since qpdf checks no object's shape: a row is overridden only where ISO 32000-1 — the version the files declare — does not require what the model says, and its reason in overrides.tsv quotes the text; a genuine ISO 32000-1 violation stays a finding, declared in the corpus manifest, however common. Every override is checked against the text before it is made, and the generator refuses one that no longer changes the pinned model. An override may bound a requirement by version rather than remove it: a form XObject's /Resources, which the model requires from PDF 1.2 on and ISO 32000-1 only recommends, stays required in PDF 2.0 files, the version the model describes. The overrides made, and the part of a candidate the text did not support, are listed in docs/website/docs/reference/validation-rules.md.

One fault, one finding. Where a hand-written rule reports a fault — the page tree's /MediaBox, /Resources, /Parent, /Kids, /Count and kids —, the generated rules stay silent on it, by a named row of overrides.tsv whose reason reads "covered by" that rule; a page tree rule judges only what its walk entered, so the silence holds there alone, and a page the tree does not list is the generated rules' to judge.

The license. NOTICE, at the repository's root, names the model, its commit and what was changed, and carries its notice; the core package carries it, with the Apache License's text as licenses/arlington-pdf-model/LICENSE. The package's license expression stays MIT, by the maintainer's choice: the Apache-2.0 data is attributed by NOTICE, as its section 4 asks.

Reviewed on 2026-09-29: the overrides confirmed, the version rule to M20

The overrides. Slice 3 made three content overrides the seven settled beforehand did not name, and refused two candidates, each after reading ISO 32000-1. The maintainer reviewed each against the text and confirmed all five:

• A form XObject's /FormType and /Matrix are optional in every version, and its /Name is required in PDF 1.0 only (8.10.2, Table 95).
• An optional-content creator's /SubType may be a name or a text string (8.11.4.4, Table 102, "Additional entries may be included").
• A sub-array of /Order nested in another stays a finding (8.11.4.3, Table 101 describes "Arrays of optional content groups").
• A Type 3 font's /Encoding given as a name stays a finding (9.6.5, Table 112, whose text requires "An encoding dictionary whose Differences array shall specify the complete character encoding").

The fifth rule. A key newer than the version the file declares stays out of the structural profile: a newer key still conforms (ADR 45), and the rule flagged 72 to 85 of the corpus's 271 sound documents. The declared version matters where a claim bounds it — PDF/A-1 on PDF 1.4 —, so the rule is M20's to settle, in a profile's terms (#123, moved to M20).