M24 — Comparison and templates

State: to do — Depends on: M14, M15, M16, M19, M22 — The AdCodicem.Pdf.Compare satellite of the architecture's package table, on the core alone (ADR 9); every heuristic answer carries its confidence (ADR 15)

Goal

Say what changed between two versions of a document — the draft sent and the contract returned for signature, the revision a signature covers and the one a reader now sees — word by word, with the passages that moved, as JSON or as a redline in PDF; and take the fields of an invoice that carries no structured data from every document of its supplier's layout, by a template defined once, each value with the confidence it has earned.

Two daily tasks of a law firm and of an accounts department motivate it. A contract comes back signed, and the question is whether it is the text that was sent; a signed document carries a later revision, and M04 can say which objects changed but not what the reader now sees in their place. And most supplier invoices still arrive without Factur-X data: their number, dates and totals are typed by hand, or read by a model nobody can explain. The failures this milestone exists to prevent are the ordinary ones: a comparison that flags every page because a page number, a Bates stamp or the producer's line breaks changed; one whose memory grows with the square of the document; a moved clause reported as an unrelated deletion and insertion; a template that returns a wrong total with high confidence, or that quietly extracts another supplier's invoice with the wrong template.

Scope

In:

the AdCodicem.Pdf.Compare satellite, depending on the core alone, AOT-compatible and trimmable;
text comparison: word by word in M15's reading order, with M15's folding, running headers, footers and pagination artifacts left out by default; pages aligned first, then words diffed in windows of pages, so that memory follows the window and the changes, never the document; moves detected; changes typed — inserted, deleted, replaced, moved, number changed, and style changed on request — and placed on both documents' pages with their quads;
what is not text, listed beside it: images replaced, added or removed; annotations (M11's listing); form field values (M16's model); attachments (M06); metadata (M14's XMP model and /Info);
revisions: two revisions of one document compared through M04's PdfRevision.Open(), each change naming the signatures that covered the older text;
outputs: a JSON report with a published schema, a Markdown summary, and a redline PDF — the newer version annotated through M11, both versions annotated, or the two side by side through M09's N-up;
the visual comparison — the comparison of two rasters into regions of change, over the core's page-raster seam, IPdfPageRasterizer and PdfRaster, which M22 put in the core for OCR and M25 implements; proven here on MuPDF's rasters and run in-process once M25 exists;
zone, anchor and pattern templates: fields defined on one document and extracted from others of the same layout; values typed — text, amounts, dates, identifiers with their check digits — and validated across fields; line items from M15's tables; each value with its confidence and evidence; templates as versioned JSON; a matcher that chooses a template for a document or none; a template proposed from one example and its expected values;
the command-line tool's compare and fields verbs.

Out, explicitly:

the rasterizer — M25. Until it exists the visual comparison runs on rasters the caller supplies; a scan without text compares only visually, or after M22's OCR;
comparing DOCX or HTML sources — they are not PDFs; M31 turns DOCX into HTML and M12 into PDF first;
semantic comparison — clauses matched across rewording, meaning — and template-free key-value detection: neither is in the roadmap, and the first is an open question if callers ask; machine-learned extraction never enters the library (invariants 1 and 6, ADR 15);
checking a received Factur-X hybrid's visible fields against its XML — M15, in the AdCodicem.Pdf.FacturX satellite, with M14's matcher over M15's search;
writing extracted fields as Factur-X or UBL — M14's model does that from the caller's values.

Design

Where it lives

AdCodicem.Pdf.Compare, depending on AdCodicem.Pdf only, as the architecture's package table says: text comparison, the redline, the pixel comparison and templates need nothing but the core — M15's extraction and search, M11's annotations, M09's imposition, M04's revisions, M06, M14 and M16's read models. JSON by source generation, month names and number formats from tables of its own (below), no dependency. It ships with its own API baseline — #42's per-package baseline, due since M10.

The raster seam is M22's, IPdfPageRasterizer and PdfRaster, already in the core, in Content/ beside M15's device seam, and not in this satellite: M25's AdCodicem.Pdf.Rendering implements it and depends on the core, SkiaSharp and AdCodicem.Pdf.Imaging only, so a seam kept in Compare would make it depend on Compare, or move a public type between packages once a stable release has shipped it (ADR 30). This milestone consumes it and adds nothing to it.

Namespace	Holds
`AdCodicem.Pdf.Compare`	`PdfComparer`, `PdfRevisionComparer`, the report, the redline
`AdCodicem.Pdf.Compare.Visual`	`PdfVisualComparer`, `PdfVisualOptions`, `PdfVisualDifference` — over the core's `IPdfPageRasterizer` and `PdfRaster` (M22)
`AdCodicem.Pdf.Compare.Templates`	Templates, locators, value types, the matcher, the extractor, the builder

Comparing text

PdfComparer            Compare(older, newer, PdfComparisonOptions, cancellationToken, progress)
                         -> PdfComparisonReport
PdfRevisionComparer    Compare(document, olderRevision, newerRevision, options) — through M04's revisions
PdfComparisonOptions   immutable: M15's extraction options (reading order, visibility, furniture), folding,
                       granularity (words; characters inside a replaced word on request), move threshold,
                       window, number and style changes, non-text changes
PdfComparisonReport    changes in the newer document's order — kind, both sides (page, page label, quads,
                       text), the other end of a move, the signatures that covered the older side, confidence
                       where a heuristic decided —; a summary; diagnostics; streamed to JSON or Markdown

Extraction through M15, page by page, furniture left out: running headers and footers, pagination artifacts, M09's exhibit stamps and Bates numbers. A document stamped for filing does not differ from its original; the stamps are compared on request.
Tokens: each word folded as asked and hashed with a 64-bit hash of our own — string.GetHashCode is randomized per process and System.IO.Hashing is a package —, into a pooled array per window. Equality is confirmed on the text, so a collision costs time, never a wrong answer.
Page alignment: each page's MinHash signature over its word three-shingles; banded dynamic programming aligns the two page sequences (band 16 by default), so that pages inserted, removed or reordered are found before any word is diffed, in time proportional to pages × band.
Word diff: Myers' O((N+M)·D) algorithm, anchored first on words unique to both sides (patience), within windows of aligned pages — four by default, overlapping by one, so that a paragraph flowing across a page break aligns —, in linear space. The edit distance a window may reach is bounded: past it the window is reported replaced as a whole, with a lowered confidence (compare.window-too-different), so that two unrelated documents cost linear time, not quadratic.
Moves: deleted and inserted runs of at least eight words, indexed by fingerprint; exact pairs first, then pairs whose shingles agree at 0.8 or more, a moved and edited passage carrying its inner diff. The index holds the changed runs only.
Placement: every change mapped back to glyphs and quads on both sides through M15.
Determinism: among edit scripts of equal cost the leftmost, moves paired by position, windows in order — the same inputs give the same report, byte for byte (invariant 6).

Kind	Means
`Inserted`, `Deleted`	Words on one side only
`Replaced`	A deletion and an insertion at one place, with the characters that changed inside a word on request
`Moved`	A run found at another place, with its inner changes
`NumberChanged`	A replaced token that reads as a number on both sides — an amount, a date, an article number — flagged apart, since that is what a reader of a contract or an invoice looks for first
`StyleChanged`	Same words, different font, size, weight or color; on request
`ImageChanged`, `AnnotationChanged`, `FieldChanged`, `AttachmentChanged`, `MetadataChanged`	Non-text changes, located by object and page

Revisions and signatures

PdfRevisionComparer opens each revision read-only through M04 and compares them as two documents. For each change it names the signatures whose covered revision holds the older side, so that "the text on page 3 changed after signature 1" is said in words, beside M04's classification of the objects. A revision M04 cannot open reliably, or an encrypted document before M16 opens it, is reported, not guessed (compare.revision-not-read).

The redline

PdfRedline          AnnotateNewer(report, newer, output, options); AnnotateBoth(report, older, newer, outputs,
                    options); SideBySide(report, older, newer, output, options)
PdfRedlineOptions   author and date (the caller's, never the clock), colors, what is marked, save mode

The newer version annotated (the default): insertions and the new side of replacements highlighted, deletions as carets whose popup holds the deleted text, number changes underlined, moves in their own color with a note naming the other place. Every annotation's /Contents states its change, so that M15's comment summary lists the redline as text.
Both versions annotated: strike-outs on the older, highlights on the newer, each linked to the other page.
Side by side: M09's N-up pairs aligned pages on one sheet, annotated on both halves; an inserted or removed page faces a blank half with a note.
M11's rules apply as they are: annotations placed in the structure tree of a tagged document; on PDF/A, M11's Refuse or RemoveClaim; on a signed document, a full rewrite by default — the redline is a working copy —, and on request an incremental update that M04's guard allows when the signatures permit annotations.

Visual comparison

IPdfPageRasterizer  (core, M22) Rasterize(page, resolution, cancellationToken) -> PdfRaster — M25's
                    AdCodicem.Pdf.Rendering implements it; a caller may bring its own
PdfRaster           (core, M22) width, height, stride, format (Gray8, Rgb24, Rgba32), a pooled buffer, the page
                    frame it covers
PdfVisualComparer   Compare(older, newer, PdfVisualOptions) -> PdfVisualDifference: regions of change in page
                    coordinates, the changed fraction, and on request a difference raster

Rasters are aligned on the crop box at one resolution; pages of different sizes are compared on their common area, the rest being a region of its own.
A pixel differs when a channel differs by more than a threshold — 24 of 255 by default, M09's cross-engine rule — and no pixel of its neighborhood in the other raster matches it, which absorbs anti-aliasing. Differing pixels are dilated, grouped by an iterative union-find over rows whose memory follows the width, and merged into rectangles; regions below a floor are dropped.
Text changes and regions are joined in the report: a region with no text change is visual only — a field appearance lying over signed content, the case M04 says only a rendering sees, is exactly that.
The difference raster is written as PNG by a small encoder of the satellite's own over the BCL's ZLibStream.

Templates

PdfExtractionTemplate   id, version, name, fingerprint (anchors that must be present, page size class), fields
PdfTemplateField        name, value type, locator (Zone, Anchor, Pattern, TableColumn), page scope, required,
                        alternatives, validators
PdfTemplateMatcher      Match(document, templates) -> the template and its confidence, or none
PdfFieldExtractor       Extract(document, template, options) -> PdfExtractedFields: per field, the typed value,
                        the raw text, confidence, evidence (page, quads, the anchor used), or absence and why
PdfTemplateBuilder      a fluent definition; FromExample(document, expectedValues) -> a proposed template
PdfTemplateJson         export and import, versioned, with a schema published on the documentation site

Locators. Zone: a rectangle in the crop box's frame, relative to the page's size — so that A4 and Letter renditions of one layout agree — or to an anchor. Anchor: a label M15's search finds — literal with folding, or a pattern; which occurrence; which pages — and the value in a direction from it: to the right on its line up to a gap or the next label, below within its column, or in a box relative to the label's quad. A field may carry alternatives — the same label in German and in English, or as an older version of the layout wrote it — and says which one answered. Pattern: a regular expression with a named group, without backtracking, within a zone or a page. TableColumn: M15's table whose header row holds given texts, a column by its header; rows become line items.
Value types: text; amounts, with grouping and decimal separators in every order a European invoice writes them (1 250,00 €, 1.250,00, 1,250.00, 571.04), a currency symbol or code, and negative forms — a leading or trailing minus, parentheses, the soft hyphen Axapta draws for a minus (M15 reads it as one); dates, numeric in the order the template states and with month names in French, German, English, Spanish, Italian and Dutch, from tables of the satellite's own — CultureInfo answers from ICU, whose data varies by platform and vanishes under InvariantGlobalization; percentages; identifiers with their check digits — IBAN (ISO 13616, modulo 97), BIC by shape, French SIREN and SIRET (Luhn, with La Poste's SIRET exception), VAT numbers (the French key computed, others by shape).
Validators across fields: net plus tax equals gross within a cent; line amounts sum to the net; quantity times unit price equals the line amount; the due date is not before the issue date. A validator that holds raises the confidence of the fields it ties; one that fails lowers it and is reported.
Confidence combines the locator's (an exact label, a folded one, an alternative, a zone alone), the parse's (strict or lenient), coverage (the value's words wholly inside the region) and the validators'. It is calibrated on the corpus in slice 7 and recorded in status.md. The promise the acceptance holds it to: no value known to be wrong is ever reported at 0.5 or above; every value known to be right that passes its own checks is reported at 0.9 or above; and a value that fails its check digits is capped below 0.5, whatever else says it is right — it is returned as written, and flagged.
Matching: a template's required anchors present, and their relative positions within a tolerance of the fingerprint's; a document no template matches gets none, never the nearest.
By example: given a document and the values expected of it — from its embedded XML, or typed by a person — the builder finds each value's occurrences, takes the nearest label to its left or above (a run ending in a colon, or set apart by style), and proposes an anchor, a zone as a fallback, and the validators the values satisfy. The caller reviews the proposal; nothing is learned silently.

Bounds, classified (invariant 12, ADR 34)

The satellite reads documents only through the core, whose guards apply. What it adds is bounded by its inputs and by options, none of them a reader limit:

Bound	Kind	Why
The edit distance a window may reach before it is reported replaced whole	Option, `PdfComparisonOptions.MaxEditsPerWindow`, reported `compare.window-too-different`	Two unrelated documents are valid inputs; the bound keeps them linear
Pages a page may move in the alignment band	Option, `AlignmentBand`	Wider bands cost time linearly; a page moved further is reported as removed and inserted
A template's pattern	`NonBacktracking`, as M15's search	A caller's pattern over a hostile document
Pixels of a raster	Width × height checked, the buffer pooled, the union-find's memory by row	The rasterizer's resolution is the caller's

Diagnostics

Each report carries its own diagnostics, as M09's operations do:

Code	Severity	Meaning
`compare.window-too-different`	Warning	A window past its edit bound, reported replaced whole
`compare.page-without-text`	Information	A page with no text on one side — compared visually or not at all
`compare.revision-not-read`	Warning	A revision M04 cannot open reliably, or an encrypted document
`compare.reading-order-uncertain`	Information	M15's confidence in a page's order is below the threshold; moves on it may be artifacts of order
`template.anchor-missing`, `template.anchor-ambiguous`	Warning	A required label not found, or found more often than the template allows
`template.value-unparsed`, `template.check-digit-failed`, `template.validation-failed`	Warning	A value that does not read as its type, fails its check digits, or breaks a validator
`template.no-match`	Information	No template matches the document

The command-line tool

compare OLD NEW [--json | --markdown] [--redline OUT] [--both OUT-OLD OUT-NEW] [--side-by-side OUT] [--revisions A B] [--visual] — --visual only once a rasterizer is present —, and fields FILE --template T.json [--json], fields FILE --learn EXPECTED.json --out T.json. The AOT binary produces what the API produces.

Slices

Each slice ends on a green commit, with its codes documented and its benchmark, if it has one, recorded in docs/status.md.

The satellite, tokens and the word diff. Delivers the package with its baseline and AOT check, the hash, Myers with patience anchoring over a single window, the report and its JSON schema. Proved by FsCheck — for any two word sequences, the reported edits applied to the older give the newer; with the patience anchoring off, on sequences up to 200 words, the edit count equals a reference longest common subsequence's, since anchoring trades minimality for readable diffs; two runs give identical reports —; integration: on the edited pairs (below), the words we insert and delete agree, as multisets, with the difference between the two sides' pdftotext words — a check no correct edit script can fail, whatever its alignment —, and fall in the regions Python's difflib aligns, in a container. Leaves long documents.
Page alignment, windows and moves. Delivers MinHash signatures, the banded alignment, windows with overlap, the edit bound, move detection. Proved by FsCheck — pages inserted, removed and swapped are found; a run of eight words or more moved elsewhere is one move —; a hostile pair of unrelated 1,000-page documents compared in linear time; the journal and its edited copy (below) within the memory budget. Leaves furniture and non-text.
Normalization, furniture and what is not text. Delivers the folding options, furniture left out, number and style changes, the non-text comparisons. Proved by unit tests (a page number changed in a footer, a Bates stamp, ﬁ against fi, a hyphenated break against a whole word, a field value, an image recompressed with the same pixels reported as changed data and not as a changed placement); integration: the invoice through five producers, and stamped copies against their originals. Leaves revisions.
Revisions and signatures. Delivers PdfRevisionComparer and the signature attribution. Proved by the revision rows below, each consecutive pair against difflib over pdftotext of each revision cut at its %%EOF, and pyHanko's account of which revision each signature covers. Leaves output.
The redline. Delivers the three redline forms through M11 and M09. Proved by unit tests (an insertion at a page break, a deletion of a whole page, a move between pages, determinism under a frozen and a moving clock); integration: pdf.js and PyMuPDF list every annotation, PyMuPDF's words under each highlight are the inserted words and under each strike-out the deleted ones, qpdf accepts every file, veraPDF finds no new PDF/UA-1 failure on a tagged input, pyHanko finds every signature intact after an incremental redline. Leaves pixels.
Visual comparison over rasters. Delivers PdfVisualComparer over M22's IPdfPageRasterizer and PdfRaster, the PNG encoder, the join with text changes. Proved by unit tests on synthetic rasters (anti-aliased edges below the threshold ignored, a box moved by ten pixels giving two regions, pages of different sizes); integration: on the edited pairs rasterized by MuPDF (mutool draw to raw PAM) at 150 dpi, our regions equal those NumPy finds on the same rasters in the referee container, every text change lies inside a region, and a damaged copy against its original gives none. Leaves templates; M25 plugs its rasterizer into M22's seam.
Templates: locators, value types and confidence. Delivers the template model and JSON, the four locators, the value types with their parsers, check digits and validators, the extractor, the calibration. Proved by unit tests per type and form — every amount and date form, IBAN, SIREN and SIRET with valid and invalid keys —; FsCheck — any amount written in any of the supported forms parses back to itself —; integration: the GnuAccounting family's values against their embedded XML, read with XPath from what pdfdetach extracts. Leaves line items and matching.
Line items, matching and templates by example. Delivers TableColumn line items, PdfTemplateMatcher, FromExample. Proved by unit tests (a table without rules, a line item wrapped over two lines, an ambiguous anchor); integration: the line items against the XML, the matcher over every invoice in the corpus, and a template proposed from invoice 505 and its XML extracting the family. Leaves the tool.
The tool, budgets and the whole. Delivers compare and fields, ComparisonBenchmarks and TemplateBenchmarks with MemoryDiagnoser, the documentation. Proved by CorpusToolTests, the budget rows and a green Remote corpus run recorded in status.md.

Tests required

Unit — tests/AdCodicem.Pdf.Tests, under Compare/, as M10 placed its satellite's:

The diff: empty documents, identical ones, one word against none, every change kind, moves across windows, pages inserted at the start and removed at the end, a window past its bound; the FsCheck properties of slices 1 and 2; determinism.
Normalization and furniture: each folding option, each furniture source, a number reformatted without changing (1 250,00 against 1250,00) reported as unchanged when numbers are compared by value.
The redline: each form, each change kind at page edges, dates and names from the caller only.
The visual comparison: synthetic rasters for every rule; an 18,000 × 18,000 raster compared with memory bounded by its width.
Templates: every value type, check digit and validator; each locator on rotated, cropped and UserUnit pages; confidence never rising when evidence is removed (FsCheck over the evidence components).
Hostile: a page of a million identical words; two unrelated documents of a thousand pages; an anchor that occurs ten thousand times; an amount of a hundred thousand digits; a template JSON that nests, lies about its version or names a pattern built to backtrack — each ends in a result, a report or a typed exception within time and allocation budgets.

Integration — tests/AdCodicem.Pdf.IntegrationTests, every referee in a container (ADR 27):

poppler — pdftotext for each side, and pdfdetach for the invoices' XML; Python difflib — the independent word diff;
PyMuPDF — words under the redline's quads; annotation listing; pdf.js — annotation listing and, through M11's harness, the redline's rendering;
MuPDF — the rasters of slice 6, and NumPy — the independent pixel comparison;
qpdf, veraPDF and pyHanko — the redline's files, their PDF/UA-1 claims and their signatures;
lxml — the XPath reading of each invoice's embedded XML, the templates' ground truth.

Acceptance conditions

"The edited pairs" are the documents below, each generated from a source and from an edited copy of it — not in the corpus (below) —, the edits recorded with the sources, so that what changed is known by construction and confirmed by an independent diff. "Agree as multisets" means that the words a report inserts, minus the words it deletes, equal the difference between the two sides' pdftotext words: every correct edit script meets it, whatever it aligns with what.

Documents	Behavior	Verified by
The edited pairs of `documents/contract/chromium-contract-fr.pdf`, `documents/report/chromium-report-fr.pdf` and `libreoffice-report-fr.pdf`, `documents/invoice/chromium-invoice-fr.pdf` — words changed, a paragraph inserted, a clause deleted, an article moved, an amount changed, text reflowed over a page break	Exactly the recorded edits: each as the kind it is, the article as one move, the amount as a number change, the reflow as nothing; the words agree as multisets	`CorpusComparisonTests.Edited_copies_report_exactly_their_edits`
Every committed document that opens and has text, edited in the test through M19's content pipeline — three words removed from a page — and M08's builder — one sentence added to another page	Exactly those words deleted and that sentence inserted, at their pages and quads; nothing else; the words agree as multisets	`CorpusComparisonTests.Every_document_reports_exactly_the_edits_made_to_it`
`documents/invoice/chromium-invoice-fr.pdf` against its damaged copies `documents/damaged/invoice-no-xref.pdf`, `invoice-shifted-offsets.pdf`, `invoice-junk-prefix.pdf`, `invoice-lying-length.pdf`, and against `invoice-truncated-tail.pdf`	No change for the four the reader recovers whole; for the truncated tail, deletions exactly where M05's report says content was lost	`CorpusComparisonTests.Damage_the_reader_recovers_is_no_change`
The invoice through five producers — Chromium, LibreOffice, Word, Microsoft Print to PDF, PDF24 (`documents/invoice/*-invoice-fr.pdf`) —, pairwise	The words agree as multisets, folded; no change from line breaks, fonts or positions alone; where `difflib` aligns a change elsewhere, the difference is recorded with its reason	`CorpusComparisonTests.Producers_differ_only_by_the_words_an_independent_diff_sees`
Every committed document that opens, stamped by M09 — exhibit stamp and Bates numbers — against its original	No change with furniture left out; exactly the stamps' text when it is compared	`CorpusComparisonTests.Stamps_are_furniture_not_changes`
Consecutive revisions of `vendor/us-federal/illustrator-irs-pub1-english.pdf` (an update that removes text), `vendor/uk-ogl/pdfmaker25-home-office-eia.pdf` (metadata only), `documents/invoice/word-invoice-fr.pdf` (an empty update), `vendor/pdf-association/indesign13-pdfua1-german-book-chapter.pdf` (four revisions), `vendor/fr-licence-ouverte/libreoffice-cerfa-13983-form.pdf` (four re-saves), `vendor/uk-ogl/pdfmaker10-word-dh-care-bill-factsheet.pdf` (pages rewritten in an object stream)	The words agree as multisets with `pdftotext` of each revision cut at its `%%EOF`, and fall where `difflib` places them; no text change for the metadata-only and empty updates, `MetadataChanged` for the metadata-only ones	`CorpusRevisionComparisonTests.Consecutive_revisions_differ_by_the_text_an_independent_diff_sees`
The signed documents with later revisions: `vendor/pyhanko/acrobat-reader-signed-twice.pdf`, `vendor/fr-licence-ouverte/fop-dictao-dila-signed-joafe-notice.pdf`, `vendor/lu-legilux/antenna-house-legilux-memorial-pades-lta.pdf`, `fop22-legilux-memorial-seal-renewed-timestamps.pdf`; remote, the EU DSS files with timestamps and 48 updates	Each change names the signatures that covered the older side, as pyHanko places them; updates that add only signatures, timestamps or a security store change no text	`CorpusRevisionComparisonTests.Changes_after_a_signature_name_the_signatures_they_follow`
The edited pairs, redlined in the three forms; the InDesign PDF/UA-1 chapter's first and last revisions, redlined	pdf.js and PyMuPDF list every annotation; each highlight covers exactly the inserted words and each strike-out the deleted ones; qpdf accepts every file; the chapter's redline passes PDF/UA-1 in veraPDF with no failure its input did not have; two runs give identical bytes	`CorpusRedlineTests.Redlines_mark_exactly_the_changed_words`, `CorpusRedlineTests.A_redline_of_a_pdfua_document_keeps_it_valid`
`acrobat-reader-signed-twice.pdf` redlined against its first revision, in an incremental update	pyHanko finds both signatures intact, the redline an annotation-level change	`CorpusRedlineTests.An_incremental_redline_keeps_approval_signatures`
The edited pairs and the damaged copies, rasterized by MuPDF	Regions equal NumPy's on the same rasters; every text change inside a region; no region for the damaged copies the reader recovers whole	`CorpusVisualComparisonTests.Changed_regions_agree_with_an_independent_pixel_diff`
M04's shadow replace fixture whose field appearance lies over signed content, against its signed revision	A `FieldChanged` change and a region of change over content the signature covered, joined in the report and named as visual; the signed text itself unchanged	`CorpusVisualComparisonTests.A_field_laid_over_signed_content_is_seen`
The GnuAccounting family: `vendor/zugferd/mustang-zugferd1-basic-invoice.pdf` (RE-20170509/505) and `pypdf2-facturx-python-false-pdfa3b.pdf` (RE-20171118/506), one layout with German labels	A template defined on 505 extracts from 506 its number, issue, delivery and due dates, seller, buyer, net, tax and gross totals, IBAN and the three line items, each equal to 506's embedded XML and at 0.9 or above	`CorpusTemplateTests.A_template_extracts_a_suppliers_other_invoices`
The rest of the family: `gnuaccounting-mustang10-zugferd-rc-invoice.pdf` (RE-20140703/502 — US Letter, the 2014 labels "Summe netto" and "Summe inkl. USt", a spaced IBAN) and `mustang-zugferd2-en16931-invoice.pdf` (RE-20190610/507 — English labels, ISO dates, decimal points)	Without alternatives, the fields whose labels changed are absent or below 0.5 and never a wrong value at 0.5 or above; with the 2014 and English labels as alternatives, every field the page shows equals the XML at 0.9 or above, the zones scaled to Letter, and the delivery date 502's XML carries but its page does not show is absent, not guessed	`CorpusTemplateTests.A_changed_layout_is_answered_by_alternatives_or_not_at_all`
The invoice through five producers, the template defined on Chromium's	The same values from every rendition as `sources/invoice-fr.html` writes them; the IBAN and the buyer's SIREN, fictitious and failing their check digits, returned as written with `template.check-digit-failed` and below 0.5	`CorpusTemplateTests.A_template_survives_a_change_of_producer`
Every other invoice of the corpus — `vendor/docentric/*` (one invoice, two layouts), weclapp, Kraxi, the Swiss QR-bill, JasperReports, OZEV; remote, SAP, Axapta, Scoro, the DWC and intarsys files	The matcher answers none for the GnuAccounting and Chromium templates; applied anyway, no value at 0.5 or above	`CorpusTemplateTests.A_template_does_not_claim_another_layout`
Invoice 505 and its XML as the example	The proposed template extracts 506 as the hand-written one does	`CorpusTemplateTests.A_template_proposed_from_one_example_extracts_the_others`
`documents/stress/reportlab-journal-1000-pages.pdf` and its edited copy — not in the corpus (below)	Memory bounded by the window, flat across 10, 100 and 1,000 pages, within the budget recorded in `status.md`; every edit found	`CorpusComparisonTests.Comparing_a_thousand_pages_holds_its_budget`, `ComparisonBenchmarks`
The same operations through the tool	The AOT binary produces what the API produces	`CorpusToolTests.Compare_and_fields_match_the_api`

The remote rows close only on a green Remote corpus run, recorded in status.md with its date. The visual row closes here on MuPDF's rasters; M25 adds the same row on its own rasterizer.

Corpus

What the corpus holds

The same text through different producers: the invoice from Chromium, LibreOffice, Word, the Print to PDF driver and PDF24, and its five damaged copies, one of them missing content.
Revision histories: an update that removes text (revision-removes-text), metadata-only updates, an empty update, four revisions of a PDF/UA-1 chapter, four re-saves of a Cerfa, pages rewritten in an object stream.
Signed documents with later revisions: approval signatures twice over, a DILA notice, two Luxembourg memorials with timestamps; remote, the EU DSS files up to 48 updates.
One supplier's invoices: the GnuAccounting family — four committed invoices, each with its CII XML (ZUGFeRD 1 RC, ZUGFeRD 1 BASIC, Factur-X BASIC, ZUGFeRD 2 EN 16931) as ground truth: two of one layout, one older version of it on US Letter, one in English.
Negative cases for matching: the Docentric pair — one invoice, two layouts —, and every other invoice.
Changes made after a signature, crafted: M04's shadow-attack fixtures, a priority-1 gap of M04's filled before it closed — among them a field appearance laid over signed content, the change only a rendering sees.
Scale: the 1000-page journal.

What it lacks

Need	Why	Priority	Likely source
Edited copies of the generated contract, report and invoice — words changed, a paragraph inserted, a clause deleted, an article moved, an amount changed, a reflow across a page break —, with the edits recorded	"Exactly the edits made" needs edits known by construction that also reflow the text, which in-place edits of a finished PDF never do	1	Generated here: edited copies of `sources/*.html` beside the originals, through Chromium and LibreOffice, recorded in `build_corpus.py`
An edited copy of the 1000-page journal, with scattered edits	The memory budget must be measured on a long document with changes, not on a document against itself	1	Generated here: `build_reportlab_stress` with a recorded seed of edits
More supplier families — three or more invoices of one layout from a real ERP, with or without XML	The template promise is proven on one supplier's family and on our own invoice	2	A contribution (W04, anonymized); generated here: our invoice template filled with other values through Chromium and LibreOffice, marked as ours
A draft and the signed version returned by a counterparty, from practice	The daily redlining case; everything above is constructed or one-sided	2	A contribution (a new "draft and signed pair" want in `docs/corpus-contributions.md`)
Multi-page invoices whose line items continue across pages	Line items are proven on one-page tables	2	Generated here: our invoice template with forty lines through Chromium and LibreOffice; a contribution
Scanned versions of the same document, and a scan against its born-digital original	The visual comparison on scans, once M25 and M22 exist	3	Generated here: the committed documents printed to raster by a recorded Ghostscript conversion, marked as derived

Traps

Reading order is the comparison's ground. Where M15 is unsure of a page's order, a diff sees moves that are artifacts of the order; the report says so rather than presenting them as the author's.
Furniture changes on every page. Page numbers, Bates numbers, exhibit stamps and running headers differ between a draft and a filed copy; left in, they drown the real changes.
Line breaks and hyphenation are layout, not text. A reflowed paragraph is not a changed one; a hyphenated break is one word.
Longest common subsequence is quadratic. Without page alignment, windows and an edit bound, two long or unrelated documents take the process.
A move is a pairing decision. Pair too eagerly and common phrases ("the Parties agree") move around; too shyly and every moved clause is a deletion and an insertion.
string.GetHashCode is randomized per process: a report ordered by it changes from run to run.
Numbers deserve their own kind. An amount that changed in a contract is the change a reader must not miss.
A redline on a signed document is a change after the signature. Written as an incremental update it must stay within what the signature allows, and the report must not be mistaken for the signed text.
Labels change between versions of one supplier's layout, and the same supplier writes in two languages: a template's anchors carry alternatives, or it answers nothing.
A value that reads well can still be wrong. A total at the wrong anchor parses as an amount; validators across fields and check digits are what catch it, and the confidence must fall when they fail.
Month names and number formats from CultureInfo depend on ICU, which differs by platform and disappears under InvariantGlobalization; the satellite's tables do not.
Anti-aliasing is not a change, and a recompressed image is a changed object with unchanged pixels.

Documentation

docs/website/docs/guides/comparing-documents.md (new): comparing two versions and two revisions, the change kinds, furniture, moves, the redline forms, signed documents.
docs/website/docs/concepts/comparison.md (new): alignment, windows, the edit bound, determinism, why memory follows the changes and not the document.
docs/website/docs/guides/visual-comparison.md (new): the rasterizer seam, rasters from M25 or the caller's, thresholds, visual-only changes.
docs/website/docs/guides/extraction-templates.md (new): locators, alternatives, value types and check digits, validators, confidence and what it promises, matching, templates by example.
docs/website/docs/reference/comparison-report-schema.md and template-schema.md, with the schemas.
docs/website/docs/reference/tool/: compare, fields.
docs/website/docs/introduction.md and docs/features/features.json: the comparison entry brought to its state.
docs/architecture.md: the Compare satellite's content, over M22's raster seam in the core, where it and AdCodicem.Pdf.Rendering meet.
docs/corpus.md and docs/corpus-contributions.md: the edited sources, and the new wants.
docs/status.md: the budgets and the template calibration.

Goal​

Scope​

Design​

Where it lives​

Comparing text​

Revisions and signatures​

The redline​

Visual comparison​

Templates​

Bounds, classified (invariant 12, ADR 34)​

Diagnostics​

The command-line tool​

Slices​

Tests required​

Acceptance conditions​

Corpus​

What the corpus holds​

What it lacks​

Traps​

Documentation​

Exit criteria​