M05 — Repair

State: to do — Depends on: M02, M03, M04

Goal

Turn a damaged document into a sound one, and state precisely what was changed and what was lost.

Scope

In: structural repair — everything needed for a file to open cleanly in any reader and to pass the M02 structural profile.

Out, explicitly: conformance remediation (M21 — embedding missing fonts, adding an output intent or XMP is not repair, it is bringing a sound document up to a standard); content recovery that would require interpreting content streams (M15); anything that invents content that was never there.

Design

Repair is the composition of three things already built: the reader's recovery, the M02 findings, and the M03 writer. It adds the decision layer between them.

PdfRepair.Analyze(document)   -> a plan: one remedy per finding, each with what it will change
PdfRepair.Apply(plan, output) -> the repaired document, plus a PdfRepairReport

Each remedy is attached to the finding that justifies it. A repair nobody asked for is a corruption: if no finding justifies a change, the change does not happen.

Remedies

Finding	Remedy
Cross-reference index wrong or missing	Rebuild it from the objects actually present
Stream length wrong	Recompute from where the data really ends
Object unparseable	Reconstruct it from what the reader recovered when the entries its type requires are there (a dictionary that lost its >>, an object without endobj); otherwise drop it and re-point what referred to it at null — recording each
Reference to a non-existent object	Replace with null, as the specification already requires readers to do
Page tree /Count or /Kids inconsistent	Re-derive the tree from the pages that exist
Page unreachable from the catalog	Re-attach it, in file order, at the end
Trailer missing /Root	Adopt the catalog found by scanning
Trailer /Size missing or wrong	Recompute it from the index, as M03's writer does
/ID missing	Derive one from the content, deterministically
Object defined twice	Keep the last definition, record the discarded one

Modes

• Conservative — change only what a finding justifies, and write an incremental update. The original bytes stay where they were, so anything that depended on them (a signature, an external byte-range reference) survives. This is the default.
• Rebuild — a full rewrite that normalizes the file: objects renumbered, duplicates dropped, index written afresh. Smaller and cleaner output, but every byte offset changes.

The report

PdfRepairReport lists: findings addressed, remedies applied, findings deliberately left alone and why, and what was lost. Loss is never implicit. A page whose content stream could not be recovered is reported as a page with lost content, not quietly emitted blank.

Signed documents, and long operations

• Every remedy names the revision it lands in (M04's revision history). Before writing, conservative mode asks M04's WouldBreakSignatures about its change set: a remedy a certification forbids is refused, typed, unless the caller sets AllowInvalidatingSignatures, and then reported; a remedy that would rewrite a signed revision is never applied in conservative mode — the finding is left alone and the report says why.
• Analyze and Apply follow M03's cancellation and progress convention — IProgress<PdfProgress>, then a CancellationToken checked per object without allocating — and a canceled repair leaves the document as it was.

Slices

1. The plan: findings to remedies, with no writing yet, tested against the damaged corpus.
2. Index, stream length and dangling reference remedies, in rebuild mode.
3. Page tree and orphan-page remedies.
4. Conservative mode with incremental output, the no-op guarantee, and signed documents: the revision each remedy lands in, and M04's write guard.
5. The report, the loss accounting, cancellation and progress, and the budget on the 1000-page journal.

Tests required

Unit

• Each remedy in isolation, on a document that needs exactly it.
• A remedy is never applied without a finding to justify it.
• A repaired document does not need repairing again: repair is idempotent.
• Repairing a document with a signature in conservative mode leaves the signed bytes untouched.
• Every remedy says which revision it lands in (M04), and conservative mode never rewrites a signed revision.
• Hostile: a plan over a document whose every object is damaged, a page tree that is one cycle, a million orphaned pages — each ends in a report or a typed exception, inside time and allocation budgets.
• Cancellation at any check leaves the document usable and a following repair equal to an uninterrupted one.

Integration — each tool in a container (ADR 27): qpdf (--check, --show-npages) on every repaired file; poppler (pdftotext, pdfinfo) for the text and page count compared with the original's; pyHanko and pdfsig for the coverage of every signature after a conservative repair.

Acceptance conditions

Documents	Behavior	Verified by
Every damaged/*	Repairs into a file that opens with no repair needed, has no error-severity finding, and that qpdf --check accepts	CorpusRepairTests.Damaged_documents_repair_into_sound_ones, QpdfRepairRefereeTests.Every_repaired_document_passes_qpdf_check
Every damaged/*	The repaired file matches the undamaged original it was derived from: same page count, and the same text as pdftotext extracts from the original — M15's extraction comes later. The corpus keeps those originals precisely so this can be asserted	CorpusRepairTests.Repaired_documents_match_their_original, PopplerRepairRefereeTests.Repaired_text_is_the_originals
Every well-formed corpus document	Conservative repair is a no-op: byte-identical output, empty report	CorpusRepairTests.Repairing_a_sound_document_changes_nothing
damaged/invoice-truncated-tail.pdf	What is genuinely gone is reported as lost, not silently omitted	CorpusRepairTests.Lost_content_is_reported
Remote: remote/eu-dss/dss-signed-widget-self-parent-startxref-past-eof.pdf (a startxref past the end, its signature update unreachable); to be added (see What it lacks): the signed twin damaged after its signed revision	The remedies land in a new revision; M04's coverage check and pyHanko show every signature still covering exactly the revision it covered before; M05 closes only on a green Remote corpus run	CorpusRepairTests.Repair_never_rewrites_a_signed_revision
The 1000-page journal	Repair holds memory bounded	CorpusRepairTests.Repair_holds_its_memory_budget

Corpus

What the corpus holds

Five copies of the French invoice damaged on purpose by build_corpus.py — no index, shifted offsets, a lost tail, a junk prefix, a lying stream length — each beside the undamaged original it was derived from, which is what lets repair be compared with an original rather than judged by eye. Beside them, 119 damaged documents from other producers and test suites: the iPRES 2017 hand-built set (88, remote), and real-world damage from PDFBox, pdf.js and the OPF format corpus, some with signatures (a DSS-signed widget whose startxref points past the end of the file, a Foxit certification signature with a garbage bounding box).

What it lacks

Need	Why	Priority	Likely source
Third-party damaged documents whose undamaged original is known	"Repaired equals original" can only be asserted on our own damaged copies today; every other damaged document is judged against qpdf alone	1	Generated here: build_corpus.py applying its damage to LibreOffice, ReportLab and vendored documents whose license allows derivatives
A signed document that needs a repair, with its intact twin	Conservative repair of a signed revision is the acceptance row most likely to go wrong	1	Generated here: a pyHanko-signed corpus document, then damaged after the signed revision
A real document damaged in transit (a truncated e-mail attachment, a file cut by a portal's size limit)	The damage users meet most, and the one tools disagree on	2	A contribution

Traps

• Repair is where a library quietly destroys data. Every change must be justified by a finding, recorded in the report, and reversible by not applying it.
• Conservative mode must genuinely not touch the original bytes: an incremental update that rewrites the first section is not incremental.
• A file that cannot be repaired must fail loudly. A half-repaired document handed back as sound is worse than the damaged one, because the caller stops looking.

Documentation

• docs/website/docs/concepts/repair.md — the two modes, what a remedy is, and how loss is reported.
• docs/website/docs/guides/repairing-a-document.md — repairing a received document conservatively or in full, and reading what was changed and what was lost.
• docs/website/docs/introduction.md — repair added to what the library can do.

Exit criteria

• Both modes implemented, with the plan/apply split and the report.
• Signed documents are repaired through M04's revision history and write guard; cancellation and progress follow M03's convention.
• The acceptance conditions above pass on the corpus, in CI, with no document skipped, and the remote row on a green Remote corpus run recorded in docs/status.md.
• Repair is idempotent, verified over the whole damaged corpus.
• Unit tests cover each remedy, its degenerate and its hostile cases.
• A benchmark measures repair of the 1000-page document, with MemoryDiagnoser; docs/status.md records it.
• Integration tests confirm, through qpdf, poppler and pyHanko in containers, that every repaired document is sound, keeps its text and keeps its signatures' coverage.
• The documentation site publishes the repair model, its modes and its guarantees.
• Every page of Documentation is written in its Diátaxis section, one mode per page (ADR 47).