feat(pdf): infer heading levels from PDF bookmarks/ToC

[DanielNg0729]

This PR is the follow-up to #3633 discussed in #3676.

Issue resolved by this Pull Request:
Resolves #3676

What this does

PR #3633 inferred PDF heading levels from numbering and font style, but deliberately left out the document's own bookmarks / table-of-contents. This is usually the most reliable hierarchy signal when a PDF has one. This PR wires bookmarks in as the authoritative source, so the precedence becomes bookmarks (when confidently matched) > numbering > style. Because the chunkers already read hierarchy from the document, the retrieval improvement flows through
automatically.

Approach

• Backend extraction. A shared pypdfium2 helper reads the outline via get_toc(), exposed as PdfDocumentBackend.get_document_outline(). Since both the pypdfium2 and the default docling-parse backends hold a PDFium handle, it's implemented once on their shared base, so the signal is available under the default backend too. The outline is surfaced on ConversionResult.pdf_outline (excluded from serialization, so the final DoclingDocument isn't bloated).
• Matching
  • Bookmark titles often differ from the on-page text, so matching is fuzzy (stdlib difflib), comparing titles with and without their leading numbering marker, with a containment boost for truncated bookmarks, gated by a page constraint and a configurable threshold.
  • Headings the layout model gets wrong are often classified as list-items, so the matcher also considers list-items and promotes a confidently matched one to a SectionHeaderItem at the bookmark's level.
• Safety. Only confidently matched entries act; partial/noisy outlines never degrade the feat(pdf): infer PDF heading levels so the hierarchy isn't flattened #3633 numbering result, unmatched headings fall back to numbering/style. PDF-only; OCR/VLM PDFs without an embedded outline simply lack the signal.

Configuration

This PR adds two options to HeadingHierarchyOptions; no existing options change behavior.

Option	Type	Default	What it controls
use_bookmarks	bool	True	New. Use PDF bookmarks/ToC as the authoritative heading signal when present. Set to False to keep the previous #3633 behavior (numbering + style only).
bookmark_match_threshold	float (0–1)	0.8	New. Minimum fuzzy title similarity for a bookmark to match a detected heading/list-item. Higher = stricter; below it the bookmark is ignored and the heading falls back to numbering/style.

These sit alongside the existing options (unchanged): enabled (still off by default — must be True for any inference to run), use_numbering, use_style, numbering_schemes, max_level.

Precedence when enabled: bookmarks (confidently matched) > numbering > style.

Usage

from docling.datamodel.base_models import InputFormat
from docling.datamodel.pipeline_options import HeadingHierarchyOptions, PdfPipelineOptions
from docling.document_converter import DocumentConverter, PdfFormatOption

pipeline_options = PdfPipelineOptions()
pipeline_options.heading_hierarchy_options = HeadingHierarchyOptions(
    enabled=True,                  # turn heading-level inference on (off by default)
    use_bookmarks=True,            # new: PDF bookmarks/ToC as the authoritative signal
    bookmark_match_threshold=0.8,  # new: fuzzy title-match strictness (0..1)
)

converter = DocumentConverter(
    format_options={InputFormat.PDF: PdfFormatOption(pipeline_options=pipeline_options)}
)
result = converter.convert("contract.pdf")
print(result.document.export_to_markdown())

Notes / scope

• Richer provenance metadata suggested in the discussion (heading_source, bookmark_path, match confidence) would need a docling-core schema change so i would happy to open a separate issue for it so it doesn't block this pipeline change.
• Known limitation: a promoted heading keeps its original tree position, so it may sit inside its former list group. This is fine for hierarchy/chunking; can be refined in a follow-up.

Thanks for the guidance on this Dr @PeterStaar-IBM . I'm happy to adjust anything!

Checklist:

• Documentation has been updated, if necessary. (option docstrings)
• Examples have been added, if necessary.
• Tests have been added, if necessary. (tests/test_heading_hierarchy_bookmarks.py)

[github-actions]

✅ DCO Check Passed

Thanks @DanielNg0729, all your commits are properly signed off. 🎉

[mergify]

Merge Protections

🟢 All 2 merge protections satisfied — ready to merge.

Show 2 satisfied protections

🟢 Enforce conventional commit

Make sure that we follow https://www.conventionalcommits.org/en/v1.0.0/

• title ~= ^(fix|feat|docs|style|refactor|perf|test|build|ci|chore|revert)(?:\(.+\))?(!)?:

🟢 Require two reviewer for test updates

When test data is updated, we require two reviewers

• #approved-reviews-by >= 2

[codecov]

Codecov Report

❌ Patch coverage is 87.43169% with 23 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
docling/backend/docling_parse_backend.py	33.33%	10 Missing ⚠️
docling/utils/pdf_outline.py	90.76%	6 Missing ⚠️
...tages/heading_hierarchy/heading_hierarchy_model.py	94.11%	5 Missing ⚠️
docling/backend/pdf_backend.py	66.66%	1 Missing ⚠️
docling/backend/pypdfium2_backend.py	80.00%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!

[PeterStaar-IBM]

@DanielNg0729 Love the work: I would suggest that we add at least 1 example pdf with an existing outline in the tests.

[DanielNg0729]

Thanks, Dr. @PeterStaar-IBM! I've added a committed sample PDF (tests/data/pdf/bookmark_sample.pdf) with a real, nested outline (PART → section → subsection across 3 pages), plus a test (test_backend_extracts_nested_outline_from_sample_pdf) that runs PdfDocumentBackend.get_document_outline() against it and asserts the full bookmark tree (title/level/page) and vertical positions.

[cau-git]

@DanielNg0729 great to see this continuation PR. I have not reviewed deeply yet, however one feedback already below:

• You are using pypdfium to retrieve the Table of Contents, for both the PyPdfiumDocumentBackend and the DoclingParseDocumentBackend (since internally it also owns a pypdfium handle). We would like the DoclingParseDocumentBackend to handle this with the native way to get the table-of-contents (no pypdfium dependency). You can find the hints here. Could you please check if that gives you the matching information needed? Thanks.
• If the above is addressed, then we should be able to use it also with the (non-default) ThreadedDoclingParseDocumentBackend, which has no pypdfium handle.

[DanielNg0729]

Thank you very much for the feedback Dr @cau-git

I've implemented it as requested: DoclingParseDocumentBackend (and the v2/v4 + threaded variants) now read the outline via docling-parse's native get_table_of_contents(), with no pypdfium2 dependency. I also added a sample PDF + tests covering both extractors. Thanks again!

[DanielNg0729]

Hi Dr. @cau-git, Dr. @PeterStaar-IBM

Quick update on the failing run-tests-ml (pdf-model) lane: the root cause was that docling-parse's get_table_of_contents() returns None for PDFs without an embedded outline, so the extractor crashed on None.children during heading-hierarchy conversion. I've fixed it to return an empty outline in that case (falling back to numbering/style as before) and added a regression test. Lint, type checks, and the core tests are green locally.

Could you kindly re-trigger the CI when you have a moment? Thank you very much!

[PeterStaar-IBM]

Hi Dr. @cau-git, Dr. @PeterStaar-IBM

Quick update on the failing run-tests-ml (pdf-model) lane: the root cause was that docling-parse's get_table_of_contents() returns None for PDFs without an embedded outline, so the extractor crashed on None.children during heading-hierarchy conversion. I've fixed it to return an empty outline in that case (falling back to numbering/style as before) and added a regression test. Lint, type checks, and the core tests are green locally.

Could you kindly re-trigger the CI when you have a moment? Thank you very much!

of course!

[PeterStaar-IBM]

@cau-git Let's review this one internally!

[cau-git]

@DanielNg0729 Thanks for the updates! I went through the code and posted a few remarks and questions below.

[PeterStaar-IBM]

lgtm!

[hadrien]

Thanks @DanielNg0729 and reviewing team.

[PierreMesure]

This is amazing work @DanielNg0729! Looking forward to trying it on our documents after the Summer! 😊