Skip to content

Improve documentation site layout#1578

Open
timsaucer wants to merge 11 commits into
apache:mainfrom
timsaucer:doc/phase1-theme-refresh
Open

Improve documentation site layout#1578
timsaucer wants to merge 11 commits into
apache:mainfrom
timsaucer:doc/phase1-theme-refresh

Conversation

@timsaucer
Copy link
Copy Markdown
Member

@timsaucer timsaucer commented Jun 5, 2026

Which issue does this PR close?

There is no open issue, but based on the recent work our sibling project Comet did to improve their site.

Rationale for this change

This is phase 1 of a 4 phase project to improve documentation. Here we are updating the site dependencies and the overall layout. This is the biggest visual win of the phases. The follow on work includes:

  • Switching from rst files to markdown. This is primarily because agents will be reading these files and agents have been highly improved to focus on markdown over other formats.
  • Setting up for multi-version builds. As the feature set has been growing from version to version, it will be useful for end users to select their version to find appropriate functions.
  • Updating the CI system to push to the asf-site branch like Comet does.

What changes are included in this PR?

  • Update dependencies for site documentation
  • Configure the modern theme in docs/source/conf.py
  • Add dark-mode variants for the inline-code accent color and the table-striped row shading so both render with adequate contrast.
  • Add site icon

Are there any user-facing changes?

Yes — purely visual to the published documentation site. No API changes, no behavioral changes to the datafusion package itself. Changes a reader will notice:

  • A top navigation bar with the project logo, the user-guide / contributor-guide links, a GitHub icon, and a light/dark theme switcher.
  • Dark mode now actually renders correctly (the old overrides assumed a single light theme).
  • The site is built using a newer Sphinx theme; published HTML will look meaningfully different.
Screenshot 2026-06-05 at 9 33 55 AM

Bump pydata-sphinx-theme 0.8.0 -> 0.16 to enable the modern navbar slot
API and dark/light theme switcher. Configure top navbar with logo,
nav links, GitHub icon, and theme switcher in conf.py. Drop the custom
docs-sidebar.html override and the layout.html block that silenced the
navbar — both predate the slot API and conflict with the new theme.
Strip CSS overrides that fought the old theme (--pst-header-height: 0,
navbar-brand sizing) and add a dark-mode variant for the inline code
color and table-stripe shading. Fix the stale github_repo
("arrow-datafusion-python" -> "datafusion-python") so future Edit-on-
GitHub links resolve. Bump copyright year and project name.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@timsaucer timsaucer marked this pull request as draft June 5, 2026 12:51
@timsaucer timsaucer changed the title docs: refresh theme — pydata-sphinx-theme 0.16, top navbar, dark mode Improve documentation site layout based on Comet improvements Jun 5, 2026
timsaucer and others added 9 commits June 5, 2026 08:53
Previous structure dumped every top-level toctree entry from index.rst
into the navbar, producing eight items including external URLs ("Github
and Issue Tracker", "Rust's API Docs", ...) that wrapped to two lines
each. Introduce user-guide/index.rst and contributor-guide/index.rst as
section landing pages with nested toctrees, then point index.rst at just
those two plus autoapi/index. The navbar now reads "User Guide",
"Contributor Guide", "API Reference" — three single-line entries. Move
the external links into the index.rst body where they're discoverable
without crowding navigation.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add Examples and Rust API as text links in the top navbar via the
pydata-sphinx-theme external_links option. Nest the code-of-conduct
link inside the Contributor Guide toctree so it appears alongside the
other contributor pages. Drop the duplicate "Further reading" bullet
list from the landing page now that every link has a permanent home.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Move the Rust API docs entry from external_links to icon_links and use
the fa-brands fa-rust gear mark. Now sits next to the GitHub icon in
navbar_end with matching visual weight instead of a wider text link.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The default pydata-sphinx-theme sidebar-nav-bs starts at the current
top-level section, so the root index — which has no parent section —
ends up with an empty sidebar. The theme's layout also explicitly
filters sidebar-nav-bs out of the sidebar list when suppress_sidebar_
toctree() returns true (which it does for root pages), so simply
overriding sidebar-nav-bs.html in templates doesn't help.

Add a sidebar-globaltoc.html template that calls Sphinx's toctree()
global directly to render the full document tree, and wire it through
html_sidebars under a name the theme's suppress filter doesn't strip.
Landing page now shows User Guide / Contributor Guide / API Reference
in the sidebar with the current section expanded on inner pages.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Switch the sidebar toctree call from toctree() to generate_toctree_html
with collapse=False, so nested <ul>s render into the DOM for every
branch. The pydata-sphinx-theme JS then wraps them in <details> with
fa-chevron-down toggles, matching the datafusion-comet sidebar where
each section with children can be expanded inline. show_nav_level=1
keeps deeper levels collapsed on first load.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Bump show_nav_level 1 -> 2 so the landing-page sidebar opens with
User Guide / Contributor Guide / API Reference already expanded to
their immediate children. Deeper levels remain collapsed behind
chevrons so the sidebar stays scannable.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Restore the "Links" sidebar heading that the previous site had —
GitHub and Issue Tracker, Rust API Docs, Code of Conduct, Examples.
Implemented as a second hidden toctree with :caption: Links so the
pydata-sphinx-theme sidebar renders the heading above the four
external URLs. Drop Code of Conduct from the Contributor Guide
toctree since it now lives under Links instead.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replace the second hidden toctree (which expanded each external URL
into its own navbar entry) with a dedicated links.rst landing page,
and add a single "links" entry to the main toctree. Top navbar now
shows User Guide / Contributor Guide / API Reference / Links — four
items, no wrapping. Clicking Links opens the page that lists GitHub,
Rust API Docs, Code of Conduct, and Examples.

Drop the external_links Examples entry from conf.py since the same
URL now lives on the Links page.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Drop in the same favicon.svg the main datafusion.apache.org site
uses (just the Apache DataFusion mark, no wordmark) and wire it
through html_favicon. Browsers and bookmarks now show the project
icon instead of the generic Sphinx page glyph.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@timsaucer timsaucer marked this pull request as ready for review June 5, 2026 13:36
@timsaucer timsaucer requested review from andygrove and Copilot June 5, 2026 13:36
@timsaucer timsaucer changed the title Improve documentation site layout based on Comet improvements Improve documentation site layout Jun 5, 2026
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Phase 1 of a documentation-site refresh, updating Sphinx theme/dependencies and restructuring the documentation TOC/layout to match newer styling patterns (including dark mode support and a site icon).

Changes:

  • Updated docs build dependencies (notably pydata-sphinx-theme to 0.16.x) and locked them in uv.lock.
  • Reworked the docs IA/TOC by introducing section index pages (User Guide / Contributor Guide) and simplifying the root index.rst tree.
  • Added theme/layout customizations: navbar configuration, global sidebar template, dark-mode CSS tweaks, and an SVG favicon.

Reviewed changes

Copilot reviewed 10 out of 12 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
uv.lock Locks updated docs dependencies (pydata theme upgrade + new transitive deps).
pyproject.toml Updates docs extras to use newer pydata-sphinx-theme and adds sphinx-reredirects.
docs/source/index.rst Simplifies root toctree to point at section index pages + links.
docs/source/user-guide/index.rst New section landing page to collect user-guide toctree items.
docs/source/contributor-guide/index.rst New section landing page to collect contributor-guide toctree items.
docs/source/links.rst New page collecting external links under a single toctree entry.
docs/source/conf.py Switches to modern pydata theme config (navbar/logo/icons/sidebar config) + favicon.
docs/source/_templates/sidebar-globaltoc.html New sidebar template rendering the full global toctree with collapsible behavior.
docs/source/_templates/layout.html Stops silencing the navbar (inherits theme navbar again) while keeping custom footer.
docs/source/_templates/docs-sidebar.html Removes the old custom sidebar template implementation.
docs/source/_static/theme_overrides.css Updates theme overrides for dark-mode code/table stripe contrast and removes obsolete sidebar sizing rules.
docs/source/_static/favicon.svg Adds site favicon asset.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread docs/source/conf.py
Comment thread docs/source/_templates/sidebar-globaltoc.html Outdated
Two small follow-ups from the Copilot reviewer on apache#1578:

- Append .html to the html_sidebars entry. Sphinx's Jinja loader
  resolves both "sidebar-globaltoc" and "sidebar-globaltoc.html" to
  the same template, but the explicit form is closer to the spelling
  in the Sphinx docs and is harder to misread.
- Update the inline comment in sidebar-globaltoc.html that still
  claimed show_nav_level=1 after we bumped it to 2 in conf.py. Now
  describes the variable wiring instead of hard-coding a number that
  has to be kept in sync with conf.py.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants