Skip to content

[py] run grid specific tests on windows during scheduled tests - #18054

Merged
titusfortner merged 2 commits into
SeleniumHQ:trunkfrom
titusfortner:py-grid-nightly
Sep 19, 2026
Merged

titusfortner merged 2 commits into
SeleniumHQ:trunkfrom
titusfortner:py-grid-nightly

Conversation

@titusfortner

Copy link
Copy Markdown
Member

🔗 Related Issues

  • Python counterpart to the Ruby Grid nightly coverage from 6bf7598

💥 What does this PR do?

  • Runs the Python Grid-specific tests (test/selenium/webdriver/remote/) on the scheduled Windows job for chrome and firefox; until now they ran only on RBE.

🔧 Implementation Notes

  • The remote/ directory moves out of the remote-common glob into its own test-<browser>-remote-grid suite with the same -<browser>-remote target suffix, so the per-file targets, the test index, and test-<browser>-remote (still 63 tests for chrome) are unchanged.
  • A test-<browser>-grid aggregate wraps that suite for the nightly matrix; it also includes test-<browser>-remote-bidi when REMOTE_BIDI_TESTS is non-empty, which it currently is not, so a BiDi file opted in later reaches the Windows job without a CI edit.
  • One Grid starts per file, 16 for 19 tests across both browsers; merging them into one target per browser would change the per-file target names, so that is left for later.

🤖 AI assistance

  • AI assisted (complete below)
    • Tool(s): Claude Code (Fable 5.1)
    • What was generated: the CI coverage analysis, the suite wiring, this description
    • I reviewed all AI output and can explain the change

💡 Additional Considerations

🔄 Types of changes

  • Cleanup (formatting, renaming)

@selenium-ci selenium-ci added C-py Python Bindings B-build Includes scripting, bazel and CI integrations labels Sep 18, 2026
@qodo-code-review

Copy link
Copy Markdown
Contributor

PR Summary by Qodo

Run Python Grid-specific suites in Windows nightly CI

🧪 Tests ⚙️ Configuration changes 📝 Documentation 🕐 20-40 Minutes

Grey Divider

AI Description

• Isolates Grid-specific Python tests into Chrome and Firefox Bazel suites.
• Preserves existing remote aggregates and per-file target naming.
• Adds Grid suites to Windows CI and documents local execution.
Diagram

graph TD
  Nightly["Windows Browser CI"] --> Matrix["Grid Matrix"] --> Aggregate["Grid Aggregate"] --> Classic["Remote Grid Suite"] --> Tests["Grid Test Files"] -->|"Remote WebDriver"| Server["Standalone Grid"]
  Aggregate -.->|when configured| Bidi["Remote BiDi Suite"] -->|"Remote WebDriver"| Server
Loading
High-Level Assessment

The following are alternative approaches to this PR:

1. Batch tests per browser
  • ➕ Reduces repeated Selenium Grid startup overhead.
  • ➕ May shorten nightly execution time.
  • ➖ Changes established per-file Bazel target names.
  • ➖ Reduces test isolation and failure granularity.
2. Run full remote suites on Windows
  • ➕ Provides broader Windows remote coverage.
  • ➕ Avoids introducing dedicated Grid aggregates.
  • ➖ Substantially increases nightly execution cost.
  • ➖ Duplicates coverage already provided by Linux RBE.

Recommendation: Keep the scoped Grid aggregates because they add targeted Windows coverage while preserving existing remote targets, per-file naming, and test indexing. Consider browser-level batching later only if repeated Grid startup becomes a material runtime problem.

Files changed (3) +53 / -3

Tests (1) +51 / -2
BUILD.bazelSplit Grid tests into dedicated browser suites +51/-2

Split Grid tests into dedicated browser suites

• Removes webdriver/remote tests from remote-common and generates dedicated Chrome and Firefox remote-grid suites. Preserves the full remote aggregates, adds lightweight Grid aggregates, and conditionally includes opted-in remote BiDi tests.

py/BUILD.bazel

Documentation (1) +1 / -0
TESTING.mdDocument the Grid-only Bazel target +1/-0

Document the Grid-only Bazel target

• Adds an example for running only Chrome's Grid-specific Python tests.

py/TESTING.md

Other (1) +1 / -1
ci-python.ymlAdd Chrome and Firefox Grid jobs to Windows CI +1/-1

Add Chrome and Firefox Grid jobs to Windows CI

• Extends the Windows browser matrix with chrome-grid and firefox-grid entries, invoking the new browser-specific Grid aggregate targets.

.github/workflows/ci-python.yml

@qodo-code-review

qodo-code-review Bot commented Sep 18, 2026 •

Copy link
Copy Markdown
Contributor

Code Review by Qodo

🐞 Bugs (0) 📘 Rule violations (0) 📜 Skill insights (0)

Grey Divider


Action required

1. Two suite comments restate build logic ✓ Resolved 📘 Rule violation ⚙ Maintainability
Description
The comments above the new Grid suite and modified remote aggregate repeat the target names,
supported browsers, and suite composition expressed directly below them. A later change to either
target list must update redundant prose separately, allowing the comments to become stale and
mislead maintainers.
Code

py/BUILD.bazel[1241]

+# Generate test-<browser>-remote-grid targets (chrome and firefox only)
Evidence
Compliance rule 7 reserves comments for rationale or non-obvious context. The cited comments
directly summarize the adjacent generated target and aggregate definitions rather than explaining
why those definitions are necessary.

AGENTS.md: Write Comments That Explain Why Rather Than What
py/BUILD.bazel[1241-1241]
py/BUILD.bazel[1351-1351]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
Two changed comments merely restate the target names, supported browsers, and suite composition already expressed by the adjacent build definitions.

## Fix Focus Areas
- py/BUILD.bazel[1241-1241]
- py/BUILD.bazel[1351-1351]

## Recommended Fix
Remove these two comments, or replace them only if there is non-obvious rationale or a constraint that the build definitions themselves cannot communicate.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


Grey Divider

Context sources
Review mode: ⏭️ Skipped: The latest push only removes comments in a BUILD file and has no behavioral or semantic effect.

Grey Divider

Tip of the day
💡 Did you know, you can describe a rule in plain language on the Rules page and Qodo drafts it for you

More tips ↗ | Customize Qodo ↗ | Qodo docs ↗

Grey Divider

Qodo Logo

Comment thread py/BUILD.bazel Outdated
@titusfortner titusfortner changed the title [py] split the Grid-specific tests into test-<browser>-grid suites and run them on the Windows nightly [py] run grid specific tests on windows during scheduled tests Sep 18, 2026
@qodo-code-review

Copy link
Copy Markdown
Contributor

Code review by qodo was updated up to the latest commit 59f0905

@titusfortner
titusfortner merged commit 3e8b086 into SeleniumHQ:trunk Sep 19, 2026
30 checks passed
This was referenced Oct 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

B-build Includes scripting, bazel and CI integrations C-py Python Bindings

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants