✨ Native Rust voting load preparation, workers and reports (v10) - #3152
Open
edulix wants to merge 70 commits into
Open
✨ Native Rust voting load preparation, workers and reports (v10)#3152edulix wants to merge 70 commits into
edulix wants to merge 70 commits into
Conversation
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
|
Warning Review limit reachedNext included review available in 19 minutes. View limit detailsLimit details: You’ve used the included review currently available. This review ran on the open-source allowance, not this organization's plan, because the pull request author doesn't have an assigned seat. Waiting won't change this — ask an organization admin to assign them a seat, or add seats in Billing if every seat is already assigned, then retry. Review configuration: ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Advanced Run ID: ⛔ Files ignored due to path filters (2)
📒 Files selected for processing (92)
Comment |
|
|
Tip For best results, initiate chat on the files or code changes. 🧩 Analysis chain⚙️ CodeRabbit configuration inspected Length of output: 5503
The usage-priced-review checkbox is account-side state. The configuration diagnostic does not expose its eligibility, billing state, or reset reason. Its reset is therefore not diagnosable from this repository. Please contact CodeRabbit Support with PR Disjoint path-filtered batches are not supported as a reliable way to review all files in one unchanged PR:
Therefore, with the current limit, CodeRabbit cannot complete a source review of all 226 changed files in this single PR. The supported resolution is an account or platform limit change. Splitting the change would also reduce the file count, but that conflicts with your single-PR requirement. 🐇 You are interacting with an AI system. |
edulix
changed the title
edulix
changed the base branch from
release/10.0
to
feat/meta-12767/v10/portal
September 8, 2026 08:40
Contributor
Author
|
@coderabbitai review |
|
Contributor
Author
|
@coderabbitai review --use-credits |
|
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
There are verified security/operational issues (network exposure in a “local” proxy and private directory permissions) plus a toolchain mismatch risk in the worker Dockerfile default Rust version.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
This PR introduces a native (Rust-driven) voting load-testing workflow centered around a new step-cli load command suite, plus a standalone “worker” binary/container that reuses the CLI’s load modules for distributed execution and reporting. It also expands step-cli’s tenant/trustee/document utilities and updates dev tooling to support the new load-testing and diagnostic flows.
Changes:
- Add Rust
step-cli loadorchestration (setup/prepare/run/report), including native ballot encryption, shard ownership, and HTML/SVG reporting assets. - Add a standalone Rust worker build + Docker images (k6 and Chromium variants) that execute finite voter shards with immutable inputs.
- Add optional Python diagnostics (HAR/traffic/SQL log capture) and supporting devcontainer/runtime updates.
File summaries
| File | Description |
|---|---|
| packages/voting-load/worker.rs | Standalone worker entrypoint wiring into shared load modules |
| packages/voting-load/worker.Cargo.toml | Minimal Cargo manifest for standalone worker build |
| packages/voting-load/worker.Cargo.lock.license | REUSE license metadata for worker lockfile |
| packages/voting-load/traffic.py | Endpoint inventory + S3 flow validation helpers |
| packages/voting-load/test_traffic.py | Unit tests for traffic inventory/S3 validation |
| packages/voting-load/test_replay_profile.py | Unit tests for Chromium-to-k6 profile compilation |
| packages/voting-load/test_capture.py | Integration-style tests for capture/reporting/PG log collection |
| packages/voting-load/target.example.json.license | REUSE license metadata for example config |
| packages/voting-load/target.example.json | Example capture/load target configuration |
| packages/voting-load/serve_portal.py | Local SPA-capable static server for built portal |
| packages/voting-load/scale.k6.js | k6 execution script for sharded, finite voter journeys |
| packages/voting-load/resources.py | HAR resource extraction with secret/session scrubbing |
| packages/voting-load/report.html.license | REUSE license metadata for report template |
| packages/voting-load/report.html | HTML report template populated by Rust report generator |
| packages/voting-load/replay.k6.js | Authenticated protocol replay engine (k6) |
| packages/voting-load/replay_profile.py | Compile verified Chromium journey into session-independent recipe |
| packages/voting-load/README.md | Overview of engines, worker build, and diagnostic tooling |
| packages/voting-load/proxy.py | Local diagnostic proxy for action timing/status capture |
| packages/voting-load/measurements.py | Percentiles + SQL/phase summary helpers |
| packages/voting-load/fixtures/election.json.license | REUSE license metadata for fixture export |
| packages/voting-load/Dockerfile.dockerignore | Narrow Docker build context for worker images |
| packages/voting-load/Dockerfile | Multi-stage images for k6 and Chromium workers |
| packages/voting-load/capture.py | One-shot private capture + SQL log collection workflow |
| packages/voting-load/bootstrap.k6.js | One-shot publication bootstrap (no casting) |
| packages/step-cli/src/utils/trustees/mod.rs | Export new trustees utility module |
| packages/step-cli/src/utils/trustees/get.rs | Add trustee (name, public_key) retrieval utility |
| packages/step-cli/src/utils/trustees/get_ceremony_status.rs | Fetch key ceremony execution status via GraphQL |
| packages/step-cli/src/utils/tally/download_document.rs | Add election-event-independent document fetch + plain URL download |
| packages/step-cli/src/utils/read_config.rs | Add config persistence + refresh-token-and-save helper |
| packages/step-cli/src/tests/e2e.rs | Update key ceremony start call signature |
| packages/step-cli/src/main.rs | Add top-level load command and new step subcommands |
| packages/step-cli/src/load/worker.rs | Shared engine execution for CLI + standalone worker |
| packages/step-cli/src/load/reference.rs | Generate Markdown reference from clap + rustdoc |
| packages/step-cli/src/load/provision.rs | Provision/setup orchestration for synthetic load events |
| packages/step-cli/src/load/input.rs | Shared typed input + “wire” JSON boundary for engines |
| packages/step-cli/src/load/image.rs | Build/push worker images from a source-only tar context |
| packages/step-cli/src/load/files.rs | Private, atomic, bounded file I/O helpers |
| packages/step-cli/src/load/executor.rs | Local/Docker/Kubernetes indexed execution and report copying |
| packages/step-cli/src/load/encryption.rs | Native ballot encryption into JSONL shards |
| packages/step-cli/src/load/coordinator.rs | End-to-end lifecycle (check/prepare/screenshot/etc.) |
| packages/step-cli/src/load/census.rs | Streaming voter census generation with shared PBKDF2 hash |
| packages/step-cli/src/graphql/schema.json | GraphQL schema update (e.g., delete_tenant) |
| packages/step-cli/src/graphql/insert_tenant.graphql | Mutation for tenant creation |
| packages/step-cli/src/graphql/import_users.graphql | Mutation for voter import task |
| packages/step-cli/src/graphql/import_tenant_config.graphql | Mutation for tenant-config import task |
| packages/step-cli/src/graphql/import_election_event.graphql | Add task_execution fields to import mutation |
| packages/step-cli/src/graphql/get_trustees.graphql | Include trustee public_key in query |
| packages/step-cli/src/graphql/get_keys_ceremony.graphql | Query ceremony execution_status |
| packages/step-cli/src/graphql/export_tenant_config.graphql | Mutation for tenant-config export task |
| packages/step-cli/src/graphql/delete_tenant.graphql | Mutation for tenant deletion task |
| packages/step-cli/src/graphql/delete_election_event.graphql | Mutation for election-event deletion task |
| packages/step-cli/src/graphql/create_trustee.graphql | Mutation for registering a trustee public key |
| packages/step-cli/src/commands/upload_document.rs | CLI command to upload a document |
| packages/step-cli/src/commands/start_key_ceremony.rs | Add automatic ceremony flag plumbing |
| packages/step-cli/src/commands/refresh_token.rs | Switch to shared refresh_and_save_token helper |
| packages/step-cli/src/commands/mod.rs | Register new step-cli commands |
| packages/step-cli/src/commands/import_voters.rs | CLI command to import voters via async task polling |
| packages/step-cli/src/commands/import_tenant_config.rs | CLI command to import tenant config via async task polling |
| packages/step-cli/src/commands/import_election_event.rs | Wait for async import task completion |
| packages/step-cli/src/commands/get_trustees.rs | CLI command to list trustees (name/public_key) |
| packages/step-cli/src/commands/get_key_ceremony_status.rs | CLI command to print ceremony status |
| packages/step-cli/src/commands/generate_voters.rs | Fix authorized-election-ids to use election IDs; improve alias selection |
| packages/step-cli/src/commands/export_tenant_config.rs | CLI command to export tenant config via async task polling |
| packages/step-cli/src/commands/download_document.rs | CLI command to download a document by ID |
| packages/step-cli/src/commands/delete_tenant.rs | CLI command to delete tenant with task polling + token refresh |
| packages/step-cli/src/commands/delete_election_event.rs | CLI command to delete election event with task polling + token refresh |
| packages/step-cli/src/commands/create_trustee.rs | CLI command to register trustee public key |
| packages/step-cli/src/commands/create_tenant.rs | CLI command to create tenant via async task polling |
| packages/step-cli/scripts/telephone-load-test-inputs/layers.yaml.example | Expanded example config for telephone load scripts |
| packages/step-cli/scripts/telephone-load-test-inputs/election-event.json.license | REUSE license metadata for election-event fixture |
| packages/step-cli/scripts/load_test_common.py | Shared helpers (config loading, HTTP helpers, step-cli runner) |
| packages/step-cli/scripts/dtmf-template.example.txt | Example DTMF template with detailed capture notes |
| packages/step-cli/scripts/cleanup_telephone_load_test.py | Safer, scoped cleanup script for events/tenants |
| packages/step-cli/Cargo.toml | Add deps for load tooling (sqlite, yaml, zip, tls, etc.) |
| packages/step-cli/build.rs | Bundle load assets into the CLI build output |
| packages/Cargo.lock | Lockfile updates for new Rust dependencies |
| devenv.nix | Add k6/chromium/fonts/docker tools and Python deps for diagnostics |
| .gitignore | Ignore load-test outputs and private run artifacts |
| .devcontainer/minio/nginx/default.conf | Preserve host:port for S3 presigned signature verification |
| .devcontainer/keycloak/import/tenant-90505c8a-23a9-4cdf-a26b-4e19f6a097d5.json | Add tenant-delete role to seeded realm |
| .devcontainer/docker-compose-e2e.yml | Overlay for local diagnostic statement/plan logging |
| .devcontainer/.env.development | Add trustee/admin creds and API_KEY_CLIENT_SECRET defaults |
Review details
- Files reviewed: 88/93 changed files
- Comments generated: 3
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| # SPDX-FileCopyrightText: 2026 Sequent Tech Inc <legal@sequentech.io> | ||
| # SPDX-License-Identifier: AGPL-3.0-only | ||
|
|
||
| ARG RUST_IMAGE=rust:1.90-bookworm |
Comment on lines
+18
to
+21
| pub fn generate(input: &Input, output: &Path) -> Result<()> { | ||
| input.validate()?; | ||
| std::fs::DirBuilder::new().create(output)?; | ||
| let start = Instant::now(); |
| with lock, args.log.open("a") as stream: | ||
| stream.write(json.dumps(record) + "\n") | ||
|
|
||
| ThreadingHTTPServer(("0.0.0.0", args.port), Handler).serve_forever() |
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
Several verified issues (toolchain mismatch in worker Docker build, an incorrect dockerignore allowlist, insecure config file permissions, and missing HTTP-status handling in voter import) should be addressed before merging.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Review details
Suppressed comments (1)
packages/voting-load/Dockerfile:6
- The worker build image pins Rust to 1.90, but the repository toolchain is Rust 1.96.0 (rust-toolchain.toml). This can cause the Docker worker build to fail due to MSRV mismatches; align the Dockerfile’s Rust image with the repo toolchain.
- Files reviewed: 88/93 changed files
- Comments generated: 3
- Review effort level: Lite
Comment on lines
+38
to
+47
| pub fn write_config(config_data: &ConfigData) -> Result<PathBuf, Box<dyn Error>> { | ||
| let config_dir = get_config_dir()?; | ||
| if !config_dir.exists() { | ||
| fs::create_dir_all(&config_dir)?; | ||
| } | ||
| let config_file = config_dir.join(CREATE_CONFIG_FILE_NAME); | ||
| let json_data = serde_json::to_string_pretty(config_data)?; | ||
| fs::write(&config_file, json_data)?; | ||
| Ok(config_file) | ||
| } |
Comment on lines
+10
to
+14
| !packages/voting-portal/playwright.scale.config.ts | ||
| !packages/voting-portal/test/ | ||
| !packages/voting-portal/test/load/ | ||
| !packages/voting-portal/test/load/flow.ts | ||
| !packages/voting-portal/test/load/runner.spec.ts |
Comment on lines
+79
to
+87
| let response = client | ||
| .post(&config.endpoint_url) | ||
| .bearer_auth(config.auth_token) | ||
| .json(&request_body) | ||
| .send()?; | ||
|
|
||
| let response_body: Response<import_users::ResponseData> = | ||
| response.json().map_err(|e| format!("{:?}", e))?; | ||
|
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Parent issue: https://github.com/sequentech/meta/issues/12767
Implement
step-cli loadorchestration, shared-hash census generation, native ballot encryption, worker scheduling and SQLite/SVG/HTML reporting in Rust. k6 and Chromium authenticate distinct voters and submit real votes; no browser recording is needed for k6 preparation. Containers compile a small Rust worker from the same ownership code as the CLI.Validation: native unit and command-failure tests; 100-voter k6 and 100-voter Chromium local runs, with all 200 receipts matched against PostgreSQL; local and Docker worker smoke tests. The guide includes the measured results and report screenshot. Kubernetes manifests are tested; a live cluster run is not yet verified.
The CLI runtime is Rust. JavaScript implements the k6/Chromium engine adapters. Standalone telephone setup scripts and optional developer capture/SQL diagnostics remain Python; they are not invoked by
step-cli load.Stack
Documentation