Repository navigation
fix: broken ci test - #1339
fix: broken ci test#1339
Conversation
WalkthroughIntroduce a shared validators.Manager across RPC and endpoints, add Redis-based validator lifecycle publishing on channel Changes
Sequence Diagram(s)sequenceDiagram
participant RPC as RPC Endpoint
participant Manager as validators.Manager
participant Registry as ModifiableValidatorsRegistry
participant Redis as Redis Pub/Sub
participant Worker as Consensus Worker
RPC->>Manager: create/update/delete (injected dependency)
Manager->>Registry: registry.create/update/delete(...)
Registry->>Registry: DB commit
Manager->>Redis: publish {"event":..., "data":...} on validator:events
Redis-->>Worker: validator:events message
Worker->>Manager: subscribed callback invoked → restart() validators
Note right of Worker: background listener runs in redis handler
Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes
Possibly related PRs
Pre-merge checks and finishing touches❌ Failed checks (1 warning, 1 inconclusive)
✅ Passed checks (1 passed)
✨ Finishing touches🧪 Generate unit tests (beta)
📜 Recent review detailsConfiguration used: CodeRabbit UI Review profile: CHILL Plan: Pro 📒 Files selected for processing (1)
🚧 Files skipped from review as they are similar to previous changes (1)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
…so consensus sees updated validators snapshot
|
There was a problem hiding this comment.
Actionable comments posted: 1
♻️ Duplicate comments (2)
backend/validators/__init__.py (1)
95-126: Validator registry interceptor correctly notifies on post-commit mutations, but relies on per-call Redis connectionsThe interceptor wiring (create/update/delete/delete_all) correctly wraps mutations in
do_write(), commits the session, and only then calls_notify_validator_change(...), which is exactly what you want for side-effectful notifications that must not affect DB semantics.The remaining concern is that
_notify_validator_changecurrently opens and closes a fresh Redis client on each call viaredis.asyncio.from_url(...). Under higher validator churn this adds unnecessary connection overhead and risk of exhaustion compared to reusing a single async client onManagerand closing it interminate(). This was already raised in a previous review; consider centralizing the client on the manager and switching this code toawait self._redis_client.publish(...)..github/workflows/backend_integration_tests_pr.yml (1)
53-57: Make .env overrides idempotent and align CPU limits with runner capacityThe sed replacements here still assume the keys already exist in
.envand hard-codeCOMPOSE_CPU_LIMIT="4":sed -i "s/VITE_FINALITY_WINDOW=\".*\"/VITE_FINALITY_WINDOW=\"10\"/" .env sed -i "s/COMPOSE_CPU_LIMIT=\".*\"/COMPOSE_CPU_LIMIT=\"4\"/" .env ...Two follow-ups (previously raised) are still worth considering:
- Make the overrides “replace or append” so they work even if the keys are missing from
.env.example.- Reduce
COMPOSE_CPU_LIMITto match the 2 vCPUs onubuntu-latestrunners to avoid oversubscription.A small shell loop that does “if key exists, replace; else append” would make this block more robust, and lowering CPU limit keeps resources aligned with the runner.
🧹 Nitpick comments (1)
tests/unit/test_simulator_sessions.py (1)
176-299: Tests correctly covervalidators_managerwiring; consider aligning fakes and cleaning unused argsThe updated tests for
create_validator,create_random_validators, andupdate_validatornow pass avalidators_manager = SimpleNamespace(registry=registry_instance)and assert againstregistry_instancecalls. That’s a good, focused way to verify that endpoint logic uses the injected manager/registry rather than constructing its own instances.Two small cleanups you may want to make:
fake_check_provider_is_available(Lines 275-276) no longer matches the realcheck_provider_is_available(validators_manager, provider)signature and its parameters are unused. Updating it toasync def fake_check_provider_is_available(_validators_manager, _provider): ...will both mirror the actual API and satisfy Ruff’s ARG001.- Similarly, where
monkeypatchis unused (e.g., intest_delete_all_validators_uses_request_session), renaming to_monkeypatchor dropping the parameter will silence the warning.Behavior is fine as-is; these are just small maintenance wins.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (8)
.github/workflows/backend_integration_tests_pr.yml(2 hunks)backend/consensus/worker_service.py(1 hunks)backend/protocol_rpc/endpoints.py(5 hunks)backend/protocol_rpc/rpc_methods.py(3 hunks)backend/validators/__init__.py(3 hunks)backend/validators/greyboxing.lua(1 hunks)docker-compose.yml(3 hunks)tests/unit/test_simulator_sessions.py(7 hunks)
🚧 Files skipped from review as they are similar to previous changes (3)
- backend/validators/greyboxing.lua
- docker-compose.yml
- backend/consensus/worker_service.py
🧰 Additional context used
📓 Path-based instructions (3)
**/*.py
📄 CodeRabbit inference engine (AGENTS.md)
Target Python 3.12, use 4-space indentation, and rely on Black (via pre-commit) for formatting
Files:
backend/validators/__init__.pybackend/protocol_rpc/endpoints.pybackend/protocol_rpc/rpc_methods.pytests/unit/test_simulator_sessions.py
tests/{unit,integration,e2e,load}/**
📄 CodeRabbit inference engine (AGENTS.md)
Organize Pytest suites by scope under tests/{unit,integration,e2e,load}
Files:
tests/unit/test_simulator_sessions.py
tests/{unit,integration,e2e,load}/test_*.py
📄 CodeRabbit inference engine (AGENTS.md)
Name backend tests as test_.py in the nearest scope folder
Files:
tests/unit/test_simulator_sessions.py
🧠 Learnings (1)
📚 Learning: 2025-09-30T10:34:33.826Z
Learnt from: CR
Repo: genlayerlabs/genlayer-studio PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-09-30T10:34:33.826Z
Learning: Applies to docker-compose*.yml : Use docker-compose*.yml files for composing services
Applied to files:
.github/workflows/backend_integration_tests_pr.yml
🧬 Code graph analysis (4)
backend/validators/__init__.py (4)
backend/protocol_rpc/endpoints.py (2)
delete_validator(336-345)delete_all_validators(349-353)backend/protocol_rpc/rpc_methods.py (2)
delete_validator(183-190)delete_all_validators(194-197)backend/database_handler/validators_registry.py (2)
delete_validator(92-96)delete_all_validators(98-100)backend/protocol_rpc/message_handler/redis_worker_handler.py (1)
close(234-239)
backend/protocol_rpc/endpoints.py (3)
backend/validators/__init__.py (5)
Manager(142-378)create_validator(95-100)update_validator(102-110)delete_validator(112-119)delete_all_validators(121-126)backend/protocol_rpc/rpc_methods.py (5)
create_validator(103-122)update_validator(160-179)delete_validator(183-190)delete_all_validators(194-197)get_all_validators(201-204)backend/database_handler/validators_registry.py (5)
create_validator(70-74)update_validator(76-90)delete_validator(92-96)delete_all_validators(98-100)get_all_validators(51-57)
backend/protocol_rpc/rpc_methods.py (3)
backend/protocol_rpc/dependencies.py (1)
get_validators_manager(121-126)backend/protocol_rpc/endpoints.py (4)
create_validator(196-232)update_validator(295-332)delete_validator(336-345)delete_all_validators(349-353)backend/validators/__init__.py (4)
create_validator(95-100)update_validator(102-110)delete_validator(112-119)delete_all_validators(121-126)
tests/unit/test_simulator_sessions.py (2)
backend/protocol_rpc/endpoints.py (3)
delete_validator(336-345)delete_all_validators(349-353)get_all_validators(356-357)backend/protocol_rpc/rpc_methods.py (3)
delete_validator(183-190)delete_all_validators(194-197)get_all_validators(201-204)
🪛 Ruff (0.14.5)
backend/validators/__init__.py
377-377: Do not catch blind exception: Exception
(BLE001)
378-378: Use logging.exception instead of logging.error
Replace with exception
(TRY400)
backend/protocol_rpc/rpc_methods.py
111-111: Do not perform function call Depends in argument defaults; instead, perform the call within the function, or read the default from a module-level singleton variable
(B008)
168-168: Do not perform function call Depends in argument defaults; instead, perform the call within the function, or read the default from a module-level singleton variable
(B008)
185-185: Do not perform function call Depends in argument defaults; instead, perform the call within the function, or read the default from a module-level singleton variable
(B008)
195-195: Do not perform function call Depends in argument defaults; instead, perform the call within the function, or read the default from a module-level singleton variable
(B008)
tests/unit/test_simulator_sessions.py
275-275: Unused function argument: provider
(ARG001)
275-275: Unused function argument: model
(ARG001)
351-351: Unused function argument: monkeypatch
(ARG001)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)
- GitHub Check: load-test
- GitHub Check: test
- GitHub Check: backend-unit-tests
🔇 Additional comments (2)
backend/protocol_rpc/endpoints.py (1)
196-353: Validator CRUD endpoints now correctly route throughvalidators_manager.registryThe refactor to inject
validators_managerand delegate all validator mutations tovalidators_manager.registryin:
create_validator(Lines 196-233),create_random_validatorandcreate_random_validators(Lines 235-291),update_validator(Lines 295-333),delete_validatoranddelete_all_validators(Lines 335-353),looks solid:
- Mutations go through
ModifiableValidatorsRegistryInterceptor, so they benefit from thedo_write()locking, snapshot refresh, and Redis-based change notifications added inbackend.validators.Manager.- You still use the request-scoped
Sessionfor account creation and provider registry access, preserving per-request DB scoping.- After
delete_all_validators, the follow-upget_all_validators()call will see an empty set due tosession.expire_all()in the registry, avoiding accidental leakage of any previous state.This wiring matches the PR goal of centralizing validator lifecycle under the shared manager.
backend/protocol_rpc/rpc_methods.py (1)
102-198: RPC layer correctly propagatesvalidators_managerto validator endpointsFor
sim_createValidator,sim_updateValidator,sim_deleteValidator, andsim_deleteAllValidators, the addedvalidators_manager=Depends(get_validators_manager)parameter and corresponding forwarding intoimpl.*ensure that all validator mutations now go through the sharedvalidators.Managerinstance.This keeps the FastAPI surface aligned with the new manager-based implementation in
endpoints.pyand guarantees that snapshots and Redis notifications are triggered consistently from RPC calls. Ruff’s B008 warnings here are expected for FastAPI DI patterns and can be ignored.
| async def _notify_validator_change(self, event_type: str, data: dict): | ||
| """ | ||
| Notify other services about validator changes via Redis. | ||
| This is only called by RPC service (not consensus-worker). | ||
| """ | ||
| import json | ||
| import os | ||
| import redis.asyncio as aioredis | ||
|
|
||
| # Get Redis URL from environment | ||
| redis_url = os.environ.get("REDIS_URL", "redis://redis:6379/0") | ||
|
|
||
| try: | ||
| # Create Redis client for publishing | ||
| redis_client = aioredis.from_url( | ||
| redis_url, encoding="utf-8", decode_responses=True | ||
| ) | ||
|
|
||
| # Prepare message | ||
| message = json.dumps( | ||
| { | ||
| "event": event_type, | ||
| "data": data, | ||
| } | ||
| ) | ||
|
|
||
| # Publish to validator events channel for consensus-worker | ||
| subscribers = await redis_client.publish("validator:events", message) | ||
|
|
||
| # Close the client | ||
| await redis_client.close() | ||
|
|
||
| logger.info( | ||
| f"Published validator change event: {event_type} to {subscribers} subscribers" | ||
| ) | ||
| except Exception as e: | ||
| logger.error(f"Failed to publish validator change event: {e}") |
There was a problem hiding this comment.
Improve Redis notifier robustness and logging (avoid blind Exception, prefer logger.exception)
Catching a bare Exception and logging via logger.error loses the stack trace and trips Ruff BLE001/TRY400. Since this notifier is already best-effort and should never break validator CRUD, you can keep failures non-fatal but still tighten things up:
- Prefer
logger.exception("Failed to publish validator change event")to capture the traceback. - Optionally narrow the
exceptto Redis-specific errors (e.g.redis.exceptions.RedisError) if you want to avoid suppressing unrelated programming errors.
This keeps behavior the same (no re-raise) while improving diagnosability and satisfying the linter.
🧰 Tools
🪛 Ruff (0.14.5)
377-377: Do not catch blind exception: Exception
(BLE001)
378-378: Use logging.exception instead of logging.error
Replace with exception
(TRY400)
🤖 Prompt for AI Agents
In backend/validators/__init__.py around lines 342 to 378, the current code
catches a bare Exception and logs with logger.error which loses traceback and
trips linters; change the except block to catch Redis-specific errors (e.g.,
except aioredis.exceptions.RedisError as e: or except aioredis.RedisError as e:)
and replace logger.error with logger.exception("Failed to publish validator
change event") so the traceback is recorded; do not re-raise so notifier remains
best-effort, and allow non-Redis exceptions to surface (so programming errors
aren't silently swallowed).
There was a problem hiding this comment.
Actionable comments posted: 1
♻️ Duplicate comments (1)
.github/workflows/backend_integration_tests_pr.yml (1)
52-55: Guard OPENAIKEY substitution and reduce CPU limits to fit runner capacity.Line 52 attempts to substitute OPENAIKEY even on forks where the secret is unavailable, which will inject an empty placeholder. Additionally,
COMPOSE_RPC_CPU_LIMITandCOMPOSE_WORKER_CPU_LIMITare both set to "4" (lines 54–55), exceeding the GitHub runner's 2 vCPU capacity and risking resource exhaustion.Apply this diff:
- name: Set OPENAIKEY in the .env file so it can be loaded from the environment env: OPENAIKEY: ${{ secrets.OPENAIKEY }} run: | - sed -i "s/<add_your_openai_api_key_here>/${OPENAIKEY}/g" .env - sed -i "s/VITE_FINALITY_WINDOW=\".*\"/VITE_FINALITY_WINDOW=\"10\"/" .env - sed -i "s/COMPOSE_RPC_CPU_LIMIT=\".*\"/COMPOSE_RPC_CPU_LIMIT=\"4\"/" .env - sed -i "s/COMPOSE_WORKER_CPU_LIMIT=\".*\"/COMPOSE_WORKER_CPU_LIMIT=\"4\"/" .env + if [ -n "$OPENAIKEY" ]; then + sed -i "s/<add_your_openai_api_key_here>/${OPENAIKEY}/g" .env + fi + sed -i "s/VITE_FINALITY_WINDOW=\".*\"/VITE_FINALITY_WINDOW=\"10\"/" .env + sed -i "s/COMPOSE_RPC_CPU_LIMIT=\".*\"/COMPOSE_RPC_CPU_LIMIT=\"2\"/" .env + sed -i "s/COMPOSE_WORKER_CPU_LIMIT=\".*\"/COMPOSE_WORKER_CPU_LIMIT=\"2\"/" .env
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
.github/workflows/backend_integration_tests_pr.yml(2 hunks)
🧰 Additional context used
🧠 Learnings (1)
📚 Learning: 2025-09-30T10:34:33.826Z
Learnt from: CR
Repo: genlayerlabs/genlayer-studio PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-09-30T10:34:33.826Z
Learning: Applies to docker-compose*.yml : Use docker-compose*.yml files for composing services
Applied to files:
.github/workflows/backend_integration_tests_pr.yml
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)
- GitHub Check: load-test
- GitHub Check: test
- GitHub Check: backend-unit-tests
🔇 Additional comments (1)
.github/workflows/backend_integration_tests_pr.yml (1)
81-81: Build and deployconsensus-workerservice.The addition of
consensus-workerto both the Docker Buildx bake targets (line 81) and the compose up command (line 89) is consistent and aligns with the PR's objective to deploy the consensus-worker for validator change event handling.Also applies to: 89-89
| sed -i "s/COMPOSE_RPC_MEM_RESERVATION=\".*\"/COMPOSE_RPC_MEM_RESERVATION=\"1\"/" .env | ||
| sed -i "s/COMPOSE_WORKER_MEM_RESERVATION=\".*\"/COMPOSE_WORKER_MEM_RESERVATION=\"1\"/" .env | ||
| sed -i "s/COMPOSE_RPC_MEM_LIMIT=\".*\"/COMPOSE_RPC_MEM_LIMIT=\"6gb\"/" .env | ||
| sed -i "s/COMPOSE_WORKER_MEM_LIMIT=\".*\"/COMPOSE_WORKER_MEM_LIMIT=\"6gb\"/" .env | ||
| sed -i "s/COMPOSE_RPC_MEM_RESERVATION=\".*\"/COMPOSE_RPC_MEM_RESERVATION=\"2gb\"/" .env | ||
| sed -i "s/COMPOSE_WORKER_MEM_RESERVATION=\".*\"/COMPOSE_WORKER_MEM_RESERVATION=\"2gb\"/" .env |
There was a problem hiding this comment.
Resolve conflicting memory reservation values and make .env updates idempotent.
Lines 56–57 set COMPOSE_RPC_MEM_RESERVATION and COMPOSE_WORKER_MEM_RESERVATION to "1", but lines 60–61 immediately overwrite them to "2gb". This is logically conflicting and suggests a copy-paste error. Additionally, the sed pattern only updates existing keys; if a key is absent from .env, it won't be added, leaving variables unset during CI.
Apply this diff to remove the duplicate/conflicting lines and make the configuration idempotent:
sed -i "s/VITE_FINALITY_WINDOW=\".*\"/VITE_FINALITY_WINDOW=\"10\"/" .env
sed -i "s/COMPOSE_RPC_CPU_LIMIT=\".*\"/COMPOSE_RPC_CPU_LIMIT=\"2\"/" .env
sed -i "s/COMPOSE_WORKER_CPU_LIMIT=\".*\"/COMPOSE_WORKER_CPU_LIMIT=\"2\"/" .env
- sed -i "s/COMPOSE_RPC_MEM_RESERVATION=\".*\"/COMPOSE_RPC_MEM_RESERVATION=\"1\"/" .env
- sed -i "s/COMPOSE_WORKER_MEM_RESERVATION=\".*\"/COMPOSE_WORKER_MEM_RESERVATION=\"1\"/" .env
sed -i "s/COMPOSE_RPC_MEM_LIMIT=\".*\"/COMPOSE_RPC_MEM_LIMIT=\"6gb\"/" .env
sed -i "s/COMPOSE_WORKER_MEM_LIMIT=\".*\"/COMPOSE_WORKER_MEM_LIMIT=\"6gb\"/" .env
- sed -i "s/COMPOSE_RPC_MEM_RESERVATION=\".*\"/COMPOSE_RPC_MEM_RESERVATION=\"2gb\"/" .env
- sed -i "s/COMPOSE_WORKER_MEM_RESERVATION=\".*\"/COMPOSE_WORKER_MEM_RESERVATION=\"2gb\"/" .env
+ sed -i "s/COMPOSE_RPC_MEM_RESERVATION=\".*\"/COMPOSE_RPC_MEM_RESERVATION=\"2gb\"/" .env
+ sed -i "s/COMPOSE_WORKER_MEM_RESERVATION=\".*\"/COMPOSE_WORKER_MEM_RESERVATION=\"2gb\"/" .env
+
+ # Make updates idempotent: append if key is missing
+ for key in VITE_FINALITY_WINDOW COMPOSE_RPC_CPU_LIMIT COMPOSE_WORKER_CPU_LIMIT COMPOSE_RPC_MEM_RESERVATION COMPOSE_WORKER_MEM_RESERVATION COMPOSE_RPC_MEM_LIMIT COMPOSE_WORKER_MEM_LIMIT; do
+ if ! grep -q "^${key}=" .env; then
+ echo "${key}=\"\"" >> .env
+ fi
+ doneAlternatively, refactor to a loop with replace-or-append logic for all variables at once (cleaner but more complex).
Committable suggestion skipped: line range outside the PR's diff.
🤖 Prompt for AI Agents
In .github/workflows/backend_integration_tests_pr.yml around lines 56 to 61,
there are duplicate/conflicting sed updates that first set
COMPOSE_RPC_MEM_RESERVATION/COMPOSE_WORKER_MEM_RESERVATION to "1" and then
overwrite them to "2gb" (likely a copy/paste error) and the sed commands only
replace existing keys (they won't add missing keys). Remove the
duplicate/conflicting lines so each variable is set once to the intended value,
and replace the per-key sed calls with idempotent replace-or-append logic (or a
small loop) that updates the key if present and appends KEY="value" if missing
so the .env changes are deterministic and safe for CI.
|
|
🎉 This PR is included in version 0.81.0 🎉 The release is available on GitHub release Your semantic-release bot 📦🚀 |




Fixes #DXP-699
What
session.expire_all()calls to validators registry to ensure fresh database readsWhy
Testing done
Decisions made
Checks
Reviewing tips
User facing release notes
Summary by CodeRabbit
New Features
Bug Fixes
Chores
Tests
✏️ Tip: You can customize this high-level summary in your review settings.