Skip to content

chore: rollback genvm version - #1412

Merged
cristiam86 merged 11 commits into
mainfrom
rollback-genvm-version
Jan 29, 2026
Merged

cristiam86 merged 11 commits into
mainfrom
rollback-genvm-version

Conversation

@cristiam86

@cristiam86 cristiam86 commented Jan 29, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #N/A

What

Rolled back GenVM version to 0.2.11

Summary by CodeRabbit

Release Notes

  • Bug Fixes

    • Improved error handling and timeout detection for system operations; errors are now more robustly categorized and handled.
  • Improvements

    • Enhanced system monitoring and error reporting capabilities with optional integration support.
  • Configuration

    • Removed GENVM_TAG environment variable from service configuration.
    • Extended timeout duration for error transaction processing to improve reliability.

✏️ Tip: You can customize this high-level summary in your review settings.

@coderabbitai

coderabbitai Bot commented Jan 29, 2026 •

Copy link
Copy Markdown
Contributor
📝 Walkthrough

Walkthrough

The PR removes deprecated agent logging infrastructure, introduces structured GenVM error handling through a new GenVMInternalError exception type, adds optional Sentry integration, adjusts timeout configurations, and updates related tests to align with the new error handling behavior.

Changes

Cohort / File(s) Summary
GenVM Error Handling
backend/node/genvm/base.py, backend/node/base.py
Added GenVMInternalError to public exports; implemented detailed error parsing with cause extraction; enhanced _run_genvm to handle GenVMInternalError distinctly for LEADER (re-raise with restart) vs VALIDATOR (construct synthetic error Receipt with TIMEOUT vote) modes; modified _set_vote to broaden timeout detection logic.
Logging Removal
backend/consensus/worker_service.py, tests/unit/test_node_execution_time.py
Removed deprecated _agent_log helper function and all NDJSON logging call sites; added optional Sentry initialization blocks; updated test mock for time.time() to remove consumed value previously used by _agent_log.
Configuration & Scripts
docker-compose.yml, scripts/run_update_error_transactions.sh
Removed GENVM_TAG environment variable from jsonrpc service; extended pod completion success timeout (3600s → 86400s) and adjusted failure timeout (10s → 60s).

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

Suggested labels

released

Poem

🐰 The logs have faded into the night,
GenVM errors now shine bright,
With Sentry watching from on high,
Timeouts extended to the sky—
A cleaner path to errors clear! 🔍

🚥 Pre-merge checks | ❌ 3
❌ Failed checks (3 warnings)
Check name Status Explanation Resolution
Description check ⚠️ Warning The description is largely incomplete, providing only a brief statement about rolling back GenVM without addressing most required template sections. Complete the PR description by adding 'Why' (reason for rollback), 'Testing done', 'Decisions made', and other relevant sections from the template.
Docstring Coverage ⚠️ Warning Docstring coverage is 71.79% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
Title check ⚠️ Warning The PR title 'chore: rollback genvm version' is vague and misleading given the actual changes. While the PR description mentions rolling back GenVM to 0.2.11, the changeset includes significant functional modifications: removal of agent logging, new error handling for GenVMInternalError, and Sentry integration—far beyond a simple version rollback. Revise the title to accurately reflect the scope of changes, such as 'refactor: improve genvm error handling and remove agent logging' or break this into multiple focused PRs with descriptive titles.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing touches
  • 📝 Generate docstrings

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 6

🤖 Fix all issues with AI agents
In `@frontend/src/components/Simulator/TransactionItem.vue`:
- Around line 425-448: The template currently assumes history.leader_result has
an element; change the guard to ensure the array is non-empty before indexing
(e.g., update the v-if from history?.leader_result to
history?.leader_result?.length > 0) and also null-check nested accesses used in
the block (references to history.leader_result[0].node_config.address and
history.leader_result[0].node_config.primary_model) so they only render when
node_config and primary_model exist.

In `@scripts/dump_transactions_to_metrics.py`:
- Around line 167-199: The extract_execution_result function should mirror the
guards used in update_error_transactions_metrics.py: before calling .get on
first_receipt, add a type check (isinstance(first_receipt, dict)) to avoid
AttributeError if it's a non-dict; also before falling back to leader_receipt
extraction, check consensus_history (tx.get("consensus_history")) and if present
inspect consensus_history.get("consensus_results", []) and if its last element's
"consensus_round" == "Undetermined" return "undetermined" to match the other
script's logic; update the code paths in extract_execution_result to perform
these checks (use symbols: extract_execution_result, consensus_data,
leader_receipt, first_receipt, consensus_history, consensus_results,
consensus_round) and keep the existing timeout/default behavior.

In `@scripts/update_error_transactions_metrics.py`:
- Around line 188-195: Update the docstring for the generator that currently
claims it yields (batch, scanned_count) to correctly state it yields (batch,
matched_count); specifically change any occurrences of "scanned_count" to
"matched_count" in the triple-quoted docstring above the generator that yields
(batch, matched_count) (and make the same change in the other docstring instance
around lines 250-252) so the documentation matches the actual returned tuple
values (batch, matched_count).
- Around line 57-74: Add explicit type hints to the new helper and streaming
functions: annotate log(message: str) -> None; ensure format_duration(seconds:
float) -> str remains typed; annotate get_total_transaction_count(conn:
Connection) -> int (use the appropriate DB connection type, e.g.,
sqlite3.Connection or your project's DB Connection type) and annotate
fetch_error_transactions(conn: Connection, *, chunk_size: int = ...) ->
Iterator[Mapping[str, Any]] (or another suitable iterable type for your
transaction records). Import needed typing symbols (Iterator, Mapping, Any) and
the concrete Connection type used in the project, and apply the same pattern to
any streaming API functions mentioned elsewhere (e.g., other fetch_* helpers) so
all parameters and return values are explicitly typed.
- Line 656: The print call using an unnecessary f-string prefix should be
changed to a normal string; locate the print statement in
scripts/update_error_transactions_metrics.py where the line reads print(f"\n 
Failed transaction hashes (first 10):", flush=True) and remove the leading f so
it becomes print("\n  Failed transaction hashes (first 10):", flush=True).
Ensure no other placeholders are needed in that call and run tests/lint to
verify the f-prefix removal.
- Around line 445-480: The current generic except Exception block in the retry
loop should be narrowed to only network-related failures: replace "except
Exception as e:" with "except requests.RequestException as e:" so only requests
errors (including Timeout) are retried and programmer errors are not swallowed;
ensure the import of requests remains and keep the existing requests.Timeout
except before the RequestException handler to preserve specific timeout handling
in the function that posts to f"{api_url}/api/ingest".
🧹 Nitpick comments (1)
backend/node/base.py (1)

911-916: Remove extraneous f prefix from string without placeholders.

The f-string on line 912 has no placeholders.

Suggested fix
             self.logger.warning(
-                f"Validator encountered GenVMInternalError",
+                "Validator encountered GenVMInternalError",
                 error_code=e.error_code,
                 causes=e.causes,
                 is_fatal=e.is_fatal,
             )

Comment on lines 425 to +448
<div
v-if="history?.leader_result"
class="flex flex-row items-center justify-between p-2 text-xs dark:border-gray-600"
>
<div class="flex items-center gap-1">
<UserPen class="h-4 w-4" />
<span class="font-mono text-xs">{{
history.leader_result[0].node_config.address
}}</span>
<div class="flex flex-col gap-0.5">
<div class="flex items-center gap-1">
<UserPen class="h-4 w-4" />
<span class="font-mono text-xs">{{
history.leader_result[0].node_config.address
}}</span>
</div>
<div
v-if="history.leader_result[0].node_config.primary_model"
class="ml-5 text-[10px] text-gray-500 dark:text-gray-400"
>
{{
history.leader_result[0].node_config.primary_model
.provider
}}
/
{{
history.leader_result[0].node_config.primary_model.model
}}
</div>

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Potential runtime error when leader_result is an empty array.

The guard v-if="history?.leader_result" passes when leader_result is [] (empty arrays are truthy). Accessing history.leader_result[0].node_config.address would then throw a TypeError since [0] returns undefined.

🐛 Proposed fix to check array length
-            v-if="history?.leader_result"
+            v-if="history?.leader_result?.length"
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
<div
v-if="history?.leader_result"
class="flex flex-row items-center justify-between p-2 text-xs dark:border-gray-600"
>
<div class="flex items-center gap-1">
<UserPen class="h-4 w-4" />
<span class="font-mono text-xs">{{
history.leader_result[0].node_config.address
}}</span>
<div class="flex flex-col gap-0.5">
<div class="flex items-center gap-1">
<UserPen class="h-4 w-4" />
<span class="font-mono text-xs">{{
history.leader_result[0].node_config.address
}}</span>
</div>
<div
v-if="history.leader_result[0].node_config.primary_model"
class="ml-5 text-[10px] text-gray-500 dark:text-gray-400"
>
{{
history.leader_result[0].node_config.primary_model
.provider
}}
/
{{
history.leader_result[0].node_config.primary_model.model
}}
</div>
<div
v-if="history?.leader_result?.length"
class="flex flex-row items-center justify-between p-2 text-xs dark:border-gray-600"
>
<div class="flex flex-col gap-0.5">
<div class="flex items-center gap-1">
<UserPen class="h-4 w-4" />
<span class="font-mono text-xs">{{
history.leader_result[0].node_config.address
}}</span>
</div>
<div
v-if="history.leader_result[0].node_config.primary_model"
class="ml-5 text-[10px] text-gray-500 dark:text-gray-400"
>
{{
history.leader_result[0].node_config.primary_model
.provider
}}
/
{{
history.leader_result[0].node_config.primary_model.model
}}
</div>
🤖 Prompt for AI Agents
In `@frontend/src/components/Simulator/TransactionItem.vue` around lines 425 -
448, The template currently assumes history.leader_result has an element; change
the guard to ensure the array is non-empty before indexing (e.g., update the
v-if from history?.leader_result to history?.leader_result?.length > 0) and also
null-check nested accesses used in the block (references to
history.leader_result[0].node_config.address and
history.leader_result[0].node_config.primary_model) so they only render when
node_config and primary_model exist.

Comment on lines +167 to +199
def extract_execution_result(tx: dict) -> str:
"""
Extract execution result from status and consensus_data.

Returns one of: "success", "error", "timeout", "undetermined"

Priority:
1. Original status timeout (LEADER_TIMEOUT, VALIDATORS_TIMEOUT) -> "timeout"
2. Original status undetermined (UNDETERMINED) -> "undetermined"
3. execution_result from consensus_data.leader_receipt[0] -> "success" or "error"
4. Default -> "success"
"""
# Check original status for timeout/undetermined
original_status = tx.get("status")
if original_status in ("LEADER_TIMEOUT", "VALIDATORS_TIMEOUT"):
return "timeout"
if original_status == "UNDETERMINED":
return "undetermined"

# Try to get execution_result from leader receipt in consensus_data
consensus_data = tx.get("consensus_data")
if consensus_data is not None:
leader_receipts = consensus_data.get("leader_receipt", [])
if leader_receipts and len(leader_receipts) > 0:
first_receipt = leader_receipts[0]
if first_receipt is not None:
execution_result = first_receipt.get("execution_result")
if execution_result is not None:
# Handle string value
return str(execution_result).lower()

# Default to success
return "success"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Inconsistency with update_error_transactions_metrics.py and missing type guard.

Two issues:

  1. Missing isinstance check: Line 192 assumes first_receipt is a dict, but if it's another type (e.g., string), calling .get() will raise AttributeError. The similar function in update_error_transactions_metrics.py guards against this at line 309.

  2. Missing consensus_history check: The equivalent function in update_error_transactions_metrics.py (lines 297-303) checks consensus_history.consensus_results[-1].consensus_round == "Undetermined" before falling through to leader receipt extraction. This script lacks that check, which could produce inconsistent results for the same transaction.

Proposed fix to add type guard and align logic
     # Try to get execution_result from leader receipt in consensus_data
     consensus_data = tx.get("consensus_data")
+    consensus_history = tx.get("consensus_history")
+
+    # Check if consensus was undetermined from consensus_history
+    if (
+        consensus_history is not None
+        and "consensus_results" in consensus_history
+        and len(consensus_history["consensus_results"]) > 0
+    ):
+        last_round = consensus_history["consensus_results"][-1]
+        if last_round.get("consensus_round") == "Undetermined":
+            return "undetermined"
+
     if consensus_data is not None:
         leader_receipts = consensus_data.get("leader_receipt", [])
         if leader_receipts and len(leader_receipts) > 0:
             first_receipt = leader_receipts[0]
-            if first_receipt is not None:
+            if first_receipt is not None and isinstance(first_receipt, dict):
                 execution_result = first_receipt.get("execution_result")
                 if execution_result is not None:
                     # Handle string value
                     return str(execution_result).lower()
🤖 Prompt for AI Agents
In `@scripts/dump_transactions_to_metrics.py` around lines 167 - 199, The
extract_execution_result function should mirror the guards used in
update_error_transactions_metrics.py: before calling .get on first_receipt, add
a type check (isinstance(first_receipt, dict)) to avoid AttributeError if it's a
non-dict; also before falling back to leader_receipt extraction, check
consensus_history (tx.get("consensus_history")) and if present inspect
consensus_history.get("consensus_results", []) and if its last element's
"consensus_round" == "Undetermined" return "undetermined" to match the other
script's logic; update the code paths in extract_execution_result to perform
these checks (use symbols: extract_execution_result, consensus_data,
leader_receipt, first_receipt, consensus_history, consensus_results,
consensus_round) and keep the existing timeout/default behavior.

Comment on lines +57 to +74
def log(message: str):
"""Print message with timestamp and flush immediately."""
timestamp = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
print(f"[{timestamp}] {message}", flush=True)


def format_duration(seconds: float) -> str:
"""Format duration in human-readable format."""
if seconds < 60:
return f"{seconds:.1f}s"
elif seconds < 3600:
minutes = int(seconds // 60)
secs = int(seconds % 60)
return f"{minutes}m {secs}s"
else:
hours = int(seconds // 3600)
minutes = int((seconds % 3600) // 60)
return f"{hours}h {minutes}m"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion | 🟠 Major

Add missing type hints for new helpers and streaming API.

log, get_total_transaction_count, and fetch_error_transactions are missing required type hints on return/connection types. This breaks the project’s typing guideline.

🔧 Proposed fix
-from typing import Optional
+from typing import Optional, Iterator
+from psycopg2.extensions import connection as PGConnection
@@
-def log(message: str):
+def log(message: str) -> None:
@@
-def get_total_transaction_count(
-    conn,
+def get_total_transaction_count(
+    conn: PGConnection,
@@
-def fetch_error_transactions(
-    conn,
+def fetch_error_transactions(
+    conn: PGConnection,
@@
-) :
+) -> Iterator[tuple[list[dict], int]]:

As per coding guidelines: Include type hints in all Python code.

Also applies to: 113-117, 180-187

🤖 Prompt for AI Agents
In `@scripts/update_error_transactions_metrics.py` around lines 57 - 74, Add
explicit type hints to the new helper and streaming functions: annotate
log(message: str) -> None; ensure format_duration(seconds: float) -> str remains
typed; annotate get_total_transaction_count(conn: Connection) -> int (use the
appropriate DB connection type, e.g., sqlite3.Connection or your project's DB
Connection type) and annotate fetch_error_transactions(conn: Connection, *,
chunk_size: int = ...) -> Iterator[Mapping[str, Any]] (or another suitable
iterable type for your transaction records). Import needed typing symbols
(Iterator, Mapping, Any) and the concrete Connection type used in the project,
and apply the same pattern to any streaming API functions mentioned elsewhere
(e.g., other fetch_* helpers) so all parameters and return values are explicitly
typed.

Comment on lines 188 to 195
"""
Fetch finalized transactions that have error results or undetermined consensus.

Yields batches of transaction dicts.
Uses a server-side cursor with Python-side filtering for memory efficiency.
The SQL query is simple and fast, filtering is done in Python.

Yields tuples of (batch, scanned_count) where batch is a list of transaction dicts.
"""

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Fix docstring mismatch with yielded value.

The docstring says (batch, scanned_count) but the generator yields (batch, matched_count). Align the docstring to avoid confusion.

✏️ Proposed fix
-    Yields tuples of (batch, scanned_count) where batch is a list of transaction dicts.
+    Yields tuples of (batch, matched_count) where batch is a list of transaction dicts.

Also applies to: 250-252

🤖 Prompt for AI Agents
In `@scripts/update_error_transactions_metrics.py` around lines 188 - 195, Update
the docstring for the generator that currently claims it yields (batch,
scanned_count) to correctly state it yields (batch, matched_count); specifically
change any occurrences of "scanned_count" to "matched_count" in the
triple-quoted docstring above the generator that yields (batch, matched_count)
(and make the same change in the other docstring instance around lines 250-252)
so the documentation matches the actual returned tuple values (batch,
matched_count).

Comment on lines +445 to +480
for attempt in range(1, max_retries + 1):
try:
response = requests.post(
f"{api_url}/api/ingest",
json=payload,
headers=headers,
timeout=60, # Increased timeout for large batches
)

if response.status_code == 200:
return True, ""

if response.status_code != 200:
print(
f"Warning: API returned status {response.status_code}: {response.text[:200]}"
error_msg = (
f"API returned status {response.status_code}: {response.text[:200]}"
)
return False
if attempt < max_retries:
log(f" Retry {attempt}/{max_retries}: {error_msg}")
time.sleep(2**attempt) # Exponential backoff
else:
return False, error_msg

return True
except requests.Timeout:
print("Warning: API request timed out")
return False
except Exception as e:
print(f"Error sending to API: {e}")
return False
except requests.Timeout:
error_msg = "API request timed out"
if attempt < max_retries:
log(f" Retry {attempt}/{max_retries}: {error_msg}")
time.sleep(2**attempt)
else:
return False, error_msg

except Exception as e:
error_msg = f"Error: {e}"
if attempt < max_retries:
log(f" Retry {attempt}/{max_retries}: {error_msg}")
time.sleep(2**attempt)
else:
return False, error_msg

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

🧩 Analysis chain

🌐 Web query:

requests.exceptions.RequestException hierarchy Timeout subclass

💡 Result:

In requests, Timeout is a subclass of RequestException, and it has these main subclasses:

OSError (aka IOError)
└── requests.exceptions.RequestException
    └── requests.exceptions.Timeout
        ├── requests.exceptions.ReadTimeout
        └── requests.exceptions.ConnectTimeout  (also a ConnectionError)

Key detail: ConnectTimeout uses multiple inheritance:

ConnectTimeout(ConnectionError, Timeout)

So catching requests.exceptions.Timeout will catch both ConnectTimeout and ReadTimeout. [1]

Source: Requests exceptions module (class definitions) [1]

[1] https://requests.readthedocs.io/en/master/_modules/requests/exceptions/


🏁 Script executed:

#!/bin/bash
# Check if file exists and examine the lines in question
if [ -f "scripts/update_error_transactions_metrics.py" ]; then
  echo "=== File exists, checking lines 445-480 ==="
  sed -n '445,480p' scripts/update_error_transactions_metrics.py
else
  echo "File not found, searching for similar files..."
  find . -name "*update_error_transactions*" -o -name "*metrics*" | head -20
fi

Repository: genlayerlabs/genlayer-studio

Length of output: 1387


🏁 Script executed:

#!/bin/bash
# Check the function definition and see if it has type hints
sed -n '430,450p' scripts/update_error_transactions_metrics.py

Repository: genlayerlabs/genlayer-studio

Length of output: 653


Replace broad Exception with requests.RequestException to avoid hiding programmer errors.

Catching Exception masks non-network errors, making failures harder to debug. Since requests.Timeout is already a subclass of requests.RequestException, catching requests.RequestException instead will only retry network-related issues while allowing programmer errors to surface.

Proposed fix
-        except Exception as e:
-            error_msg = f"Error: {e}"
+        except requests.RequestException as e:
+            error_msg = f"Request error: {e}"
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
for attempt in range(1, max_retries + 1):
try:
response = requests.post(
f"{api_url}/api/ingest",
json=payload,
headers=headers,
timeout=60, # Increased timeout for large batches
)
if response.status_code == 200:
return True, ""
if response.status_code != 200:
print(
f"Warning: API returned status {response.status_code}: {response.text[:200]}"
error_msg = (
f"API returned status {response.status_code}: {response.text[:200]}"
)
return False
if attempt < max_retries:
log(f" Retry {attempt}/{max_retries}: {error_msg}")
time.sleep(2**attempt) # Exponential backoff
else:
return False, error_msg
return True
except requests.Timeout:
print("Warning: API request timed out")
return False
except Exception as e:
print(f"Error sending to API: {e}")
return False
except requests.Timeout:
error_msg = "API request timed out"
if attempt < max_retries:
log(f" Retry {attempt}/{max_retries}: {error_msg}")
time.sleep(2**attempt)
else:
return False, error_msg
except Exception as e:
error_msg = f"Error: {e}"
if attempt < max_retries:
log(f" Retry {attempt}/{max_retries}: {error_msg}")
time.sleep(2**attempt)
else:
return False, error_msg
for attempt in range(1, max_retries + 1):
try:
response = requests.post(
f"{api_url}/api/ingest",
json=payload,
headers=headers,
timeout=60, # Increased timeout for large batches
)
if response.status_code == 200:
return True, ""
error_msg = (
f"API returned status {response.status_code}: {response.text[:200]}"
)
if attempt < max_retries:
log(f" Retry {attempt}/{max_retries}: {error_msg}")
time.sleep(2**attempt) # Exponential backoff
else:
return False, error_msg
except requests.Timeout:
error_msg = "API request timed out"
if attempt < max_retries:
log(f" Retry {attempt}/{max_retries}: {error_msg}")
time.sleep(2**attempt)
else:
return False, error_msg
except requests.RequestException as e:
error_msg = f"Request error: {e}"
if attempt < max_retries:
log(f" Retry {attempt}/{max_retries}: {error_msg}")
time.sleep(2**attempt)
else:
return False, error_msg
🧰 Tools
🪛 Ruff (0.14.14)

474-474: Do not catch blind exception: Exception

(BLE001)

🤖 Prompt for AI Agents
In `@scripts/update_error_transactions_metrics.py` around lines 445 - 480, The
current generic except Exception block in the retry loop should be narrowed to
only network-related failures: replace "except Exception as e:" with "except
requests.RequestException as e:" so only requests errors (including Timeout) are
retried and programmer errors are not swallowed; ensure the import of requests
remains and keep the existing requests.Timeout except before the
RequestException handler to preserve specific timeout handling in the function
that posts to f"{api_url}/api/ingest".

print(f" Success rate: {success_rate:.1f}%", flush=True)

if failed_hashes:
print(f"\n Failed transaction hashes (first 10):", flush=True)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Remove unused f-string prefix.

This f-string has no placeholders.

✏️ Proposed fix
-            print(f"\n  Failed transaction hashes (first 10):", flush=True)
+            print("\n  Failed transaction hashes (first 10):", flush=True)
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
print(f"\n Failed transaction hashes (first 10):", flush=True)
print("\n Failed transaction hashes (first 10):", flush=True)
🧰 Tools
🪛 Ruff (0.14.14)

656-656: f-string without any placeholders

Remove extraneous f prefix

(F541)

🤖 Prompt for AI Agents
In `@scripts/update_error_transactions_metrics.py` at line 656, The print call
using an unnecessary f-string prefix should be changed to a normal string;
locate the print statement in scripts/update_error_transactions_metrics.py where
the line reads print(f"\n  Failed transaction hashes (first 10):", flush=True)
and remove the leading f so it becomes print("\n  Failed transaction hashes
(first 10):", flush=True). Ensure no other placeholders are needed in that call
and run tests/lint to verify the f-prefix removal.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Fix all issues with AI agents
In `@backend/consensus/worker_service.py`:
- Around line 401-419: SENTRY_DSN should have an explicit type hint and the
Sentry initialization should use environment-driven, conservative defaults
instead of hardcoded aggressive settings: add a type hint for SENTRY_DSN (e.g.,
Optional[str]) and read traces_sample_rate, profile_session_sample_rate, and
send_default_pii from env vars with safe fallbacks (e.g., traces_sample_rate
default 0.1, profile_session_sample_rate default 0.0, send_default_pii default
False), parse/validate the values (ensure sampling rates are floats in [0.0,1.0]
and booleans for PII), and only pass
profile_session_sample_rate/profile_lifecycle to sentry_sdk.init if explicitly
configured; reference SENTRY_DSN, sentry_sdk.init, traces_sample_rate,
profile_session_sample_rate, profile_lifecycle, and send_default_pii when
implementing these changes.

In `@backend/node/base.py`:
- Around line 836-846: The warning call is using an unnecessary f-string and the
error_message is double-prefixing the GenVMInternalError text; update the
logger.warning invocation in the exception handler (logger.warning) to use a
plain string literal (not f"...") and keep the structured fields (error_code,
causes, is_fatal), and change the construction of error_message (used with
public_abi.ResultCode.VM_ERROR and the bytes prefix) to use the exception text
directly (e.g., str(e) or e.message) without adding "GenVM internal error:" so
the final payload does not repeat the prefix.
🧹 Nitpick comments (1)
backend/consensus/worker_service.py (1)

401-402: Add a type hint for SENTRY_DSN.

Use an explicit annotation to satisfy the “type hints in all Python code” guideline.

♻️ Proposed tweak
-SENTRY_DSN = os.getenv("SENTRY_DSN", None)
+SENTRY_DSN: str | None = os.getenv("SENTRY_DSN")

As per coding guidelines, include type hints in all Python code.

Comment on lines +401 to +419
SENTRY_DSN = os.getenv("SENTRY_DSN", None)
if SENTRY_DSN:
import sentry_sdk

sentry_sdk.init(
dsn=SENTRY_DSN,
# Add data like request headers and IP for users,
# see https://docs.sentry.io/platforms/python/data-management/data-collected/ for more info
send_default_pii=True,
# Set traces_sample_rate to 1.0 to capture 100%
# of transactions for tracing.
traces_sample_rate=1.0,
# Set profile_session_sample_rate to 1.0 to profile 100%
# of profile sessions.
profile_session_sample_rate=1.0,
# Set profile_lifecycle to "trace" to automatically
# run the profiler on when there is an active transaction
profile_lifecycle="trace",
)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🏁 Script executed:

cd backend/consensus && head -n 430 worker_service.py | tail -n 35

Repository: genlayerlabs/genlayer-studio

Length of output: 1185


🌐 Web query:

sentry-sdk 2.50.0 Python profiling profile_session_sample_rate profile_lifecycle configuration

💡 Result:

In sentry-sdk (sentry-python) 2.50.0, Continuous/UI Profiling is configured via two top-level sentry_sdk.init(...) options:

  • profile_session_sample_rate: float | None

    • Value between 0.0 and 1.0.
    • Controls the chance that the current process/session is profiled.
    • The sampling decision is evaluated once at SDK initialization (so it won’t change until the process restarts). [1][2]
  • profile_lifecycle: "manual" | "trace" (default: "manual")

    • "manual": you explicitly start/stop profiling (and it will only do anything if the session was sampled by profile_session_sample_rate). [1][2]
    • "trace": profiler auto-starts when there is at least one active root span/transaction, and auto-stops when there are none; requires both:
      • the session is sampled (profile_session_sample_rate)
      • and tracing is sampled (traces_sample_rate or traces_sampler). [1][2]

Example configurations

A) Continuous profiling, controlled by tracing lifecycle

import sentry_sdk

sentry_sdk.init(
    dsn="https://examplePublicKey@o0.ingest.sentry.io/0",
    traces_sample_rate=0.2,
    profile_session_sample_rate=0.1,
    profile_lifecycle="trace",
)

(Options and allowed values are documented in the SDK options/API.) [1][3]

B) Continuous profiling with fully manual control

import sentry_sdk
from sentry_sdk import profiler

sentry_sdk.init(
    dsn="https://examplePublicKey@o0.ingest.sentry.io/0",
    profile_session_sample_rate=0.1,
    profile_lifecycle="manual",
)

profiler.start_profiler()
# ... work ...
profiler.stop_profiler()

(Manual vs trace behavior is defined in the Python options docs.) [1]

Important precedence note

If you still have transaction-based profiling configured (profiles_sample_rate / profiles_sampler), it can take precedence; Sentry recommends migrating to profile_session_sample_rate for Continuous/UI Profiling. [2]

Sources: Sentry Python options docs (profile lifecycle + session sampling) [1], Sentry Profiling FAQ (lifecycle + sampling + precedence) [2], sentry-python 2.50.0 API reference (init parameters) [3].


Add type hint and configure Sentry sampling/PII defaults safely.

The SENTRY_DSN variable lacks a type hint (violates **/*.py guideline requiring type hints). Additionally, the hardcoded aggressive defaults—100% tracing (traces_sample_rate=1.0), 100% profiling (profile_session_sample_rate=1.0), and PII enabled—create production compliance and cost risks. Both profile_session_sample_rate and profile_lifecycle are valid sentry-sdk 2.50.0 options, but safer defaults and environment-driven overrides are recommended.

🔧 Suggested type hint and config hardening
-SENTRY_DSN = os.getenv("SENTRY_DSN", None)
+SENTRY_DSN: str | None = os.getenv("SENTRY_DSN", None)
 if SENTRY_DSN:
     import sentry_sdk

     sentry_sdk.init(
         dsn=SENTRY_DSN,
         # Add data like request headers and IP for users,
         # see https://docs.sentry.io/platforms/python/data-management/data-collected/ for more info
-        send_default_pii=True,
+        send_default_pii=os.getenv("SENTRY_SEND_DEFAULT_PII", "false").lower() == "true",
         # Set traces_sample_rate to 1.0 to capture 100%
         # of transactions for tracing.
-        traces_sample_rate=1.0,
+        traces_sample_rate=float(os.getenv("SENTRY_TRACES_SAMPLE_RATE", "0.1")),
         # Set profile_session_sample_rate to 1.0 to profile 100%
         # of profile sessions.
-        profile_session_sample_rate=1.0,
+        profile_session_sample_rate=float(
+            os.getenv("SENTRY_PROFILE_SESSION_SAMPLE_RATE", "0.0")
+        ),
         # Set profile_lifecycle to "trace" to automatically
         # run the profiler on when there is an active transaction
-        profile_lifecycle="trace",
+        profile_lifecycle=os.getenv("SENTRY_PROFILE_LIFECYCLE", "manual"),
     )
🤖 Prompt for AI Agents
In `@backend/consensus/worker_service.py` around lines 401 - 419, SENTRY_DSN
should have an explicit type hint and the Sentry initialization should use
environment-driven, conservative defaults instead of hardcoded aggressive
settings: add a type hint for SENTRY_DSN (e.g., Optional[str]) and read
traces_sample_rate, profile_session_sample_rate, and send_default_pii from env
vars with safe fallbacks (e.g., traces_sample_rate default 0.1,
profile_session_sample_rate default 0.0, send_default_pii default False),
parse/validate the values (ensure sampling rates are floats in [0.0,1.0] and
booleans for PII), and only pass profile_session_sample_rate/profile_lifecycle
to sentry_sdk.init if explicitly configured; reference SENTRY_DSN,
sentry_sdk.init, traces_sample_rate, profile_session_sample_rate,
profile_lifecycle, and send_default_pii when implementing these changes.

Comment thread backend/node/base.py
Comment on lines +836 to +846
self.logger.warning(
f"Validator encountered GenVMInternalError",
error_code=e.error_code,
causes=e.causes,
is_fatal=e.is_fatal,
)
processing_time = int((time.time() - start_time) * 1000)
error_message = f"GenVM internal error: {e}"
error_result = bytes(
[public_abi.ResultCode.VM_ERROR]
) + error_message.encode("utf-8")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

🧩 Analysis chain

🏁 Script executed:

# First, check if the file exists and read the specific lines
wc -l backend/node/base.py

Repository: genlayerlabs/genlayer-studio

Length of output: 96


🏁 Script executed:

# Read lines 836-846 from the file
sed -n '830,850p' backend/node/base.py

Repository: genlayerlabs/genlayer-studio

Length of output: 906


🏁 Script executed:

# Run ruff check as suggested in the review
ruff check backend/node/base.py --select F541

Repository: genlayerlabs/genlayer-studio

Length of output: 3097


🏁 Script executed:

# Search for GenVMInternalError definition to understand its __str__ behavior
rg "class GenVMInternalError" -A 20

Repository: genlayerlabs/genlayer-studio

Length of output: 1482


🏁 Script executed:

# Get more of the GenVMInternalError class definition to see __str__ method
rg "class GenVMInternalError" -A 50 backend/node/genvm/error_codes.py

Repository: genlayerlabs/genlayer-studio

Length of output: 1696


🏁 Script executed:

# Also check if there's a __str__ method defined
rg "__str__" backend/node/genvm/error_codes.py -B 2 -A 5

Repository: genlayerlabs/genlayer-studio

Length of output: 54


🏁 Script executed:

# Search for where GenVMInternalError is raised/instantiated
rg "raise GenVMInternalError|GenVMInternalError(" backend/node/ | head -20

Repository: genlayerlabs/genlayer-studio

Length of output: 210


🏁 Script executed:

# More specifically, search for where the error message is constructed
rg "GenVM internal error" backend/node/ -B 2 -A 2

Repository: genlayerlabs/genlayer-studio

Length of output: 1344


Remove the no-placeholder f-string and avoid double-prefixing the error message.

Line 837 has an f-string with no placeholders (Ruff F541), and error_message on line 843 double-prefixes the error because GenVMInternalError messages are already constructed with the "GenVM internal error:" prefix in backend/node/genvm/base.py.

🔧 Proposed fix
             self.logger.warning(
-                f"Validator encountered GenVMInternalError",
+                "Validator encountered GenVMInternalError",
                 error_code=e.error_code,
                 causes=e.causes,
                 is_fatal=e.is_fatal,
             )
             processing_time = int((time.time() - start_time) * 1000)
-            error_message = f"GenVM internal error: {e}"
+            err_str = str(e)
+            error_message = (
+                err_str
+                if err_str.startswith("GenVM internal error")
+                else f"GenVM internal error: {err_str}"
+            )
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
self.logger.warning(
f"Validator encountered GenVMInternalError",
error_code=e.error_code,
causes=e.causes,
is_fatal=e.is_fatal,
)
processing_time = int((time.time() - start_time) * 1000)
error_message = f"GenVM internal error: {e}"
error_result = bytes(
[public_abi.ResultCode.VM_ERROR]
) + error_message.encode("utf-8")
self.logger.warning(
"Validator encountered GenVMInternalError",
error_code=e.error_code,
causes=e.causes,
is_fatal=e.is_fatal,
)
processing_time = int((time.time() - start_time) * 1000)
err_str = str(e)
error_message = (
err_str
if err_str.startswith("GenVM internal error")
else f"GenVM internal error: {err_str}"
)
error_result = bytes(
[public_abi.ResultCode.VM_ERROR]
) + error_message.encode("utf-8")
🧰 Tools
🪛 Ruff (0.14.14)

837-837: f-string without any placeholders

Remove extraneous f prefix

(F541)

🤖 Prompt for AI Agents
In `@backend/node/base.py` around lines 836 - 846, The warning call is using an
unnecessary f-string and the error_message is double-prefixing the
GenVMInternalError text; update the logger.warning invocation in the exception
handler (logger.warning) to use a plain string literal (not f"...") and keep the
structured fields (error_code, causes, is_fatal), and change the construction of
error_message (used with public_abi.ResultCode.VM_ERROR and the bytes prefix) to
use the exception text directly (e.g., str(e) or e.message) without adding
"GenVM internal error:" so the final payload does not repeat the prefix.

@sonarqubecloud

Copy link
Copy Markdown

Quality Gate Failed Quality Gate failed

Failed conditions
7.4% Coverage on New Code (required ≥ 80%)
14.6% Duplication on New Code (required ≤ 3%)

See analysis details on SonarQube Cloud

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Fix all issues with AI agents
In `@scripts/run_update_error_transactions.sh`:
- Around line 172-173: The current sequential kubectl wait calls (using POD_NAME
and two separate --for=jsonpath checks for 'Succeeded' and 'Failed') can hang up
to 24h because the first wait will not give up early if the pod is impossible to
reach Succeeded; replace the two sequential waits with a single loop that polls
the pod phase (e.g., kubectl get pod "$POD_NAME" -o jsonpath='{.status.phase}')
until either "Succeeded" or "Failed" is observed or a shared timeout expires,
then break and act on the observed phase (capture exit code/cleanup accordingly)
to avoid the long blocking behavior.

Comment on lines +172 to +173
kubectl wait pod "$POD_NAME" --for=jsonpath='{.status.phase}'=Succeeded --timeout=86400s 2>/dev/null || \
kubectl wait pod "$POD_NAME" --for=jsonpath='{.status.phase}'=Failed --timeout=60s 2>/dev/null || true

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🌐 Web query:

kubectl wait jsonpath condition cannot be met behavior exit immediately vs timeout

💡 Result:

kubectl wait does not try to determine whether a --for=jsonpath=... condition is impossible to satisfy. It will keep waiting until either the JSONPath comparison becomes true or the timeout is reached. The --timeout flag is “the length of time to wait before giving up”; --timeout=0 means “check once and don’t wait” (i.e., exit immediately based on the current state), and a negative timeout waits up to a week. [1]

Key practical behaviors that often look like “exit immediately vs timeout”:

  • Immediate exit (no waiting): use --timeout=0 (single evaluation). If the JSONPath condition is not met at that moment, kubectl wait exits non-zero rather than waiting. [1]
  • Otherwise, it waits until timeout: if the JSONPath never matches (even if it seems it “can’t” match), you’ll get a timeout failure like “timed out waiting for the condition”. [1]
  • Timeout is effectively per resource: if you wait on multiple objects (e.g., via -l ...), the elapsed time can exceed the specified timeout because it can be applied per resource rather than as a single global deadline. [2]

JSONPath-specific gotcha:

  • Waiting for “any value appears at this JSONPath” (without knowing the value in advance) has historically been awkward; a Kubernetes SIG CLI issue explicitly calls out that you often can’t use kubectl wait for “any IP/any dynamically assigned value” and end up writing polling loops instead. [3]

Sources:
[1] Kubernetes kubectl wait reference (timeout semantics, --for=jsonpath syntax) (kubernetes.io)
[2] kubectl issue about --timeout being per-resource (github.com)
[3] Kubernetes issue on kubectl wait JSONPath with unknown/dynamic values (github.com)

Citations:


Failure path can hang for up to 24h due to sequential kubectl wait commands.

When a pod fails, the Succeeded wait blocks for the full 86400s timeout before the Failed wait runs, delaying exit-code retrieval and cleanup. kubectl wait does not exit early when a condition cannot be met—it waits until timeout. Consider a single loop that checks for either phase with a shared timeout.

✅ Safer wait loop with a shared timeout
- kubectl wait pod "$POD_NAME" --for=jsonpath='{.status.phase}'=Succeeded --timeout=86400s 2>/dev/null || \
- kubectl wait pod "$POD_NAME" --for=jsonpath='{.status.phase}'=Failed --timeout=60s 2>/dev/null || true
+ deadline=$((SECONDS + 86400))
+ while true; do
+   phase=$(kubectl get pod "$POD_NAME" -o jsonpath='{.status.phase}' 2>/dev/null || echo "")
+   if [[ "$phase" == "Succeeded" || "$phase" == "Failed" ]]; then
+     break
+   fi
+   if (( SECONDS >= deadline )); then
+     echo "Timed out waiting for pod to complete"
+     break
+   fi
+   sleep 5
+ done
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
kubectl wait pod "$POD_NAME" --for=jsonpath='{.status.phase}'=Succeeded --timeout=86400s 2>/dev/null || \
kubectl wait pod "$POD_NAME" --for=jsonpath='{.status.phase}'=Failed --timeout=60s 2>/dev/null || true
deadline=$((SECONDS + 86400))
while true; do
phase=$(kubectl get pod "$POD_NAME" -o jsonpath='{.status.phase}' 2>/dev/null || echo "")
if [[ "$phase" == "Succeeded" || "$phase" == "Failed" ]]; then
break
fi
if (( SECONDS >= deadline )); then
echo "Timed out waiting for pod to complete"
break
fi
sleep 5
done
🤖 Prompt for AI Agents
In `@scripts/run_update_error_transactions.sh` around lines 172 - 173, The current
sequential kubectl wait calls (using POD_NAME and two separate --for=jsonpath
checks for 'Succeeded' and 'Failed') can hang up to 24h because the first wait
will not give up early if the pod is impossible to reach Succeeded; replace the
two sequential waits with a single loop that polls the pod phase (e.g., kubectl
get pod "$POD_NAME" -o jsonpath='{.status.phase}') until either "Succeeded" or
"Failed" is observed or a shared timeout expires, then break and act on the
observed phase (capture exit code/cleanup accordingly) to avoid the long
blocking behavior.

@cristiam86 cristiam86 changed the title Rollback genvm version chore: rollback genvm version Jan 29, 2026
@cristiam86
cristiam86 merged commit 5778826 into main Jan 29, 2026
24 of 27 checks passed
@github-actions

Copy link
Copy Markdown
Contributor

🎉 This PR is included in version 0.96.1 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant