Skip to content

Removes default config values from provider JSONs - #1341

Merged
cristiam86 merged 7 commits into
mainfrom
edgars/fix-temperature-config
Oct 31, 2025
Merged

cristiam86 merged 7 commits into
mainfrom
edgars/fix-temperature-config

Conversation

@MuncleUscles

@MuncleUscles MuncleUscles commented Oct 13, 2025 •

Copy link
Copy Markdown
Member

What

  • Removes default configuration values (temperature, max_tokens, top_k, top_p, timeout) from the JSON files defining default providers.
  • Updates frontend to display a warning if a provider or model has a potential configuration issue

Why

  • The default configuration values in the provider JSONs are incorrect and should be determined dynamically.
  • To improve user experience by displaying warning if providers or models are not available

Testing done

  • Verified that the default configuration values are no longer present in the provider JSON files.
  • Checked that warnings are displayed in the UI for unavailable providers/models

Decisions made

Checks

  • I have tested this code
  • I have reviewed my own PR
  • I have created an issue for this PR
  • I have set a descriptive PR title compliant with conventional commits

Reviewing tips

User facing release notes

Summary by CodeRabbit

  • New Features

    • Simulator: Labels providers/models with “potential config issue” when detected.
    • Simulator: Shows all models for a provider and auto-selects the first on initial load for new validators.
  • Bug Fixes

    • Simulator: More reliable initial value setup after providers load.
    • Mocking: Falls back to real provider when no mock match, improving simulation accuracy.
  • Chores

    • Standardized provider configs to rely on service defaults across many models.
    • Validation relaxed to no longer require a max token limit.
    • Backend container build: removed legacy docker-group setup.

@coderabbitai

coderabbitai Bot commented Oct 13, 2025 •

Copy link
Copy Markdown
Contributor

Walkthrough

Replaced populated config objects with empty {} across many default provider JSONs, removed max_tokens from anthropic config schema required list, changed greyboxing mock return shape and fallback logic, and updated ValidatorModal to surface potential config issues and initialize models once providers load.

Changes

Cohort / File(s) Summary of Changes
Anthropic provider configs
backend/node/create_nodes/default_providers/anthropic_claude-3-5-haiku-20241022.json, backend/node/create_nodes/default_providers/anthropic_claude-3-7-sonnet-20250219.json, backend/node/create_nodes/default_providers/anthropic_claude-4-opus.json, backend/node/create_nodes/default_providers/anthropic_claude-4-sonnet.json
Replaced populated config (temperature, max_tokens, top_k, top_p, timeout) with {}; structure otherwise unchanged.
Google Gemini provider configs
backend/node/create_nodes/default_providers/google_gemini-2.5-flash-lite.json, .../google_gemini-2.5-flash.json, .../google_googlegemini-2.0-flash-lite-001.json
Emptied config objects (removed explicit temperature/max_tokens); other fields unchanged.
HeuristAI provider configs
backend/node/create_nodes/default_providers/heuristai_deepseekdeepseek-v3.json, .../heuristai_meta-llamalllama-3.3-70b-instruct.json, .../heuristai_mistralaimixtral-8x7b-instruct.json
Cleared config objects (removed temperature/max_tokens); no structural changes.
OpenAI provider configs
backend/node/create_nodes/default_providers/openai_gpt-4-1106-preview.json, .../openai_gpt-4.1-mini.json, .../openai_gpt-4.1-nano.json, .../openai_gpt-4.1.json, .../openai_gpt-4o.json, .../openai_gpt-5-mini.json, .../openai_gpt-5-nano.json, .../openai_gpt-5.json
Replaced populated config with {}; removed temperature/max_tokens defaults; rest unchanged.
xAI provider config
backend/node/create_nodes/default_providers/xai_grok-3.json
Emptied config object; other fields unchanged.
Provider schema
backend/node/create_nodes/providers_schema.json
Removed max_tokens from required list under the anthropic plugin's config schema.
Greyboxing validator
backend/validators/greyboxing.lua
get_mock_response_from_table now returns { data, consumed_gen }; default is { data = "no match", consumed_gen = 0 }. just_in_backend only uses mock when result.data != "no match" and logs fallback paths.
Simulator Validator modal
frontend/src/components/Simulator/ValidatorModal.vue
Added watch import and watcher for provider list load; provider/model labels annotate "potential config issue"; model list now includes all models for selected provider; initializes selection on first providers load when creating.
Docker backend build
docker/Dockerfile.backend
Removed ARG DOCKER_GID, removed docker group creation/usermod steps, and dropped docker.io from apt-get install list; genvm setup retained.

Sequence Diagram(s)

sequenceDiagram
  autonumber
  participant C as Caller
  participant G as greyboxing.lua
  participant M as MockTable
  participant P as RealProvider

  C->>G: just_in_backend(request)
  G->>M: get_mock_response_from_table(key)
  M-->>G: { data, consumed_gen }  or  { data: "no match", consumed_gen: 0 }
  alt Mock match (data ≠ "no match")
    G-->>C: return mock { data, consumed_gen }
  else No match (data == "no match")
    G->>P: invoke real provider
    P-->>G: provider response
    G-->>C: return provider response
  end
Loading
sequenceDiagram
  autonumber
  participant UI as ValidatorModal.vue
  participant S as Store/API

  UI->>S: load providers
  S-->>UI: providers list
  note over UI: watch(providers length) triggers on first non-empty load (create mode)
  UI->>UI: initialize provider selection (set provider, compute labels)
  UI->>S: fetch all models for selected provider
  S-->>UI: models list (all)
  UI->>UI: set initial model = first model if any
  note over UI: Labels append "potential config issue" based on is_available / is_model_available
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Potential points needing extra attention:

  • providers_schema.json change impact on validation and any consumers that assumed max_tokens required.
  • Behavioural differences from emptied provider configs (where defaults are applied) across runtime code paths.
  • greyboxing.lua return-shape changes and callers that consume mock responses.

Possibly related PRs

Suggested labels

run-tests

Suggested reviewers

  • kstroobants

Poem

A rabbit finds the configs bare,
Knobs tucked in with gentle care.
Mocks now speak in tidy pairs,
Vue wakes up and chooses chairs.
Hop — defaults set off to air. 🐇✨

Pre-merge checks and finishing touches

❌ Failed checks (1 warning)
Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. You can run @coderabbitai generate docstrings to improve docstring coverage.
✅ Passed checks (2 passed)
Check name Status Explanation
Title Check ✅ Passed The pull request title "Removes default config values from provider JSONs" is clear, concise, and directly related to the main change in the changeset. It accurately summarizes the primary modification across the majority of files in the PR, which involves removing explicit configuration values (temperature, max_tokens, etc.) from provider JSON definitions. The title is specific enough that a teammate scanning the history would understand the core change without being overly verbose or vague.
Description Check ✅ Passed The pull request description is largely complete and covers the essential sections of the template. The "What" section clearly describes the removal of default configuration values and frontend warning updates. The "Why" section provides solid rationale about incorrect defaults and UX improvements. The "Testing done" section outlines verification steps for both JSON changes and UI warnings. The Checks section shows the author completed most items (marked tested, self-reviewed, and used conventional commits). However, the description omits the issue reference at the top, and several sections (Decisions made, Reviewing tips, User facing release notes) are left empty, though these are less critical than the core explanation.
✨ Finishing touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch edgars/fix-temperature-config

📜 Recent review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 0984a5c and 016b7e6.

📒 Files selected for processing (1)
  • docker/Dockerfile.backend (3 hunks)
🧰 Additional context used
📓 Path-based instructions (1)
docker/**

📄 CodeRabbit inference engine (AGENTS.md)

Maintain container orchestration assets in docker/

Files:

  • docker/Dockerfile.backend
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
  • GitHub Check: backend-unit-tests
🔇 Additional comments (4)
docker/Dockerfile.backend (4)

15-25: Clarify the purpose of musl and musl-dev packages at line 17-18.

The removal of docker.io appears intentional per the PR objectives, but the addition or retention of musl and musl-dev at lines 17-18 is unclear. These are C standard library implementations typically used for static linking or Alpine-based builds. Confirm this is intentional and necessary for the GenVM or Python build process.

Can you clarify the purpose of these packages in this Ubuntu base image context? If they're unnecessary, they should be removed to keep the image lean.


5-5: Verify removal of DOCKER_GID build argument.

Line 5 appears to have the ARG DOCKER_GID=999 argument removed. Confirm this is intentional and that no downstream images or build scripts depend on this argument.


31-36: Verify /genvm directory permissions after docker group removal.

With the removal of docker group setup and the docker-in-docker capabilities, confirm that the backend-user:backend-group ownership at line 35 and the /genvm directory creation at line 36 provide adequate permissions for GenVM execution. Specifically:

  • Can backend-user execute binaries in /genvm/executor/ (line 70)?
  • Are there any GenVM operations that previously relied on docker group membership?

Run a test in a built container to verify backend-user can properly execute GenVM post-install and related operations without docker group permissions.


90-93: Verify debug stage retains necessary capabilities.

The debug stage (lines 87-93) inherits from base and should work correctly without docker group permissions. Confirm debugpy and uvicorn hot-reload functionality are unaffected by the removal of docker-in-docker setup.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
frontend/src/components/Simulator/ValidatorModal.vue (1)

1-1: Fix Prettier formatting.

The pre-commit hook detected formatting issues. Run the following command to fix:

prettier --write frontend/src/components/Simulator/ValidatorModal.vue
🧹 Nitpick comments (1)
backend/validators/greyboxing.lua (1)

46-59: Consider using a constant for the sentinel value.

The magic string "no match" is used as a sentinel value to indicate no mock was found. For better maintainability, consider defining this as a module-level constant (e.g., local NO_MATCH = "no match").

Apply this diff:

+local NO_MATCH = "no match"
+
 local function get_mock_response_from_table(table, message)
 	for key, value in pairs(table) do
 		if string.find(message, key) then
 			return {
 				data = value,
 				consumed_gen = 0
 			}
 		end
 	end
 	return {
-		data = "no match",
+		data = NO_MATCH,
 		consumed_gen = 0
 	}
 end

Then update line 193:

-	if result.data ~= "no match" then
+	if result.data ~= NO_MATCH then
📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 3429c2f and 0984a5c.

📒 Files selected for processing (22)
  • backend/node/create_nodes/default_providers/anthropic_claude-3-5-haiku-20241022.json (1 hunks)
  • backend/node/create_nodes/default_providers/anthropic_claude-3-7-sonnet-20250219.json (1 hunks)
  • backend/node/create_nodes/default_providers/anthropic_claude-4-opus.json (1 hunks)
  • backend/node/create_nodes/default_providers/anthropic_claude-4-sonnet.json (1 hunks)
  • backend/node/create_nodes/default_providers/google_gemini-2.5-flash-lite.json (1 hunks)
  • backend/node/create_nodes/default_providers/google_gemini-2.5-flash.json (1 hunks)
  • backend/node/create_nodes/default_providers/google_googlegemini-2.0-flash-lite-001.json (1 hunks)
  • backend/node/create_nodes/default_providers/heuristai_deepseekdeepseek-v3.json (1 hunks)
  • backend/node/create_nodes/default_providers/heuristai_meta-llamallama-3.3-70b-instruct.json (1 hunks)
  • backend/node/create_nodes/default_providers/heuristai_mistralaimixtral-8x7b-instruct.json (1 hunks)
  • backend/node/create_nodes/default_providers/openai_gpt-4-1106-preview.json (1 hunks)
  • backend/node/create_nodes/default_providers/openai_gpt-4.1-mini.json (1 hunks)
  • backend/node/create_nodes/default_providers/openai_gpt-4.1-nano.json (1 hunks)
  • backend/node/create_nodes/default_providers/openai_gpt-4.1.json (1 hunks)
  • backend/node/create_nodes/default_providers/openai_gpt-4o.json (1 hunks)
  • backend/node/create_nodes/default_providers/openai_gpt-5-mini.json (1 hunks)
  • backend/node/create_nodes/default_providers/openai_gpt-5-nano.json (1 hunks)
  • backend/node/create_nodes/default_providers/openai_gpt-5.json (1 hunks)
  • backend/node/create_nodes/default_providers/xai_grok-3.json (1 hunks)
  • backend/node/create_nodes/providers_schema.json (1 hunks)
  • backend/validators/greyboxing.lua (3 hunks)
  • frontend/src/components/Simulator/ValidatorModal.vue (4 hunks)
🧰 Additional context used
📓 Path-based instructions (5)
frontend/**/*.{ts,tsx,vue}

📄 CodeRabbit inference engine (CLAUDE.md)

TypeScript code should follow ESLint + Prettier conventions

Files:

  • frontend/src/components/Simulator/ValidatorModal.vue
frontend/src/**/*.vue

📄 CodeRabbit inference engine (CLAUDE.md)

Use Vue Composition API with <script setup> syntax in Vue components

Use kebab-case for Vue single-file component (.vue) filenames

Files:

  • frontend/src/components/Simulator/ValidatorModal.vue
frontend/src/components/Simulator/**/*.vue

📄 CodeRabbit inference engine (CLAUDE.md)

Contract execution UI components belong under frontend/src/components/Simulator/

Files:

  • frontend/src/components/Simulator/ValidatorModal.vue
frontend/**

📄 CodeRabbit inference engine (CLAUDE.md)

Use Socket.io for real-time consensus updates in the frontend

Files:

  • frontend/src/components/Simulator/ValidatorModal.vue
frontend/src/**/*.{ts,vue}

📄 CodeRabbit inference engine (AGENTS.md)

frontend/src/**/*.{ts,vue}: Place all Vue 3 + TypeScript application source under frontend/src
Use ESLint + Prettier for Vue/TypeScript code
Prefer PascalCase for component identifiers and camelCase for stores/utilities in Vue/TypeScript code

Files:

  • frontend/src/components/Simulator/ValidatorModal.vue
🪛 GitHub Actions: pre-commit
frontend/src/components/Simulator/ValidatorModal.vue

[error] 1-1: Prettier formatting changes were applied to the file during the pre-commit hook. Run 'prettier --write' to fix code style issues. Command 'pre-commit run --show-diff-on-failure --color=always --all-files' reported a failure due to formatting.

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (4)
  • GitHub Check: test
  • GitHub Check: load-test
  • GitHub Check: Cursor Bugbot
  • GitHub Check: backend-unit-tests
🔇 Additional comments (8)
backend/node/create_nodes/default_providers/openai_gpt-5.json (1)

5-5: LGTM!

The removal of explicit config defaults aligns with the PR objective to let configuration values be determined dynamically.

backend/node/create_nodes/default_providers/anthropic_claude-3-5-haiku-20241022.json (1)

5-5: LGTM!

Consistent with the pattern across provider JSON files to remove hardcoded defaults.

backend/node/create_nodes/default_providers/anthropic_claude-3-7-sonnet-20250219.json (1)

5-5: LGTM!

The config change is consistent with the broader refactoring to move defaults out of provider definitions.

backend/validators/greyboxing.lua (1)

176-199: LGTM! Improved fallback logic.

The explicit check for "no match" and fallthrough to real providers is clear and well-documented. The debug logging will help troubleshoot mock behavior.

frontend/src/components/Simulator/ValidatorModal.vue (4)

111-121: Verify backend error handling for unavailable providers.

Provider options now show warnings but remain selectable (disabled: false) even when is_available is false. Ensure the backend gracefully handles attempts to create validators with unavailable providers and returns clear error messages.


124-139: Verify backend error handling for unavailable models.

Similar to provider options, models with potential config issues remain selectable. Confirm that backend validation provides clear feedback when users attempt to create validators with unavailable models.


141-151: LGTM! Aligns with the PR objective.

Loading all models (not just available ones) and defaulting to the first model is consistent with the approach of showing warnings for potential config issues rather than hiding unavailable options.


208-217: No race condition—both initialization calls target different lifecycle events. The watch(..., { immediate: true }) handles the initial provider load on mount, while @onOpen="tryInitValues" ensures values reset each time the modal opens. No changes needed.

Likely an incorrect or invalid review comment.

@sonarqubecloud

Copy link
Copy Markdown

Quality Gate Failed Quality Gate failed

Failed conditions
0.0% Coverage on New Code (required ≥ 80%)

See analysis details on SonarQube Cloud

@cristiam86
cristiam86 merged commit 80b1fe1 into main Oct 31, 2025
1 check passed
@github-actions

github-actions Bot commented Nov 3, 2025

Copy link
Copy Markdown
Contributor

🎉 This PR is included in version 0.79.3 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants