Skip to content

agent/17: Log tail default drift: lemonade returns 200 lines, protocol/vllm return 80 - #25

Merged
mikeroySoft merged 5 commits into
mainfrom
agent/17
Oct 25, 2026
Merged

mikeroySoft merged 5 commits into
mainfrom
agent/17

Conversation

@mikeroySoft

Copy link
Copy Markdown
Owner

Closes #17

Gate report

Gate report

  • conflict-markers: PASS
  • clippy: PASS
  • tests: PASS
  • smoke: PASS
  • leak-scan: PASS

@mikeroySoft

Copy link
Copy Markdown
Owner Author

Standards

  • Finding — missing required Gherkin coverage (engines/lemonade/src/lib.rs). Changing the default from 200 to the protocol-defined 80 alters user-observable service-log output. AGENTS.md §3 requires such behavior to be covered by a scenario in tests/e2e-cucumber/features/. The diff adds only a unit test, and no existing scenario covers Lemonade’s omitted-tail_lines default.

No other standards violations or notable code smells were found. The supplied gate report is accepted as authoritative.

Spec

  • Finding — regression test does not exercise the reported behavior (log_tail_default_matches_protocol_contract). The test compares an imported constant with the fully qualified form of that same constant. It does not verify that a Lemonade request omitting tail_lines actually returns the protocol-default number of lines. Test logs_response with more than 80 lines and tail_lines: None, or test an extracted effective-limit helper.

The production change itself correctly removes Lemonade’s local 200 default and uses the shared protocol value, matching vLLM. No scope creep was found.

GitHub issue #17’s exact text could not be retrieved because GitHub API access was unavailable; the Spec assessment therefore uses the issue description recorded in the commit message.

VERDICT: REVISE

@mikeroySoft

Copy link
Copy Markdown
Owner Author

Standards

  • Finding — required Gherkin coverage is still missing (engines/lemonade/src/lib.rs:1013). The diff changes an omitted tail_lines request from 200 returned lines to 80, altering observable service-log output. AGENTS.md §3 requires user-observable behavior changes to have a scenario under tests/e2e-cucumber/features/; the added Rust test does not satisfy that requirement. If the existing harness cannot exercise EngineMethod::Logs, the harness needs extending or the documented requirement must otherwise be resolved.

No other standards violations or notable code smells were found. The supplied gate report is accepted as authoritative.

Spec

No findings. The diff removes Lemonade’s local default, imports the unchanged protocol constant, and verifies that an omitted limit returns the newest 80 lines. The helper extraction and explicit-limit assertion are directly relevant and do not constitute scope creep.

Summary: Standards—1 finding (missing Gherkin coverage). Spec—0 findings.

VERDICT: REVISE

@mikeroySoft

Copy link
Copy Markdown
Owner Author

Standards

No findings. The diff follows repository conventions for protocol ownership, regression testing, and user-observable Gherkin coverage. No material code smells identified.

Spec

No findings. The issue text was unavailable because GitHub access failed. Based on the diff and commit context, the change consistently applies the protocol’s 80-line default to Lemonade, preserves explicit limits, and adds focused and end-to-end regression coverage across both built-in engines.

The supplied deterministic gate report is authoritative: all checks passed.

Summary: Standards—0 findings; Spec—0 findings.

VERDICT: APPROVE

@mikeroySoft mikeroySoft added the factory-approved Reviewer APPROVE recorded by the factory; merge-stage precondition label Sep 2, 2026
@mikeroySoft
mikeroySoft force-pushed the agent/17 branch 2 times, most recently from c2cd5f3 to 891781d Compare September 2, 2026 04:17
@mikeroySoft mikeroySoft removed the factory-approved Reviewer APPROVE recorded by the factory; merge-stage precondition label Sep 2, 2026
@mikeroysoft-agent

Copy link
Copy Markdown
Collaborator

{"type":"session","version":3,"id":"01a16ec7-9605-71e8-972f-ad7b6125471a","timestamp":"2026-10-24T16:38:56.261Z","cwd":"/tmp/factory-review-gejanq6y"}

Factory rejected reviewer evidence: review_exit:1.

@mikeroysoft-agent

Copy link
Copy Markdown
Collaborator

Review: #17, Lemonade log tail default drift

Spec check (issue #17)

The #17 change itself is correct:

  • Local constant removed. engines/lemonade/src/lib.rs drops const DEFAULT_LOG_TAIL_LINES: usize = 200; (review.diff:12220) and adds DEFAULT_LOG_TAIL_LINES to the existing rocm_engine_protocol::{…} import (review.diff:12208).
  • Protocol value unchanged. The diff does not touch crates/rocm-engine-protocol, as the issue requires.
  • Acceptance 1 holds. In the snapshot, DEFAULT_LOG_TAIL_LINES under engines/ appears only as an import or as unwrap_or(DEFAULT_LOG_TAIL_LINES) (lemonade lib.rs:9,770, vllm state.rs:10,99). There are no local definitions.
  • Acceptance 2 is covered. tests/e2e-cucumber/features/logs.feature (review.diff:13117-13121) adds a scenario, with steps in tests/e2e-cucumber/tests/e2e/logs_steps.rs (review.diff:14968-15059):
    • It writes 200 log lines for each engine.
    • It sends a logs request with no tail_lines through __engine-stdio to both lemonade and vllm.
    • It asserts that each engine returns exactly 80 lines, ending at line 200.

Required fixes (blocking)

  1. The diff contains a large amount of work unrelated to Log tail default drift: lemonade returns 200 lines, protocol/vllm return 80 #17, and it is almost all deletions.
    • What the diff does. Besides the three in-scope files, it changes about 60 others and removes roughly 5,000 lines. Examples:

      • Deletes apps/rocm/src/advised_commands.rs (review.diff:1163).
      • Deletes apps/rocm/src/therock_version_proptests.rs and its proptest regressions file (review.diff:6315, 1146).
      • Deletes tests/e2e-cucumber/src/polled_deadline.rs (review.diff:13416) and tests/e2e-cucumber/src/quit_and_wait.rs (review.diff:13836).
      • Removes the public verified_torchaudio_release from engines/vllm/src/lib.rs (review.diff:12315), along with its implementation and test in engines/vllm/src/install.rs (review.diff:12263-12296).
      • Removes import_failure_hint and five of its tests from engines/vllm/src/process.rs (review.diff:12370, 12447-12646).
      • Removes 183 lines from crates/rocm-core/src/managed_runtime.rs (review.diff:9205).
      • Changes wording in CONTRIBUTING.md (review.diff:16-91).
      • Edits .github/workflows/ci.yml (review.diff:1).
    • Likely cause. All of these remove features, tests and docs that the snapshot no longer has (for example, verified_torchaudio_release and import_failure_hint appear nowhere in snapshot/). That fits a branch cut from an older main. The base..head diff then shows the newer main work as reverts.

    • Rule broken. AGENTS.md:355: "keep each PR scoped to one logical change", repeated in the pre-merge checklist at AGENTS.md:394. The issue's Fix section also asks only to change the Lemonade constant.

    • Impact. If the gated tree lands as-is (squash, or tree replacement), it silently reverts unrelated shipped behaviour and tests: the rocm install sdk torchaudio pinning, the vLLM import-failure hint, the advised-commands module and the TheRock version proptests. The gate passing does not catch this, because the deleted code takes its own tests with it.

    • Fix. Rebase agent/17 onto current main so base..head contains only:

      • engines/lemonade/src/lib.rs
      • tests/e2e-cucumber/features/logs.feature
      • tests/e2e-cucumber/tests/e2e/logs_steps.rs

      Re-run the gate after the rebase.

Optional suggestions (non-blocking)

  • tests/e2e-cucumber/tests/e2e/logs_steps.rs:14976 (review.diff) copies the value 80 into PROTOCOL_DEFAULT_TAIL instead of importing it. That is acceptable for a black-box contract test, and the doc comment names the source constant. If rocm-engine-protocol is ever a dev-dependency of e2e-cucumber, importing it would remove the copy. It is not needed for this issue.

VERDICT: REVISE

mikeroySoft and others added 5 commits October 24, 2026 12:56
A logs request without tail_lines returned 200 lines from lemonade but 80
from vllm. The default is a protocol contract value; import it as vllm
already does. Adds a regression test pinning the engine default to the
protocol constant.

Fixes #17

Signed-off-by: Michael Roy <mike@mikeroysoft.com>
Replace the tautological constant-equality test with one that writes a
200-line log and asserts a request with tail_lines omitted returns exactly
DEFAULT_LOG_TAIL_LINES (80) lines ending at the newest line. Extract
recent_log_lines(path, Option<usize>) from logs_response so the default
resolution is testable without going through AppPaths::discover.

No Gherkin scenario: neither rocm nor rocmd invokes EngineMethod::Logs;
`rocm logs --service` reads the log file directly with the CLI's own
DEFAULT_LOG_TAIL_LINES. The changed default is observable only over the
engine-plugin JSON protocol, which the black-box e2e harness does not drive.

Signed-off-by: Michael Roy <mike@mikeroysoft.com>
The changed default is observable only over the engine-plugin `logs`
method, which neither `rocm` nor `rocmd` invokes on a user's behalf.
Drive the built-in engines through the hidden `rocm __engine-stdio`
entry point instead: plant a 200-line service log for lemonade and
vllm, request logs without `tail_lines`, and assert both return
exactly the 80-line protocol default ending at the newest line.

Scenario @id:logs-engine-omitted-tail-uses-protocol-default fails with
the old lemonade-local 200 default and passes with the protocol import.
It reads planted files only, so it runs on the mock lane every PR.

Signed-off-by: Michael Roy <mike@mikeroysoft.com>
Signed-off-by: mikeroysoft-agent <339795455+mikeroysoft-agent@users.noreply.github.com>
Signed-off-by: mikeroysoft-agent <339795455+mikeroysoft-agent@users.noreply.github.com>
@mikeroysoft-agent

Copy link
Copy Markdown
Collaborator

Standards

No required fixes.

  • Regression scenario (AGENTS.md §3). AGENTS.md requires a bug fix to ship with a regression test that fails before the fix and passes after it, and it requires a Gherkin scenario for user-observable behaviour. The diff meets both. The new scenario is at tests/e2e-cucumber/features/logs.feature:45-49, with its steps in tests/e2e-cucumber/tests/e2e/logs_steps.rs:74-92,101-128,137-149. It plants 200 lines (logs_steps.rs:63) and expects exactly 80 from each engine (logs_steps.rs:66,144). Before the fix, Lemonade returned 200 lines, so the scenario would have failed then.
  • Fix location (AGENTS.md §5). The fix is at the root cause: it deletes the shadowing constant (engines/lemonade/src/lib.rs:26) and imports the protocol constant (engines/lemonade/src/lib.rs:14). The protocol crate is unchanged, so the §6 rule to verify all engines after a protocol change does not apply.
  • Planted log paths. The data/engines/<engine>/logs/<service>.log path at logs_steps.rs:82-89 matches the real paths: AppPaths::engine_dir → engine_logs_dir (crates/rocm-core/src/paths.rs:104-114), used by service_files in both engines.
  • Abstractions. None added. The test uses the existing __engine-stdio entry point and the existing run_rocm_with_stdin and record/ok_output helpers.

Optional:

  • logs_steps.rs:66 hard-codes 80 instead of importing the protocol constant. The comment at logs_steps.rs:64-65 explains this as a deliberate black-box pin, and the issue says "Do not change the protocol value," so it is defensible. If the protocol value ever changes on purpose, this scenario will have to be edited by hand.

Spec (issue #17)

  • Fix as specified. The issue says to "Delete the local constant in engines/lemonade and import rocm_engine_protocol::DEFAULT_LOG_TAIL_LINES." The diff does this at engines/lemonade/src/lib.rs:14 and :26. The protocol value is untouched.
  • Acceptance 1: "grep -rn "DEFAULT_LOG_TAIL_LINES" engines/ shows imports only, no local definitions." In the snapshot, the only hits are the import at engines/lemonade/src/lib.rs:9, its use at :770, and the import and use in engines/vllm/src/state.rs:10,99. No local definition remains. Met.
  • Acceptance 2: "A logs request with tail_lines: None yields the same line limit from both engines." Both handlers now use request.tail_lines.unwrap_or(DEFAULT_LOG_TAIL_LINES), which resolves to the same protocol constant. The scenario checks this end to end through both engines (logs_steps.rs:101-149). Met.
  • Scope. Nothing beyond the issue: one import change plus the required regression scenario.

Summary:

  • Standards: 0 required, 1 optional (the hard-coded 80 in the test).
  • Spec: 0 findings; both acceptance criteria are met.

VERDICT: APPROVE

@mikeroysoft-agent mikeroysoft-agent added factory-approved Reviewer APPROVE recorded by the factory; merge-stage precondition and removed factory-approved Reviewer APPROVE recorded by the factory; merge-stage precondition labels Oct 24, 2026
@mikeroysoft-agent

Copy link
Copy Markdown
Collaborator

Gate failed after rebase onto current main:

Gate report

Host toolchain

ROCm: unavailable
Kernel: 7.2.5-3-omarchy
amdgpu: unavailable

  • conflict-markers: PASS
  • clippy: PASS
  • tests: FAIL
  • smoke: PASS
  • leak-scan: PASS

tests failure

test therock::archive_properties::sdk_tarball_unpack_extracts_a_benign_tarball_into_the_install_root ... ok
test uninstall::tests::a_block_for_another_install_folder_stays ... ok
test uninstall::tests::a_block_in_a_profile_the_installer_created_leaves_it_empty ... ok
test uninstall::tests::a_folder_that_cannot_be_listed_is_an_error_not_an_empty_one ... ok
test uninstall::tests::a_program_whose_name_is_not_utf8_still_counts_as_another_program ... ok
test uninstall::tests::a_replacement_left_by_an_interrupted_uninstall_does_not_block_the_next ... ok
test uninstall::tests::a_users_own_path_line_is_never_touched ... ok
test uninstall::tests::an_unterminated_block_is_left_alone ... ok
test uninstall::tests::crlf_profiles_are_handled ... ok
test uninstall::tests::other_entries_ignore_rocm_cli_files_and_what_is_being_removed ... ok
test uninstall::tests::plan::a_block_from_an_earlier_install_in_another_shell_is_also_undone ... ok
test uninstall::tests::plan::a_block_written_through_a_linked_home_is_still_found ... ok
test uninstall::tests::plan::a_failure_after_the_path_undo_still_reports_what_was_done ... ok
test uninstall::tests::plan::a_linked_profile_is_edited_where_it_lives_and_stays_a_link ... ok
test uninstall::tests::plan::a_path_change_that_cannot_be_undone_stops_before_the_binaries_go ... ok
test uninstall::tests::plan::a_profile_that_is_not_utf8_is_still_edited_byte_for_byte ... ok
test uninstall::tests::plan::a_record_naming_a_profile_that_is_not_utf8_is_still_followed ... ok
test uninstall::tests::plan::a_recorded_profile_block_is_undone_and_the_record_goes_last ... ok
test uninstall::tests::plan::a_recorded_profile_that_cannot_be_read_stops_before_anything_goes ... ok
test uninstall::tests::plan::a_recorded_profile_without_the_block_is_reported_not_edited ... ok
test uninstall::tests::plan::a_recorded_spelling_is_matched_even_when_no_derived_one_is ... ok
test uninstall::tests::plan::an_install_folder_whose_path_is_not_utf8_is_still_found ... ok
test uninstall::tests::plan::an_unrecorded_profile_that_cannot_be_read_is_a_warning_not_an_absence ... ok
test uninstall::tests::plan::another_program_in_the_folder_keeps_the_change_and_its_record ... ok
test uninstall::tests::plan::keeping_the_binaries_keeps_the_path_change_and_says_so ... ok
test uninstall::tests::plan::without_a_record_the_marked_block_in_a_known_profile_is_found ... ok
test uninstall::tests::profile_exprs_cover_the_literal_and_home_relative_forms ... ok
test uninstall::tests::removing_the_block_restores_the_profile_as_it_was ... ok
test uninstall::tests::the_fish_block_and_the_literal_path_form_are_removed ... ok
test uninstall::tests::the_record_parses_each_kind_and_skips_blank_or_malformed_lines ... ok
test uninstall::tests::the_replacement_profile_is_never_more_readable_than_the_original ... ok
test uninstall::tests::the_review_lists_the_record_the_running_exe_and_the_path_changes ... ok
test uninstall::tests::the_running_executable_is_matched_whatever_case_windows_reports_it_in ... ok
test therock::version_proptests::retention_never_lets_a_plain_install_evict_its_devel_sibling ... ok
test therock::version_proptests::runtime_freshness_never_points_the_wrong_way ... ok
test therock::version_proptests::select_runtime_update_source_is_order_independent ... ok
test therock::version_proptests::select_startup_update_manifest_is_order_independent ... ok
test therock::version_proptests::select_runtimes_to_remove_is_order_independent ... ok
test therock::tests::wait_with_output_bounded_kills_a_slow_child_at_the_deadline ... ok
test remote::install::tests::resolvable_findings_on_their_own_do_promise_a_rerun ... ok
test therock::version_proptests::select_rocm_version_picks_the_newest_candidate ... ok
test therock::version_proptests::sorting_versions_is_order_independent ... ok
test therock::version_proptests::generator_reach_report ... ok
test advised_commands::every_advised_command_parses ... ok
test therock::version_proptests::select_rocm_version_is_order_independent ... ok
test tests::lemonade_stop_unloads_selected_model_over_http ... ok
test tests::services_prune_waits_for_the_managed_launch_lock_before_sweeping ... ok
test tests::serve_readiness_wait_withholds_ready_while_the_model_only_lists ... ok
test therock::archive_properties::sdk_tarball_unpack_never_writes_outside_the_install_root ... ok
test therock::tests::cli_family_overrides_a_stale_env_value_naming_an_unsupported_gpu ... ok
test therock::tests::env_override_is_ignored_end_to_end_without_the_opt_in ... ok
test therock::tests::extracting_the_sdk_archive_removes_it ... ok
test therock::tests::extracting_the_sdk_archive_reports_cleanup_failure ... ok
test therock::tests::next_layout_resolves_its_own_bases ... ok
test therock::tests::wait_with_output_bounded_waits_unbounded_when_no_timeout_is_given ... ok
test therock::tests::python_launcher_prefers_path_python_before_saved_managed_python ... ok
test therock::tests::python_launcher_prefers_path_python_over_managed_when_venv_capable ... ok
test therock::tests::render_update_json_installs_the_suppression_guard_around_resolution ... ok
test therock::tests::http_get_respects_max_time_secs ... ok
test therock::tests::metadata_signature_verification_accepts_generated_key_and_rejects_tamper ... ok
test comfyui::archive_properties::source_unpack_without_a_top_level_symlink_never_writes_outside_the_app_root ... ok
test comfyui::tests::reinstall_is_refused_while_comfyui_runs_until_it_is_stopped ... ok
test comfyui::tests::reinstall_is_refused_while_comfyui_starts_until_it_is_stopped ... ok
test tests::lemonade_stop_unload_is_bounded_by_the_request_timeout ... ok

failures:

---- comfyui::tests::status_reports_stopped_when_saved_comfyui_pid_is_gone stdout ----

thread 'comfyui::tests::status_reports_stopped_when_saved_comfyui_pid_is_gone' (2588554) panicked at apps/rocm/src/comfyui.rs:2758:9:
assertion failed: rendered.contains("status: stopped")
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace


failures:
    comfyui::tests::status_reports_stopped_when_saved_comfyui_pid_is_gone

test result: FAILED. 1309 passed; 1 failed; 7 ignored; 0 measured; 0 filtered out; finished in 5.17s

error: test failed, to rerun pass `-p rocm --bin rocm`

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Log tail default drift: lemonade returns 200 lines, protocol/vllm return 80

2 participants