Fix #1782: Reward pipeline skips abandoned episodes — 98% of closed episodes never scored#1872
Open
Memtensor-AI wants to merge 1 commit into
Open
Fix #1782: Reward pipeline skips abandoned episodes — 98% of closed episodes never scored#1872Memtensor-AI wants to merge 1 commit into
Memtensor-AI wants to merge 1 commit into
Conversation
…escore - Add closeReason === 'abandoned' to episodeRewardIsDirty() check - Add 10-minute periodic timer in init() to rescore dirty episodes - Timer uses unref() to prevent blocking shutdown - Fixes #1782: 98% of closed episodes were skipped (219 of 224 had closeReason: 'abandoned') Before: Only 7 episodes scored on bootstrap (stale open episodes) After: All 219 abandoned episodes eligible for scoring
Memtensor-AI
requested review from
CarltonXiang,
MatthewZhuang and
syzsunshine219
Collaborator
Author
|
Collaborator
Author
✅ Automated Test Results: PASSED测试通过 (35/71)。memos_local_plugin/smoke: 0/1, memos_local_plugin/contract: 35/70。耗时 4s Branch: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Fixed reward pipeline bug that caused 98% of closed episodes to be skipped for scoring. The issue was caused by two interacting bugs in memory-core.ts:
episodeRewardIsDirty() excluded abandoned episodes: The function only matched closeReason === "finalized" or recoveryReason === "missed_session_end", but 219 of 224 closed episodes had closeReason === "abandoned". Added the abandoned check to line 1287.
No polling fallback after bootstrap: autoRescoreDirtyClosedEpisodes() was called once during init() with no retry mechanism. Added a 10-minute periodic timer (lines 927-937) with unref() to prevent blocking shutdown. The timer safely overlaps with the existing 30-second dedup guard in autoRescoreDirtyClosedEpisodes.
Impact: Before the fix, only 7 episodes were scored (stale open episodes recovered at startup). After the fix, all 219 abandoned episodes become eligible for scoring. The issue reporter confirmed that within 3 minutes of restart with this fix, scoring increased from 45 traces to 257 traces (212 freshly scored).
Changes: Single file modified (apps/memos-local-plugin/core/pipeline/memory-core.ts), 16 insertions, 1 deletion. The fix precisely matches the solution proposed in the issue description with exact code snippets verified.
Related Issue (Required): Fixes #1782
Type of change
Please delete options that are not relevant.
How Has This Been Tested?
Executor did not report tests.
Checklist
@MatthewZhuang, @CarltonXiang, @syzsunshine219 please review this PR.
Reviewer Checklist