serve: the reasoning loop check also sees a loop written as one long word (#1753)

A periodic loop emitted as digits (a 24k-digit string cycling an 11-digit
pattern) is a single word to the word count and never one token twice
(the tokenizer has no multi-digit tokens), so both #606 and #728 passed
it and the run burned the whole reasoning budget.  The reasoning's last
2,048 chars are now checked for a period of at most 64 chars, and a
periodic tail reads as full coverage.  The opt-in path only: the default
path and the word count are unchanged.

Author: BaaRai

docs/DETAILS.md

@@ -746,7 +746,10 @@ print(r.choices[0].message.content)
 - **Repeated reasoning (opt-in, #728).** The single-token guard above does not see a model that repeats whole
   passages. `"reasoning_loop_recovery"` in `strata-<model>.json` is `false` (the default), `"stop"` or `"recover"`
   (`true` means `"recover"`). Every 512 output tokens, at a complete character and parser boundary, the reasoning is
-  measured over its last 2,000 words and punctuation marks (counting passages over the last 30,000 words). If at
+  measured over its last 2,000 words and punctuation marks (counting passages over the last 30,000 words), and its
+  last 2,048 characters are checked for a period of at most 64 characters (#1753: a loop written as one long word,
+  such as a 24k-digit string cycling an 11-digit pattern, is a single word to the count and never repeats one
+  token); a periodic tail counts as fully repeated. If at
   least 25% belong to 12-word passages seen three times, `"stop"` ends the reply there as `"length"` and says so in
   the server window. `"recover"` stops and drains that generation, then goes on once from all its generated token
   ids with the template's low-effort sentence in place of the xhigh one in the first system message (a splice of

serve/server.py

@@ -157,14 +157,23 @@ def write_temporary(path: Path, parts: list) -> None:
 LOOP_CHECK_EVERY = 512           # output tokens between two looks at the reasoning (at a clean parser boundary)
 LOOP_COVERAGE = 0.25             # the share of the last 2,000 words inside 12-word passages seen three times
 LOOP_HISTORY_WORDS = 30000       # how far back the passages are counted (bounds the cost of a look)
+LOOP_TAIL_CHARS = 2048           # chars at the end of the reasoning also checked for a short period (#1753)
+LOOP_TAIL_PERIOD = 64            # a period at most this long in that tail reads as a fully repeated window
 
 
 def reasoning_repeat_coverage(text):
     """Coverage of recent words by 12-word passages seen at least three times.
 
     Only reasoning is supplied. The history (the last LOOP_HISTORY_WORDS words) detects repeated verification passes
-    separated by long code drafts; the recent window excludes old repetitions.
+    separated by long code drafts; the recent window excludes old repetitions.  #1753: a loop can also be one long
+    word (a 24k-digit string cycling an 11-digit pattern): the word split makes it a single word and no word count
+    grows, and #606's token run never grows either (the tokenizer has no multi-digit tokens), so before the word
+    count the last LOOP_TAIL_CHARS chars are checked for a period of at most LOOP_TAIL_PERIOD, and a periodic tail
+    reads as full coverage: it is all of the recent output, repeated.
     """
+    tail = text[-LOOP_TAIL_CHARS:].lower()
+    if len(tail) == LOOP_TAIL_CHARS and any(tail[p:] == tail[:-p] for p in range(1, LOOP_TAIL_PERIOD + 1)):
+        return 1.0
     words = re.findall(r"\w+|[^\w\s]", text.lower())[-LOOP_HISTORY_WORDS:]
     width, window = 12, 2000
     if len(words) < window:

serve/test_reasoning_loop_recovery.py

@@ -36,6 +36,13 @@ def test_repetition_and_novel_work(self):
         self.assertEqual(reasoning_repeat_coverage(' '.join(str(i) for i in range(4000))), 0)
         self.assertGreater(reasoning_repeat_coverage(self.loop), .9)
 
+    def test_a_loop_written_as_one_long_word(self):
+        """#1753: a 24k-digit string cycling an 11-digit pattern is one word to the split and never one token twice."""
+        self.assertEqual(reasoning_repeat_coverage('Counting the digits of 9^104. ' + '58747693387' * 3000), 1)
+        counted = 'Digits: ' + ''.join(str(i) for i in range(1000))     # 2,893 chars that never repeat
+        self.assertEqual(reasoning_repeat_coverage(counted), 0)
+        self.assertEqual(reasoning_repeat_coverage('58747693387' * 100), 0)   # a tail too short to judge
+
     def test_user_instruction_is_never_rewritten(self):
         ids = self.tok.encode('<|im_start|>user\n' + HIGH_EFFORT, parse_special=True)
         self.assertIsNone(focused_recovery_prompt(self.tok, ids, []))
@@ -110,6 +117,17 @@ def test_stop_mode_ends_the_reply_at_the_loop(self):
         self.assertLess(events[-1][1]['completion_tokens'], 20000)
         self.assertEqual(events[-1][1]['reasoning_recoveries'], 0)
 
+    def test_stop_mode_ends_a_loop_written_as_one_long_word(self):
+        """#1753: the word count never sees the digits, so only the tail's period ends the reply."""
+        engine = RecordingEngine(self.tok, ['9^104 = ' + '58747693387' * 3000])
+        service = Service(engine, self.tok, None)
+        service.reasoning_loop_recovery = "stop"
+        events = list(service.run(self.ids, True, [], 40000, {}, threading.Event()))
+        self.assertEqual(len(engine.calls), 1)                   # no second pass
+        self.assertEqual(events[-1][1]['finish'], 'length')
+        self.assertLess(events[-1][1]['completion_tokens'], 5000)
+        self.assertEqual(events[-1][1]['reasoning_recoveries'], 0)
+
     def test_disabled_is_one_pass(self):
         engine, events = self.run_case(False)
         self.assertEqual(len(engine.calls), 1)