Skip to content

perf(device): give the init SCPI exchange a reply to finish on instead of waiting out its timeout - #700

Merged
tylerkron merged 2 commits into
mainfrom
perf/init-scpi-error-queue-terminator
Aug 28, 2026
Merged

tylerkron merged 2 commits into
mainfrom
perf/init-scpi-error-queue-terminator

Conversation

@tylerkron

@tylerkron tylerkron commented Aug 28, 2026 •

Copy link
Copy Markdown
Contributor

What was wrong

Every connect to a device spent most of a second doing nothing.

The initialization sequence sends four SCPI setup commands, none of which the device answers unless
something goes wrong. So the text exchange had nothing to complete on and simply sat out its entire
1000 ms response timeout, every time, on a device that had finished its work almost immediately.

How it was fixed

Ask the device a question at the end, so there is an answer to finish on. Appending
SYSTem:ERRor? lets the wait loop leave its long first-response window as soon as the reply lands,
instead of waiting out silence. This is not a new idea here — SdCardOperations already uses exactly
this terminator to bound the SD directory listing (#396).

Three things a reviewer should push on:

The settle delay before the query is deliberate, and it costs 100 ms of the saving. It is what
makes the reply mean anything: the firmware has to have acted on SetProtobufStreamFormat and
queued any complaint about it before we ask, or a command that failed would answer 0,"No error".

The reply is read, not discarded. The error-queue form carries a bare code
(-200,"Execution error") with no ERROR token, so IsScpiErrorLine does not — and should not —
match it. The init retry check now also parses that code, which means a setup command that failed
silently, recording an error rather than volunteering one, is caught for the first time. Before
this, only the volunteered form was ever seen.

The completion window is 500 ms, and that is a deliberate divergence. The two other users of this
terminator both raise their window to 1000 ms, because a verdict that trails an echo by more than the
window would be lost. This path does not follow them, because the consequence differs: losing the
verdict there means an incomplete listing or failing a command the device accepted, whereas here it
means only that the error queue is not consulted — exactly where initialization stood before the
terminator existed, with the volunteered **ERROR: form still read either way. 1000 ms would also
erase the saving outright, since the exchange ends one full inactivity window after the last line.
The value is stated explicitly and pinned by a test rather than left to the 250 ms default.

An observe session (PreserveActiveStream) deliberately skips the terminator and keeps the old
timing. Reading the error queue pops the entry it returns, and taking one that belongs to the
session actually driving the device isn't worth half a second on the rarer path (#385).

Verified

Bench Nq1, fw 3.7.2, over USB/serial. Thirteen interleaved A/B pairs of connect + init + 1 s
stream, alternating between a CLI built from origin/main and one built from this branch —
interleaved because #667 records a previous timing claim that turned out to be a wash when measured
properly, and because this device's own run-to-run spread is comparable to the effect size:

mean median
origin/main 5.657 s 5.436 s
this branch 5.091 s 5.128 s
saving 0.566 s 0.544 s

12 of 13 pairs favour the branch; the one that does not is −0.187 s, within the device's noise.
Streaming and SD listing re-verified healthy on the branch afterwards.

Full suite green on net9.0 and net10.0 — 4,122 in Daqifi.Core.Tests, 217 in Daqifi.Mcp.Tests,
0 failures, Release build warning-clean. Seven new tests; the two covering error-queue detection were
confirmed to fail without the fix.

Scope

Refs #667 rather than closing it. That ticket has four items and this is item 2:

🤖 Generated with Claude Code

…d of waiting out its timeout

The four commands the init sequence sends produce no reply unless something
goes wrong, so the text exchange never had anything to complete on and sat
out its whole 1000ms response timeout on every single connect.

Appending SYSTem:ERRor? gives it a reply to finish on: the wait loop flips
into its 250ms completion window as soon as that lands. This is the same
terminator trick SdCardOperations already uses to bound the SD directory
listing (#396).

Measured on a bench Nq1 (fw 3.7.2, USB/serial), 5 interleaved A/B pairs of
connect + init + 1s stream:

  origin/main  mean 5.620s  median 5.426s
  this branch  mean 5.082s  median 4.900s
  delta        0.538s mean, 0.526s median; all 5 pairs favour the branch

The settle delay before the query is not optional -- the firmware has to
have acted on SetProtobufStreamFormat and queued any complaint before we
ask, or a failed command would answer "0, No error".

The reply is also read rather than discarded. The error-queue form carries a
bare code (-200,"Execution error") with no ERROR token, so IsScpiErrorLine
does not and should not match it; the init retry check now falls back to
parsing that code. A setup command that failed silently -- recorded in the
queue rather than volunteered -- was previously never noticed at all.

An observe session (PreserveActiveStream) deliberately forgoes the
terminator and keeps the old timing: reading the error queue pops the entry
it returns, and taking one that belongs to the session actually driving the
device is not worth 0.5s on the rarer path (#385).

Refs #667 -- item 1 landed already in #694, item 3 is a stated won't-do
pending the #485 single-reader rework, and item 4 remains open.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@tylerkron
tylerkron requested a review from a team as a code owner August 28, 2026 14:18
@tylerkron

Copy link
Copy Markdown
Contributor Author

/agentic_review

@qodo-code-review

Copy link
Copy Markdown

PR Summary by Qodo

Complete initialization SCPI exchange with an error-queue reply

🐞 Bug fix ✨ Enhancement 🧪 Tests 🕐 20-40 Minutes

Grey Divider

AI Description

• End normal initialization exchanges with a reply-producing system error query.
• Detect queued setup failures while accepting healthy zero-code responses.
• Preserve observer sessions by leaving their shared error queue untouched.
Diagram

sequenceDiagram
    actor Caller
    participant Core as DaqifiDevice
    participant Firmware as Device Firmware
    participant Parser as SCPI Classifier
    Caller->>Core: Initialize
    Core->>Firmware: Send setup commands
    Core->>Core: Await settle delay
    Core->>Firmware: Query error queue
    Firmware-->>Core: Return status code
    Core->>Parser: Classify reply
    alt Non-zero error
        Core->>Firmware: Retry setup once
        Core-->>Caller: Throw if persistent
    else Zero code
        Core-->>Caller: Continue initialization
    end
Loading
High-Level Assessment

Appending the existing system-error query to the same initialization exchange is the best fit because it supplies a deterministic completion reply and validates setup outcomes without introducing another round trip. A separate query exchange or a shorter fixed timeout would either add latency or remain timing-dependent; the deliberate settle delay and observer-session exemption address firmware processing and queue ownership safely.

Files changed (2) +129 / -1

Bug fix (1) +54 / -1
DaqifiDevice.csTerminate setup exchanges with a classified error-queue reply +54/-1

Terminate setup exchanges with a classified error-queue reply

• Adds a settle delay and system error query after normal initialization setup commands so the text exchange completes on a device reply instead of its timeout. Parses non-zero error-queue codes as setup failures while retaining volunteered-error detection and skips the query when preserving an active stream.

src/Daqifi.Core/Device/DaqifiDevice.cs

Tests (1) +75 / -0
DaqifiDeviceInitializeTests.csCover error-query termination and initialization outcomes +75/-0

Cover error-query termination and initialization outcomes

• Adds coverage that the system error query follows setup commands, zero-code replies succeed without retry, and non-zero queued errors produce the typed initialization exception after retry. Verifies observer sessions do not consume the shared device error queue.

src/Daqifi.Core.Tests/Device/DaqifiDeviceInitializeTests.cs

@qodo-code-review

qodo-code-review Bot commented Aug 28, 2026 •

Copy link
Copy Markdown

Code Review by Qodo

🐞 Bugs (1) 📘 Rule violations (0) 📜 Skill insights (0)

Grey Divider


Action required

1. Verdict window closes early ✓ Resolved 🐞 Bug ≡ Correctness
Description
Any echo or other setup output switches the exchange to its default 250 ms completion window, so the
newly appended SYSTem:ERRor? reply can arrive after collection has stopped despite the configured
1000 ms response timeout. Initialization then sees no failed queue reply and can mark a device ready
after a setup command failed silently.
Code

src/Daqifi.Core/Device/DaqifiDevice.cs[3333]

+                        Send(ScpiMessageProducer.GetSystemError);
Relevance

●●● Strong

PR #455 explicitly accepted the identical early-echo/default-250ms-window bug and added an explicit
longer completion timeout.

PR-#455

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
The query is last, while the async exchange overload defaults to a 250 ms inactivity timeout after
any response evidence. The administration path documents and tests the identical echo-before-verdict
failure mode and explicitly uses a 1000 ms completion timeout.

src/Daqifi.Core/Device/DaqifiDevice.cs[3285-3334]
src/Daqifi.Core/Device/DaqifiDevice.cs[1879-1904]
src/Daqifi.Core/Device/Internal/TextExchangeEngine.cs[633-666]
src/Daqifi.Core/Device/Internal/TextExchangeEngine.cs[739-755]
src/Daqifi.Core/Device/Internal/DeviceAdministrationOperations.cs[240-254]
src/Daqifi.Core.Tests/Device/Internal/DeviceAdministrationOperationsTests.cs[291-305]
PR-#455

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The trailing error-queue verdict can be omitted when earlier setup output starts the default 250 ms completion timer.

## Issue Context
This repository already gives command-plus-verdict exchanges an explicit completion window long enough for `SYSTem:ERRor?` to trail an echo.

## Fix Focus Areas
- src/Daqifi.Core/Device/DaqifiDevice.cs[3333-3334]
- src/Daqifi.Core.Tests/Device/DaqifiDeviceInitializeTests.cs[272-330]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools



Informational

2. Settle delay precedes write 🐞 Bug ☼ Reliability
Description
The new 100 ms delay starts after Send(SetProtobufStreamFormat) only enqueues the command, not
after its bytes reach the device. If the producer is delayed, it can write the format command and
error query back-to-back after the delay, allowing the query to return clean before the firmware
queues the setup failure that this change is intended to detect.
Code

src/Daqifi.Core/Device/DaqifiDevice.cs[R3330-3333]

+                        await Task.Delay(InitScpiSettleDelayMs, token).ConfigureAwait(false);
+                        token.ThrowIfCancellationRequested();
+
+                        Send(ScpiMessageProducer.GetSystemError);
Relevance

● Weak

Recent PR #591 explicitly rejected requiring outbound queue drain/write completion before exchange
boundaries; same async Send timing concern.

PR-#591

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
The changed comments say the delay is required for firmware to act before the query, but Send
returns before writing and the producer later drains all queued messages in one batch. There is no
write-completion wait between the format command and the delay.

src/Daqifi.Core/Device/DaqifiDevice.cs[1580-1587]
src/Daqifi.Core/Device/DaqifiDevice.cs[1658-1665]
src/Daqifi.Core/Device/DaqifiDevice.cs[3317-3333]
src/Daqifi.Core/Communication/Producers/MessageProducer.cs[199-222]
src/Daqifi.Core/Communication/Producers/MessageProducer.cs[255-275]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The final settle interval is measured from queue insertion rather than physical transmission, so it does not guarantee firmware processing time before the error query.

## Issue Context
`DaqifiDevice.Send` and `MessageProducer.Send` are explicitly fire-and-forget. Wait until the format command has actually been written before starting the settle delay, while preserving ordering and cancellation.

## Fix Focus Areas
- src/Daqifi.Core/Device/DaqifiDevice.cs[3317-3333]
- src/Daqifi.Core/Communication/Producers/MessageProducer.cs[199-222]
- src/Daqifi.Core.Tests/Device/DaqifiDeviceInitializeTests.cs[272-330]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


Grey Divider

Context sources
Review mode: ⚖️ Balanced

Grey Divider

Tip of the day
💡 Did you know, you can reply 'qodo' on any finding to push back, ask questions, or dig deeper

More tips ↗ | Customize Qodo ↗ | Qodo docs ↗

Grey Divider

Qodo Logo

Comment thread src/Daqifi.Core/Device/DaqifiDevice.cs
…aking the 250ms default

Qodo review, PR #700. The two existing users of the SYSTem:ERRor? terminator
-- the SD directory listing and the confirming-administration exchange --
both raise completionTimeoutMs to 1000ms, with a test pinning it, because
on a device that echoes it is an echo line that starts the inactivity clock
and a verdict trailing behind it would be missed. Taking the 250ms default
here was an unstated divergence from that convention.

Now stated explicitly at 500ms, with the reasoning recorded. It does not
follow the siblings to 1000ms because the consequence differs: losing the
verdict there means an incomplete listing or failing a command the device
accepted, while here it means only that the queue is not consulted -- which
is where initialization stood before the terminator existed, and the
volunteered **ERROR: form is still read either way. 1000ms would also erase
the saving entirely, since the exchange ends one inactivity window after the
last line.

Bench Nq1, 13 interleaved A/B pairs at this setting:
  origin/main  mean 5.657s   this branch  mean 5.091s
  saving       mean 0.566s, median 0.544s, 12/13 pairs favour the branch

The extra headroom over 250ms costs nothing measurable -- device variance
dominates it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@tylerkron

Copy link
Copy Markdown
Contributor Author

Re bug 2, "Settle delay precedes write" (marked Weak) — declining this one, for the reason the finding itself cites.

Send enqueues to MessageProducer rather than writing, so the 100ms delay does not start from the write. That is true, and it is equally true of the three settle delays that already sit between DisableDeviceEcho, StopStreaming, TurnDeviceOn and SetProtobufStreamFormat — this PR adds a fourth delay with exactly the same property, not a new one. If the timing model is wrong, it has been wrong for the whole sequence since well before this change, and tightening it is a separate piece of work on all four.

The commands also share one ordered producer queue, so SetProtobufStreamFormat is always written before SYSTem:ERRor? regardless of scheduling. The concern is therefore about firmware processing time between the two writes, which is the same thing the existing delays already assume.

And the cited precedent argues the other way: #591 explicitly rejected requiring an outbound-queue drain before an exchange boundary. TextExchangeEngine records the reasoning at length — draining first put the device's genuine terminator on the far side of the stale-line boundary, so it was discarded as stale and SYSTem:LOG? reported "the device did not answer" for a device that answered on time, 10/10 runs. Adding a drain here would re-introduce precisely that.

Bug 1 is accepted and fixed in 4d412c6 — see the inline thread.

@tylerkron

Copy link
Copy Markdown
Contributor Author

/agentic_review

@qodo-code-review

Copy link
Copy Markdown

Code review by qodo was updated up to the latest commit 4d412c6

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant