Skip to content

feat(intune): discover and parse the Device Inventory Agent log family - #397

Merged
adamgell merged 18 commits into
mainfrom
codex/intune-device-inventory
Aug 3, 2026
Merged

adamgell merged 18 commits into
mainfrom
codex/intune-device-inventory

Conversation

@adamgell

@adamgell adamgell commented Jul 31, 2026 •

Copy link
Copy Markdown
Owner

Implements Device Inventory Agent log ingestion for #354 (epic #356).

12 commits, 26 files. Adds cmtraceopen_parser::intune::device::windows::inventory and wires it through detection, the dispatcher, known sources, file association, tailing, and the frontend.

What it does

The Device Inventory Agent writes several formats under C:\Program Files\Microsoft Device Inventory Agent\Logs, and all of them previously fell through to the generic timestamped or plain-text readers. Three dialects are now recognized by content, with the filename only able to lower the confidence threshold, never to decide on its own:

Dialect Files What was being lost
Harvester IntuneInventoryHarvesterLog.log + timestamped rotations [Information]/[Warning]/[Error] stayed in the message, and generic keyword severity could override the producer's own level — an explicit [Warning] containing the word "error" rendered as Error
Inventory Adaptor InventoryAdaptor.log, InventoryAdaptor.log_ Fell back to plain text: timestamp and PID lost, continuation JSON detached from the record that introduced it
Rotation failure ISO-8601 header + .NET exception Exception and at ... stack frames became unrelated physical records

English weekday/month tokens are parsed independently of the OS display locale. The literal trailing-underscore .log_ is treated as a supported rotation, not an unknown extension. Real-time tailing preserves logical records across appends (a 605-line change to watcher/tail.rs with pending-record state, debounce, bounded overflow, and flush on parser change/stop/disconnect).

Hardening applied before opening this PR

I audited the branch against the issue and fixed a panic plus four contract gaps in 1841eba7 rather than shipping them into review:

  • Panic in the dispatcher. Both parse_lines_with_selection and parse_content_with_selection called .expect("Device Inventory selection carries a dialect"). A selection with implementation IntuneDeviceInventory and specialization None panicked the whole parse. Unreachable via detection today, but a panic in the parser dispatcher is precisely what Harden all supported log parsers and add format contracts #310 hardening exists to prevent. Now degrades to the generic timestamped reading, with a regression test on both entry points.
  • Rotation-failure detection required the exact literal "Failed to rotate Device Inventory". Different wording, or an exception with no such literal, fell through. Now requires an ISO-8601 header plus either failure wording or .NET exception evidence (namespaced *Exception type, or an indented at stack frame).
  • Rotation-failure severity was unconditional — every ISO-8601 header became Error, so "Starting scheduled inventory rotation." read as a failure. Severity is now resolved after continuations attach, since the evidence is usually the exception beneath the header. Matching is deliberately narrow: a continuation merely containing "error" does not promote a record, which the issue calls out explicitly.
  • Declared-contract mismatches: group_label was "Device Inventory" (issue specifies "Device Inventory Agent"); tauri.conf.json never declared log_, so the installer did not register the extension even though file_association.rs had learned it; parser README table and CHANGELOG had no entry.

Verified

Run on this exact commit:

cargo test --locked -p cmtraceopen-parser            (TZ=UTC)
  lib 357 | esp_diagnostics 222 | intune_device_inventory 15 | 0 failed
cd src-tauri && cargo test                           -> 0 failed binaries
cargo clippy --locked -p cmtraceopen-parser --all-targets -- -D warnings   -> 0
cd src-tauri && cargo clippy --all-targets -- -D warnings                  -> 0
cargo check --locked -p cmtraceopen-parser --target wasm32-unknown-unknown -> Finished
npx tsc --noEmit                                     -> exit 0
cargo fmt --check --all    -> no diff in any file this branch touches
tauri.conf.json            -> parses as valid JSON

Focused suite went 11 -> 15 with the four hardening tests.

Known gaps — stated rather than left for review to find

  1. Harvester secondary bracketed context (e.g. [Registry]) is never mapped to component metadata; component is always None, and pure-parser matrix item 7 has no test.
  2. Whole-file continuation payloads are unbounded. parse_content appends continuations with no size limit; only the tail path is bounded by MAX_PENDING_LOGICAL_RECORD_BYTES. Matrix item 11 is unmet for the initial-parse path.
  3. Framing divergence between tail and initial parse. parse_content folds non-header lines into the previous harvester record, but the tail path declares Harvester as PhysicalLine. A continuation appended live becomes a detached record while the same file reopened merges it. The issue requires both paths to use the same framing contract.
  4. Timestamp handling. Harvester renders %m-%d-%Y, Adaptor renders %Y-%m-%d, so an aggregated folder load shows two date orders in one list. Harvester and Adaptor naive timestamps are stored as if UTC with timezone_offset: None, so aggregate ordering can be wrong off-UTC. Only the rotation dialect carries a real offset.
  5. Missing test coverage: collision cases (bracketed syslog-like timestamps without a PID; arbitrary JSON not selecting Adaptor), wire-name serialization stability for the camelCase intuneDeviceInventory* values src/types/log.ts depends on, UTF-8 vs CRLF equivalence, and aggregate-load native items 5/6.
  6. No structured access-denied error path (native matrix item 8).
  7. Fixtures carry no explicit synthetic-data marker, which the issue's privacy policy asks for. Content is synthetic, but unlabelled.
  8. Windows acceptance not performed. Verified on macOS; the #[cfg(windows)] registry and menu paths compile only in CI's Windows job.
  9. In the tail loop, the periodic-poll branch emits only when batch.reset || !batch.entries.is_empty(), so a batch with parse_errors > 0 and no entries is dropped (the stop and disconnect branches do check).

Items 1-4 and 9 are small. Item 5 is the bulk of the remaining work.

Refs #354

Summary by CodeRabbit

  • New Features
    • Added support for Microsoft Intune Device Inventory logs.
    • Recognizes Harvester, Inventory Adaptor, and rotation-failure formats.
    • Supports rotated .log_ files, source discovery, folder aggregation, and real-time tailing.
    • Displays severity, date/time, messages, and thread information.
  • Bug Fixes
    • Improved handling of split, continued, truncated, and oversized records.

adamgell and others added 12 commits July 31, 2026 11:33
A pre-PR audit of this branch against issue #354 found one panic path and
four contract gaps. Fixing them here rather than shipping them into review.

Panic in the dispatcher
-----------------------
Before: both `parse_lines_with_selection` and `parse_content_with_selection`
called `.expect("Device Inventory selection carries a dialect")`. Any
ParserSelectionInfo carrying implementation IntuneDeviceInventory with
specialization None panicked the entire parse. It was unreachable today
because detection always sets the dialect, but a panic in the parser
dispatcher is exactly what cf4fb41 ("Harden all supported log parsers")
exists to prevent, and a hand-built ResolvedParser or a future override
surface reaches it.

Now the arm degrades to the generic timestamped reading. All three dialects
are timestamp-led, so the fallback still produces usable records instead of
taking down the caller. The error-code-span pass was rewritten from a
`matches!` on the implementation to a check that mirrors the new guard, so a
dialect-less selection is not annotated twice.

Rotation-failure detection was too narrow
-----------------------------------------
Before: `has_rotation_failure_signature` required the exact literal
"Failed to rotate Device Inventory". A real artifact with different rotation
wording, or a .NET exception with no such literal, fell through to the
generic timestamped parser.

The issue asks for rotation-failure OR .NET exception evidence. Detection now
requires an ISO-8601 header plus either a header that states a rotation
failure or a .NET exception continuation beneath one — a namespaced
`*Exception` type or an indented `at ` stack frame. A file of plain ISO-8601
records with neither is still not this dialect.

Rotation-failure severity was unconditional
-------------------------------------------
Before: `parse_rotation` stamped Severity::Error on every ISO-8601 header in
the file, so an ordinary "Starting scheduled inventory rotation." record read
as an error.

The issue requires Error only when the header or exception content identifies
a failure. Severity is now resolved in `parse_content` after continuations are
attached, because the failure evidence is usually the exception beneath the
header rather than the header text. The matching is deliberately narrow: a
continuation that merely contains the word "error" does not promote a record,
which the issue calls out explicitly.

Declared contract mismatches
----------------------------
* known_sources group_label was "Device Inventory"; the issue specifies
  "Device Inventory Agent". Nothing depended on the old string (verified by
  grep across src/, src-tauri/, e2e/).
* tauri.conf.json bundle.fileAssociations declared only log / lo_ / cmtlog,
  so the NSIS/WiX installer never registered the producer's literal `.log_`
  rotation extension even though file_association.rs had learned it. Added.
* The parser crate README's supported-format table and the CHANGELOG
  [Unreleased] section had no Device Inventory entry.

Tests
-----
Four new tests, 11 -> 15 in the focused suite:
* a dialect-less selection degrades instead of panicking, on both the
  line and whole-content entry points (regression test for the .expect);
* rotation-failure is detected from exception evidence when the header uses
  different wording;
* severity comes from failure evidence, not from every header (mixed file:
  Info / Error / Info);
* a generic keyword in a continuation does not promote a record to Error.

Verified on this commit:
  cargo test --locked -p cmtraceopen-parser   (TZ=UTC)
    -> lib 357, esp_diagnostics 222, intune_device_inventory 15, 0 failed
  cd src-tauri && cargo test          -> 0 failed binaries
  cargo clippy --locked -p cmtraceopen-parser --all-targets -- -D warnings
    -> 0 issues
  cd src-tauri && cargo clippy --all-targets -- -D warnings   -> 0 issues
  cargo check --locked -p cmtraceopen-parser --target wasm32-unknown-unknown
    -> Finished
  npx tsc --noEmit                    -> exit 0
  cargo fmt --check --all             -> no diff in any file this branch touches
  tauri.conf.json                     -> parses as valid JSON

Refs #354

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings July 31, 2026 15:41
@coderabbitai

coderabbitai Bot commented Jul 31, 2026 •

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

The pull request adds Intune Device Inventory ingestion for three log dialects. It connects parsing, dispatch, logical-record tailing, Windows discovery, rotated-file associations, native tests, frontend metadata, and documentation.

Changes

Intune Device Inventory support

Layer / File(s) Summary
Parser contracts and dialect implementation
docs/superpowers/..., crates/cmtraceopen-parser/src/intune/..., crates/cmtraceopen-parser/src/models/log_entry.rs, crates/cmtraceopen-parser/tests/intune_device_inventory.rs
Adds dialect detection, logical-record framing, parsing, metadata, and parser tests.
Detection and parser dispatch
crates/cmtraceopen-parser/src/parser/...
Routes recognized dialects through specialized parser selections and multiline or whole-content parsing paths.
Incremental logical-record tailing
src-tauri/src/watcher/tail.rs
Buffers split records, attaches continuations, flushes pending state, bounds overflow, and reports parse errors.
Windows sources and native integration
src-tauri/src/commands/..., src-tauri/tests/..., src-tauri/tauri.conf.json
Adds known sources, .log_ associations, parser descriptions, fixtures, and native contract coverage.
Frontend metadata and documentation
src/types/log.ts, src/stores/log-store.ts, src/lib/..., CHANGELOG.md, crates/cmtraceopen-parser/README.md
Adds parser values, display mappings, source-loading tests, and documentation.

Estimated code review effort: 5 (Critical) | ~120 minutes

Possibly related issues

Possibly related PRs

Suggested labels: test

🚥 Pre-merge checks | ✅ 4
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title follows Conventional Commits and accurately describes the Device Inventory Agent ingestion changes.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/intune-device-inventory

Comment @coderabbitai help to get the list of available commands.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds first-class support for the Microsoft Device Inventory Agent log family across the pure parser crate, the Tauri backend, and the React frontend, so these logs are detected and parsed by dedicated dialect-aware logic rather than falling back to generic timestamp/plain-text parsing.

Changes:

  • Introduces a new IntuneDeviceInventory parser kind with three dialect specializations (Harvester / Inventory Adaptor / Rotation Failure), with fixture-backed contracts in both the parser crate and the native app corpus tests.
  • Wires the new parser through selection metadata/serialization, frontend labels/types, and column configuration.
  • Adds Windows known-sources metadata for the Device Inventory Agent folder + primary files, registers .log_, and enhances tailing to preserve logical records across appends for logical-record dialects.

Reviewed changes

Copilot reviewed 25 out of 26 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
src/types/log.ts Extends serialized parser unions with intuneDeviceInventory and its specializations.
src/stores/log-store.ts Adds UI labels for the new parser kind/implementation.
src/lib/column-config.ts Defines default column set for intuneDeviceInventory.
src/lib/log-source.test.ts Adds Vitest coverage for Device Inventory known-source routing (folder vs direct-file behavior).
crates/cmtraceopen-parser/src/models/log_entry.rs Adds new ParserKind/ParserImplementation and dialect specializations.
crates/cmtraceopen-parser/src/intune/mod.rs Exposes the new intune::device namespace.
crates/cmtraceopen-parser/src/intune/device/mod.rs Adds Intune device family module root.
crates/cmtraceopen-parser/src/intune/device/windows/mod.rs Adds Windows device module root.
crates/cmtraceopen-parser/src/intune/device/windows/inventory.rs Implements dialect detection, parsing, and logical-record framing helpers.
crates/cmtraceopen-parser/src/parser/detect.rs Integrates dialect detection and adds a ResolvedParser::intune_device_inventory constructor.
crates/cmtraceopen-parser/src/parser/mod.rs Dispatch integration + dialect-safe degradation behavior.
crates/cmtraceopen-parser/src/parser/ccm.rs Defensive specialization handling in CCM content parsing.
crates/cmtraceopen-parser/tests/intune_device_inventory.rs Pure-parser + dispatcher contract tests and collision coverage.
crates/cmtraceopen-parser/README.md Documents the newly supported parser kind in the supported-formats table.
src-tauri/tests/parser_supported_formats.rs Adds the new parser kind to the supported-format contract and adds dialect fixtures.
src-tauri/tests/parser_regression_corpus.rs Adds corpus-level regression assertions for selection/framing/message assembly.
src-tauri/tests/corpus/intune_device_inventory/clean/IntuneInventoryHarvesterLog.log Synthetic harvester fixture for app corpus.
src-tauri/tests/corpus/intune_device_inventory/clean/InventoryAdaptor.log_ Synthetic adaptor fixture for app corpus.
src-tauri/tauri.conf.json Declares .log_ as a handled extension for bundling/association metadata.
src-tauri/src/commands/known_sources.rs Adds Windows Intune Device Inventory known sources + tests.
src-tauri/src/commands/file_association.rs Adds .log_ to the Windows association set + test to keep extensions unique.
src-tauri/src/commands/bundle_ops.rs Updates selection description strings and adds tests for Device Inventory descriptions.
src-tauri/src/watcher/tail.rs Adds pending logical-record state + framing integration for Device Inventory logical dialects and expands tail tests.
docs/superpowers/specs/2026-07-30-intune-device-inventory-ingestion-design.md Adds an approved design spec for the ingestion/parsing contract and test matrix.
docs/superpowers/plans/2026-07-30-intune-device-inventory-ingestion.md Adds an implementation plan document for the work breakdown and verification gates.
CHANGELOG.md Adds an Unreleased entry describing the new Device Inventory support.
Suppressed comments (1)

src-tauri/src/watcher/tail.rs:602

  • Similarly, the watcher-disconnect path handles parse_errors > 0, but the periodic poll branch just above still drops batches that contain only parse errors. This can hide parse failures or overflow conditions when no valid entries were produced in that poll interval.
                    let batch = tail_reader.flush_pending_logical_record();
                    if !batch.entries.is_empty() || batch.parse_errors > 0 {
                        on_new_entries(batch);
                    }
                    log::info!("Watcher channel disconnected");

Comment thread src-tauri/src/watcher/tail.rs Resolved
adamgell added a commit that referenced this pull request Jul 31, 2026
The workflow added in #383 has never succeeded on a pull request. All three
pull_request_target runs so far failed with:

  POST /repos/adamgell/cmtraceopen/issues/<n>/labels
  403 Resource not accessible by integration

The script computes the labels correctly — the failing run for #397 had
body '{"labels":["enhancement","intune","parser","device","feature"]}' — and
then cannot write them.

Cause: labelling a pull request requires `pull-requests: write`, even though
the REST path is under /issues. The workflow declared only `issues: write`,
and because an explicit `permissions:` block replaces the repository default
rather than adding to it, and this repo's default_workflow_permissions is
`read`, the token had no pull-request scope at all. The issues: opened trigger
was unaffected.

Adding `pull-requests: write` is the minimum needed; issue labelling keeps
working through the existing `issues: write`.

Verified: `gh run list --workflow=label-new-items.yml` shows failure on all
three pull_request_target runs to date (#397, and the two before it), and
`gh api repos/:owner/:repo/actions/permissions/workflow` reports
default_workflow_permissions=read.

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
@adamgell

Copy link
Copy Markdown
Owner Author

Code review

Found 1 issue:

  1. RotationFailure detection is overbroad and claims unrelated ISO-8601 logs. rotation_header_states_failure matches any message that merely starts_with("failed ") or starts_with("unhandled "), and has_rotation_failure_signature needs only one matching line anywhere in the file — rotation_re() is the generic ^\d{4}-\d{2}-\d{2}T[^ ]+ (?P<message>.*)$. Harvester and Adaptor require two headers, or one plus a filename hint; this dialect requires neither. Because inventory::detect_dialect is called at detect.rs#L447-L450 ahead of every other content detector, a line such as 2026-07-30T10:15:00Z Failed to load configuration. in any unrelated ISO-8601 log routes the whole file to the Device Inventory parser. This was widened from the previous exact-literal check in 1841eba and overshot. The existing negative test only covers "Completed an unrelated service operation.", so it passes without exercising the starts_with arms.

/// happens to contain a generic keyword, so this does not match a bare "error".
fn rotation_header_states_failure(message: &str) -> bool {
let lowered = message.trim().to_ascii_lowercase();
lowered.contains("failed to rotate")
|| lowered.contains("rotation failed")
|| lowered.contains("failed to roll")
|| lowered.starts_with("failed ")
|| lowered.starts_with("unhandled ")
}

Suggested direction: drop the bare starts_with("failed ") / starts_with("unhandled ") arms, keep the rotation-specific phrases, and require the .NET exception evidence to be a continuation of the header record rather than anywhere in the file.

Five other findings were raised and scored below the reporting threshold, so they are not filed here: Harvester detection accepting any two M/D/YYYY h:mm:ss AM/PM [Level] lines with no header or path signal (75), detect_dialect scanning unbounded content and bypassing the 20-line sample window documented in detect.rs (75), a .expect on pid.parse::<u32>() that panics on a PID above u32::MAX (75), the RecordFraming::PhysicalLine vs whole-file folding divergence for Harvester (75, already disclosed in the PR body), and TailBatch.parse_errors never reaching TailPayload (55, partly disclosed).

🤖 Generated with Claude Code

- If this code review was useful, please react with 👍. Otherwise, react with 👎.

@adamgell adamgell added device Device management related enhancement New feature or request feature New feature intune Microsoft Intune related parser Log parser related labels Jul 31, 2026
Copilot review on PR #397 flagged three contract gaps in the Device
Inventory work.

The harvester dialect reported RecordFraming::PhysicalLine while the
parser attached a non-header line to the record above it. Whole-file
parsing merged a continuation, tailing emitted it as a detached record,
and the reported framing misdescribed the parse for consumers. All three
dialects now report LogicalRecord and the tail frames harvester input
with the same helper the initial parse uses.

Incremental Device Inventory parsing rejoined its batch into one String
on every read. The parser now exposes parse_lines beside parse_content
over a shared core, so callers that already hold lines skip the copy.

The tail watcher emitted a batch only when it carried a reset or
entries, so a batch that only reported parse errors was dropped until
the session stopped. All four emission sites now share
TailBatch::is_reportable, which counts parse errors.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The plan's resolved-parser snippet still selected PhysicalLine for the
harvester dialect after the constructor moved every dialect to
LogicalRecord.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 25 out of 26 changed files in this pull request and generated no new comments.

Suppressed comments (3)

src/lib/log-source.test.ts:116

  • The direct-file routing test uses a single parseResult whose filePath is always the harvester path, and openLogSourceFile is mocked to always return that same result. This means the it.each(deviceInventoryFiles) case doesn’t actually validate that each known-source file returns its own selected path (the adaptor case would still pass even if the implementation opened the wrong file).
    commands.parseFilesBatch.mockResolvedValue([parseResult]);
    commands.openLogSourceFile.mockResolvedValue(parseResult);
    commands.stopTail.mockResolvedValue(undefined);

src-tauri/tests/parser_supported_formats.rs:323

  • ParserKind::IntuneDeviceInventory is now part of the supported-formats contract, but this file only exercises the Harvester and Adaptor specializations. There’s no corresponding contract test/fixture for the RotationFailure specialization, so a regression in detection/framing/severity for that dialect wouldn’t be caught at the app-level supported-format boundary.
#[test]
fn text_contract_intune_device_inventory_harvester() {
    let (result, selection) =
        parse_fixture("intune_device_inventory/clean/IntuneInventoryHarvesterLog.log");
    assert_selection(

src-tauri/tests/parser_regression_corpus.rs:734

  • The app regression corpus test for Device Inventory currently covers only Harvester + Adaptor. Since the parser also implements the RotationFailure dialect (with a distinct detection signature and severity rules), it should be represented in the corpus snapshot too; otherwise the most failure-prone dialect isn’t protected against app-level regression.
#[test]
fn intune_device_inventory_corpus_selects_and_frames_harvester_and_adaptor_records() {
    let harvester = parse_fixture("intune_device_inventory/clean/IntuneInventoryHarvesterLog.log");
    assert_parsed_selection(
        &harvester,

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 22 out of 23 changed files in this pull request and generated no new comments.

Suppressed comments (3)

crates/cmtraceopen-parser/src/intune/device/windows/inventory/mod.rs:373

  • has_rotation_failure_signature currently returns true for any file that contains (1) an ISO-8601 header line and (2) any subsequent .NET exception/stack-frame line, even if the header message has no rotation/inventory context. This can misclassify unrelated ISO logs that include exceptions as the Device Inventory rotation-failure dialect and bypass the generic timestamped parser. Consider requiring rotation/inventory context on the header when using exception-only evidence.
fn has_rotation_failure_signature(content: &str) -> bool {
    let mut saw_header = false;
    let mut header_states_failure = false;
    let mut exception_under_header = false;

crates/cmtraceopen-parser/src/intune/device/windows/inventory/mod.rs:331

  • rotation_header_states_failure currently treats any ISO-8601 header whose message starts with "failed " or "unhandled " as rotation failure evidence. Those prefixes are not rotation-specific and can cause false-positive selection of the Device Inventory rotation-failure dialect for unrelated ISO logs (contradicting the design intent that generic ISO logs should remain on the generic timestamped parser).

This issue also appears on line 369 of the same file.

    lowered.contains("failed to rotate")
        || lowered.contains("rotation failed")
        || lowered.contains("failed to roll")
        || lowered.starts_with("failed ")
        || lowered.starts_with("unhandled ")

crates/cmtraceopen-parser/src/intune/device/windows/inventory/mod.rs:281

  • In parse_records, continuation lines are appended to the previous record's message with no size bound. A single record with a very large continuation payload (e.g., multi-MB JSON) can grow the in-memory String without limit during whole-file parsing, risking high memory usage or OOM for large artifacts. The tail path has a 1 MiB bound, so this also makes initial-parse vs tail behavior inconsistent under oversized continuations.
        } else if let Some(previous) = records.last_mut() {
            previous.message.push('\n');
            previous.message.push_str(line);
        } else {
            records.push(orphan_record(line, line_number));

Follow-up to 8570192, found while re-reading that diff. Making the
framed record's physical span load-bearing turned an existing `retain`
into a line-numbering bug: it dropped empty records from the flush path,
and dropping the record now also drops the physical line it accounts for.

A blank line framed on its own parses to no entries either way, so the
retain used to be a harmless optimisation. It is not harmless now. The
empty record is kept, which costs one no-op parse call and keeps the
line accounting exact.

Test, failing before the fix with `left: 1, right: 2`:
`test_tail_reader_counts_a_flushed_blank_line_as_a_physical_line` tails a
blank first line, lets it flush, then appends a harvester header and
asserts that record is numbered at physical line 2.

Refs #354

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@adamgell
adamgell requested a review from Copilot August 1, 2026 14:36
@adamgell

adamgell commented Aug 1, 2026

Copy link
Copy Markdown
Owner Author

@coderabbitai full review

@coderabbitai

coderabbitai Bot commented Aug 1, 2026 •

Copy link
Copy Markdown
✅ Action performed

Full review finished.


Your included review limit is currently reached under our Fair Usage Limits Policy. This review may still proceed through usage-based billing if eligible. Your next included review will be available in 43 minutes.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Nitpick comments (1)
src-tauri/src/commands/known_sources.rs (1)

98-136: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Assert the file-source patterns and source order.

The test does not verify the Harvester and Adaptor active-file patterns, rotated-file patterns, or source_order values. A later change can remove rotated-file discovery while this test still passes. Assert both patterns and the expected orders 20 and 30.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src-tauri/src/commands/known_sources.rs` around lines 98 - 136, Extend the
known-source assertions for the Harvester and Adaptor entries in the existing
source lookup loop to validate both active-file and rotated-file patterns, and
assert their expected source_order values of 20 and 30 respectively. Keep the
existing path_kind, default_path, and grouping checks intact, using the matched
source’s known-source fields for these additional assertions.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@crates/cmtraceopen-parser/README.md`:
- Line 118: Add IntuneDeviceInventory.log to the IntuneDeviceInventory source
entry in the README supported-format table, preserving the existing Harvester,
Adaptor, and rotation filename entries.

In `@src-tauri/src/watcher/tail.rs`:
- Around line 342-343: Update the filtering logic before parse_logical_records
so empty records contribute their physical_lines span to next_line instead of
being discarded from accounting. Preserve non-empty records for parsing, and
ensure blank records produced after a flush advance line numbering consistently
with the whole-file parse_records path.

---

Nitpick comments:
In `@src-tauri/src/commands/known_sources.rs`:
- Around line 98-136: Extend the known-source assertions for the Harvester and
Adaptor entries in the existing source lookup loop to validate both active-file
and rotated-file patterns, and assert their expected source_order values of 20
and 30 respectively. Keep the existing path_kind, default_path, and grouping
checks intact, using the matched source’s known-source fields for these
additional assertions.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 3e81569d-2113-4b6c-95ee-32be3679a594

📥 Commits

Reviewing files that changed from the base of the PR and between 88e9f13 and 8570192.

⛔ Files ignored due to path filters (1)
  • src-tauri/tests/corpus/intune_device_inventory/clean/IntuneInventoryHarvesterLog.log is excluded by !**/*.log
📒 Files selected for processing (22)
  • CHANGELOG.md
  • crates/cmtraceopen-parser/README.md
  • crates/cmtraceopen-parser/src/intune/device/windows/inventory/mod.rs
  • crates/cmtraceopen-parser/src/models/log_entry.rs
  • crates/cmtraceopen-parser/src/parser/ccm.rs
  • crates/cmtraceopen-parser/src/parser/detect.rs
  • crates/cmtraceopen-parser/src/parser/mod.rs
  • crates/cmtraceopen-parser/tests/intune_device_inventory.rs
  • docs/superpowers/plans/2026-07-30-intune-device-inventory-ingestion.md
  • docs/superpowers/specs/2026-07-30-intune-device-inventory-ingestion-design.md
  • src-tauri/src/commands/bundle_ops.rs
  • src-tauri/src/commands/file_association.rs
  • src-tauri/src/commands/known_sources.rs
  • src-tauri/src/watcher/tail.rs
  • src-tauri/tauri.conf.json
  • src-tauri/tests/corpus/intune_device_inventory/clean/InventoryAdaptor.log_
  • src-tauri/tests/parser_regression_corpus.rs
  • src-tauri/tests/parser_supported_formats.rs
  • src/lib/column-config.ts
  • src/lib/log-source.test.ts
  • src/stores/log-store.ts
  • src/types/log.ts

Comment thread crates/cmtraceopen-parser/README.md Resolved
Comment thread src-tauri/src/watcher/tail.rs Resolved

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 22 out of 23 changed files in this pull request and generated no new comments.

Suppressed comments (10)

crates/cmtraceopen-parser/src/intune/device/windows/inventory/mod.rs:412

  • parse_harvester maps severity by exact string match, which conflicts with the documented case-insensitive level contract (and the regex can be made case-insensitive). Normalize the captured level before mapping so [warning] / [WARNING] still parse correctly.
    let message = caps.name("message").expect("message capture").as_str();
    let severity = match caps.name("level").expect("level capture").as_str() {
        "Information" => Severity::Info,
        "Warning" => Severity::Warning,
        "Error" => Severity::Error,

src-tauri/src/watcher/tail.rs:1071

  • This sleep duration is only 25ms above LOGICAL_RECORD_DEBOUNCE (250ms). On busy/low-granularity CI runners, sleep(275ms) can wake early enough to make the test flaky. Prefer sleeping based on the debounce constant with a larger cushion.
        std::thread::sleep(std::time::Duration::from_millis(275));

src-tauri/src/watcher/tail.rs:1159

  • This sleep duration is only 25ms above LOGICAL_RECORD_DEBOUNCE (250ms). On busy/low-granularity CI runners, sleep(275ms) can wake early enough to make the test flaky. Prefer sleeping based on the debounce constant with a larger cushion.
        std::thread::sleep(std::time::Duration::from_millis(275));

src-tauri/src/watcher/tail.rs:1181

  • This sleep duration is only 25ms above LOGICAL_RECORD_DEBOUNCE (250ms). On busy/low-granularity CI runners, sleep(275ms) can wake early enough to make the test flaky. Prefer sleeping based on the debounce constant with a larger cushion.
        std::thread::sleep(std::time::Duration::from_millis(275));

src-tauri/src/watcher/tail.rs:1229

  • This sleep duration is only 25ms above LOGICAL_RECORD_DEBOUNCE (250ms). On busy/low-granularity CI runners, sleep(275ms) can wake early enough to make the test flaky. Prefer sleeping based on the debounce constant with a larger cushion.
        std::thread::sleep(std::time::Duration::from_millis(275));

src-tauri/src/watcher/tail.rs:1314

  • This sleep duration is only 25ms above LOGICAL_RECORD_DEBOUNCE (250ms). On busy/low-granularity CI runners, sleep(275ms) can wake early enough to make the test flaky. Prefer sleeping based on the debounce constant with a larger cushion.
        std::thread::sleep(std::time::Duration::from_millis(275));

crates/cmtraceopen-parser/src/intune/device/windows/inventory/mod.rs:162

  • Harvester level tokens are documented as case-insensitive, but the regex only matches [Information|Warning|Error] with exact casing. This will fail to detect/parse real logs if the producer ever varies casing (and also prevents content-first detection).

This issue also appears on line 408 of the same file.

fn harvester_re() -> &'static Regex {
    static CELL: OnceLock<Regex> = OnceLock::new();
    CELL.get_or_init(|| {
        Regex::new(r"^(?P<date>\d{1,2}/\d{1,2}/\d{4}) (?P<time>\d{1,2}:\d{2}:\d{2} [AP]M) \[(?P<level>Information|Warning|Error)\] (?P<message>.*)$")
            .expect("Device Inventory Harvester regex must compile")

crates/cmtraceopen-parser/src/intune/device/windows/inventory/mod.rs:389

  • Rotation-failure detection treats any .NET exception line after the first ISO-8601 header as sufficient evidence, even if the header/messages contain no Device Inventory / rotation context. This can misclassify generic ISO-8601 logs with exception stacks as the Device Inventory rotation dialect (the design doc explicitly calls this collision out).
        } else if saw_header && looks_like_dotnet_exception(line) {
            exception_under_header = true;
        }

src-tauri/src/watcher/tail.rs:1011

  • This sleep duration is only 25ms above LOGICAL_RECORD_DEBOUNCE (250ms). On busy/low-granularity CI runners, sleep(275ms) can wake early enough to make the test flaky. Prefer sleeping based on the debounce constant with a larger cushion.

This issue also appears in the following locations of the same file:

  • line 1071
  • line 1159
  • line 1181
  • line 1229
  • line 1314
        std::thread::sleep(std::time::Duration::from_millis(275));

src-tauri/tests/parser_supported_formats.rs:360

  • App-level supported-format contracts cover Harvester and Adaptor dialects, but not the RotationFailure dialect (specialization IntuneDeviceInventoryRotationFailure). Adding a small synthetic rotation-failure fixture + contract here would help prevent selection/framing regressions for that dialect at the application layer.
#[test]
fn text_contract_intune_device_inventory_adaptor() {
    let (result, selection) = parse_fixture("intune_device_inventory/clean/InventoryAdaptor.log_");
    assert_selection(
        &selection,
        ParserKind::IntuneDeviceInventory,
        ParserImplementation::IntuneDeviceInventory,
        RecordFraming::LogicalRecord,
    );
    assert_eq!(
        selection.specialization,
        Some(ParserSpecialization::IntuneDeviceInventoryAdaptor)
    );
    assert_eq!(result.format_detected, LogFormat::Timestamped);
    assert_eq!(result.parse_errors, 0);
    assert_eq!(result.entries.len(), 2);
    assert_eq!(result.entries[0].thread, Some(8604));
    assert_eq!(
        result.entries[0].message,
        "Adapter result:\n{\"Status\":200,\"HResult\":\"0x00000000\",\"Data\":{\"Example\":\"value\"}}"
    );
}

@coderabbitai coderabbitai Bot added the windows Windows platform related label Aug 1, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

♻️ Duplicate comments (1)
crates/cmtraceopen-parser/README.md (1)

118-118: 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

List IntuneDeviceInventory.log in the supported-format table.

The rotation-failure dialect uses IntuneDeviceInventory.log, but this row lists only the Harvester and Adaptor names. Add the filename so users can identify all supported Device Inventory sources.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@crates/cmtraceopen-parser/README.md` at line 118, Update the
IntuneDeviceInventory row in the supported-format table to include
IntuneDeviceInventory.log alongside the existing Harvester, Adaptor, and rotated
.log_ filenames.
🧹 Nitpick comments (3)
crates/cmtraceopen-parser/src/parser/ccm.rs (1)

141-152: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Document why Device Inventory specializations reach the CCM multiline path.

The three Device Inventory specializations now share the None arm, which reads as "a Device Inventory selection parses as a CCM multiline record". That pairing cannot occur today: ResolvedParser::intune_device_inventory always sets ParserImplementation::IntuneDeviceInventory, and parser/mod.rs only calls ccm::parse_content under ParserImplementation::Ccm.

The arm exists because ParserSpecialization is matched exhaustively here. Keep the exhaustive match, and state the reason so a later reader does not read it as intended routing.

♻️ Proposed clarification
         None
+        // A Device Inventory specialization never pairs with
+        // `ParserImplementation::Ccm`, so these variants are listed only to
+        // keep this match exhaustive. If one ever arrives here it came from a
+        // hand-built `ResolvedParser`, and the CCM multiline reading is the
+        // same conservative fallback the `None` case already uses.
         | Some(ParserSpecialization::IntuneDeviceInventoryHarvester)
         | Some(ParserSpecialization::IntuneDeviceInventoryAdaptor)
         | Some(ParserSpecialization::IntuneDeviceInventoryRotationFailure) => {
             parse_content_multiline(content, file_path)
         }
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@crates/cmtraceopen-parser/src/parser/ccm.rs` around lines 141 - 152, Add a
concise comment to the exhaustive match in the CCM parser explaining that the
three Device Inventory specializations share this arm only because
ParserSpecialization must be matched exhaustively, not because they are routed
through CCM in normal operation. Keep the existing match arms and parsing
behavior unchanged.
crates/cmtraceopen-parser/src/parser/mod.rs (1)

168-204: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Resolve the dialect once instead of three times.

device_inventory_dialect(selection.specialization) is called in the match guard on Line 176, again inside the arm on Line 181, and a third time on Line 198. The third call recomputes the same predicate that the match arm already decided, and the second call needs .expect("guarded by the match arm above") only because the guard cannot bind its result.

Bind the dialect once before the match. That removes the expect and the recomputation, and it makes used_whole_content_path read directly off the branch that was taken.

♻️ Proposed refactor
     let total_lines = content.lines().count() as u32;
+    let device_inventory_dialect = device_inventory_dialect(selection.specialization);
+    let mut used_whole_content_path = matches!(
+        selection.implementation,
+        crate::models::log_entry::ParserImplementation::Ccm
+    );
     let (mut entries, parse_errors) = match selection.implementation {
         crate::models::log_entry::ParserImplementation::Ccm => {
             ccm::parse_content(content, file_path, selection.specialization)
         }
         // See the matching arm in `parse_lines_with_selection`: a Device
         // Inventory selection missing its dialect degrades to the generic
         // timestamped reading rather than panicking.
         crate::models::log_entry::ParserImplementation::IntuneDeviceInventory
-            if device_inventory_dialect(selection.specialization).is_some() =>
+            if device_inventory_dialect.is_some() =>
         {
-            inventory::parse_content(
-                file_path,
-                content,
-                device_inventory_dialect(selection.specialization)
-                    .expect("guarded by the match arm above"),
-            )
+            used_whole_content_path = true;
+            let Some(dialect) = device_inventory_dialect else {
+                unreachable!("guarded by the match arm above")
+            };
+            inventory::parse_content(file_path, content, dialect)
         }
         _ => {
             let lines: Vec<&str> = content.lines().collect();
             parse_lines_with_selection(&lines, file_path, selection)
         }
     };
     // For whole-content parse paths (which don't go through
     // parse_lines_with_selection), ensure error code spans are annotated. The
     // Device Inventory arm only takes that path when it resolved a dialect; a
     // dialect-less selection fell through to parse_lines_with_selection, which
     // already annotated.
-    let used_whole_content_path = match selection.implementation {
-        crate::models::log_entry::ParserImplementation::Ccm => true,
-        crate::models::log_entry::ParserImplementation::IntuneDeviceInventory => {
-            device_inventory_dialect(selection.specialization).is_some()
-        }
-        _ => false,
-    };
     if used_whole_content_path {

A simpler variant keeps the current structure and only replaces the three calls with the bound value, leaving used_whole_content_path as a match. Either form removes the expect.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@crates/cmtraceopen-parser/src/parser/mod.rs` around lines 168 - 204, Resolve
device_inventory_dialect(selection.specialization) once before the parser match
and reuse the bound result in the IntuneDeviceInventory branch and
used_whole_content_path logic. Remove the match guard’s repeated lookup and the
expect("guarded by the match arm above"), while preserving fallback to
parse_lines_with_selection when no dialect is available.
src-tauri/src/watcher/tail.rs (1)

446-466: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

The specialization-to-dialect mapping now exists in three places.

This function maps ParserSpecialization to DeviceInventoryLogDialect. crates/cmtraceopen-parser/src/parser/mod.rs::device_inventory_dialect holds the same table, and crates/cmtraceopen-parser/src/parser/detect.rs::intune_device_inventory holds the inverse. Adding a fourth dialect requires edits in all three, and only the detect.rs direction fails to compile when a variant is missed, because both mapping functions end in _ => None.

Expose one conversion from the parser crate, then keep only the framing precondition here.

♻️ Proposed shared conversion

Add to crates/cmtraceopen-parser/src/intune/device/windows/inventory/mod.rs:

impl DeviceInventoryLogDialect {
    /// The dialect a parser specialization identifies, if it identifies one.
    pub fn from_specialization(specialization: ParserSpecialization) -> Option<Self> {
        match specialization {
            ParserSpecialization::IntuneDeviceInventoryHarvester => Some(Self::Harvester),
            ParserSpecialization::IntuneDeviceInventoryAdaptor => Some(Self::InventoryAdaptor),
            ParserSpecialization::IntuneDeviceInventoryRotationFailure => {
                Some(Self::RotationFailure)
            }
            ParserSpecialization::Ime => None,
        }
    }
}

The exhaustive match makes a new specialization a compile error at the single source of truth. Then reduce this function to:

 fn inventory_logical_dialect(selection: &ResolvedParser) -> Option<DeviceInventoryLogDialect> {
     if selection.record_framing != RecordFraming::LogicalRecord {
         return None;
     }
 
-    match selection.specialization {
-        // Harvester headers usually frame a single line, but the parser still
-        // attaches a non-header line to the record above it, so tailing has to
-        // frame it the same way the initial parse does.
-        Some(ParserSpecialization::IntuneDeviceInventoryHarvester) => {
-            Some(DeviceInventoryLogDialect::Harvester)
-        }
-        Some(ParserSpecialization::IntuneDeviceInventoryAdaptor) => {
-            Some(DeviceInventoryLogDialect::InventoryAdaptor)
-        }
-        Some(ParserSpecialization::IntuneDeviceInventoryRotationFailure) => {
-            Some(DeviceInventoryLogDialect::RotationFailure)
-        }
-        _ => None,
-    }
+    // Harvester headers usually frame a single line, but the parser still
+    // attaches a non-header line to the record above it, so tailing has to
+    // frame it the same way the initial parse does.
+    selection
+        .specialization
+        .and_then(DeviceInventoryLogDialect::from_specialization)
 }

parser/mod.rs::device_inventory_dialect then becomes a call to the same conversion.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src-tauri/src/watcher/tail.rs` around lines 446 - 466, Centralize the
ParserSpecialization-to-DeviceInventoryLogDialect mapping in
DeviceInventoryLogDialect::from_specialization within the inventory module using
an exhaustive match. Update inventory_logical_dialect and
parser/mod.rs::device_inventory_dialect to retain only their existing
framing/precondition checks and delegate the conversion; remove their duplicated
mapping matches while preserving None for non-logical framing and unsupported
specializations.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@crates/cmtraceopen-parser/src/intune/device/windows/inventory/mod.rs`:
- Around line 183-207: Bound the scanning performed by detect_dialect: update
count_headers to stop immediately once its count reaches two and limit both
count_headers and has_rotation_failure_signature to a fixed early-line window
consistent with the existing sample limits. Preserve the current header
thresholds, filename-hint behavior, and rotation-failure detection while
ensuring all three scans avoid traversing the entire file.

In `@src-tauri/src/watcher/tail.rs`:
- Around line 17-18: Align oversized Adaptor-record behavior between
parse_records and the tailing path using MAX_PENDING_LOGICAL_RECORD_BYTES.
Update the tail logic to preserve oversized records as a single entry without
emitting split-related parse errors, or apply the same limit and handling to
parse_records; ensure both paths follow the chosen explicit contract
consistently.

---

Duplicate comments:
In `@crates/cmtraceopen-parser/README.md`:
- Line 118: Update the IntuneDeviceInventory row in the supported-format table
to include IntuneDeviceInventory.log alongside the existing Harvester, Adaptor,
and rotated .log_ filenames.

---

Nitpick comments:
In `@crates/cmtraceopen-parser/src/parser/ccm.rs`:
- Around line 141-152: Add a concise comment to the exhaustive match in the CCM
parser explaining that the three Device Inventory specializations share this arm
only because ParserSpecialization must be matched exhaustively, not because they
are routed through CCM in normal operation. Keep the existing match arms and
parsing behavior unchanged.

In `@crates/cmtraceopen-parser/src/parser/mod.rs`:
- Around line 168-204: Resolve
device_inventory_dialect(selection.specialization) once before the parser match
and reuse the bound result in the IntuneDeviceInventory branch and
used_whole_content_path logic. Remove the match guard’s repeated lookup and the
expect("guarded by the match arm above"), while preserving fallback to
parse_lines_with_selection when no dialect is available.

In `@src-tauri/src/watcher/tail.rs`:
- Around line 446-466: Centralize the
ParserSpecialization-to-DeviceInventoryLogDialect mapping in
DeviceInventoryLogDialect::from_specialization within the inventory module using
an exhaustive match. Update inventory_logical_dialect and
parser/mod.rs::device_inventory_dialect to retain only their existing
framing/precondition checks and delegate the conversion; remove their duplicated
mapping matches while preserving None for non-logical framing and unsupported
specializations.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: c5766b77-5b9c-4b2f-94ea-22394f80efa2

📥 Commits

Reviewing files that changed from the base of the PR and between 88e9f13 and 750472b.

⛔ Files ignored due to path filters (1)
  • src-tauri/tests/corpus/intune_device_inventory/clean/IntuneInventoryHarvesterLog.log is excluded by !**/*.log
📒 Files selected for processing (22)
  • CHANGELOG.md
  • crates/cmtraceopen-parser/README.md
  • crates/cmtraceopen-parser/src/intune/device/windows/inventory/mod.rs
  • crates/cmtraceopen-parser/src/models/log_entry.rs
  • crates/cmtraceopen-parser/src/parser/ccm.rs
  • crates/cmtraceopen-parser/src/parser/detect.rs
  • crates/cmtraceopen-parser/src/parser/mod.rs
  • crates/cmtraceopen-parser/tests/intune_device_inventory.rs
  • docs/superpowers/plans/2026-07-30-intune-device-inventory-ingestion.md
  • docs/superpowers/specs/2026-07-30-intune-device-inventory-ingestion-design.md
  • src-tauri/src/commands/bundle_ops.rs
  • src-tauri/src/commands/file_association.rs
  • src-tauri/src/commands/known_sources.rs
  • src-tauri/src/watcher/tail.rs
  • src-tauri/tauri.conf.json
  • src-tauri/tests/corpus/intune_device_inventory/clean/InventoryAdaptor.log_
  • src-tauri/tests/parser_regression_corpus.rs
  • src-tauri/tests/parser_supported_formats.rs
  • src/lib/column-config.ts
  • src/lib/log-source.test.ts
  • src/stores/log-store.ts
  • src/types/log.ts

Comment on lines +183 to +207
/// Detect a Device Inventory dialect from log content and an optional filename hint.
///
/// A filename can lower the Harvester or Adaptor confidence threshold to one matching
/// header, but it is never sufficient without a matching content header.
pub fn detect_dialect(file_path: &str, content: &str) -> Option<DeviceInventoryLogDialect> {
let harvester_headers = count_headers(content, harvester_re());
if harvester_headers >= 2
|| (harvester_headers >= 1 && has_filename_hint(file_path, "intuneinventoryharvesterlog"))
{
return Some(DeviceInventoryLogDialect::Harvester);
}

let adaptor_headers = count_headers(content, adaptor_re());
if adaptor_headers >= 2
|| (adaptor_headers >= 1 && has_filename_hint(file_path, "inventoryadaptor"))
{
return Some(DeviceInventoryLogDialect::InventoryAdaptor);
}

if has_rotation_failure_signature(content) {
return Some(DeviceInventoryLogDialect::RotationFailure);
}

None
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚀 Performance & Scalability | 🟠 Major | 🏗️ Heavy lift

Bound the detection scan; it now reads whole files on every open.

detect_dialect runs before every other content detector in detect_parser, and it scans the complete file content up to three times: count_headers for Harvester, count_headers for Adaptor, and has_rotation_failure_signature. Each scan visits every line and applies a regex.

Every other detector in detect_parser works from sample_lines, which is capped at 20 or 50 non-empty lines. So opening any large log file, including files that are not Device Inventory logs, now costs three full-content regex passes before the cheap detectors run.

Two bounds remove the cost without weakening detection:

  • count_headers only needs to know whether the count reaches 2. It can stop at the threshold.
  • Both count_headers and has_rotation_failure_signature can stop after a bounded number of lines. The dialects are header-led, so evidence appears early.
♻️ Proposed bounds
+/// Lines sampled for content detection. The dialects are header-led, so
+/// evidence appears near the start of the file.
+const DETECTION_SAMPLE_LINES: usize = 50;
+
-fn count_headers(content: &str, regex: &Regex) -> usize {
+fn count_headers(content: &str, regex: &Regex, needed: usize) -> usize {
     content
         .lines()
+        .take(DETECTION_SAMPLE_LINES)
         .map(|line| line.trim_end_matches('\r'))
         .filter(|line| regex.is_match(line))
-        .count()
+        .take(needed)
+        .count()
 }
 fn has_rotation_failure_signature(content: &str) -> bool {
     let mut saw_header = false;
     let mut header_states_failure = false;
     let mut exception_under_header = false;
 
-    for line in content.lines().map(|line| line.trim_end_matches('\r')) {
+    for line in content
+        .lines()
+        .take(DETECTION_SAMPLE_LINES)
+        .map(|line| line.trim_end_matches('\r'))
+    {
         if let Some(captures) = rotation_re().captures(line) {
             saw_header = true;
             if captures
                 .name("message")
                 .is_some_and(|message| rotation_header_states_failure(message.as_str()))
             {
                 header_states_failure = true;
             }
         } else if saw_header && looks_like_dotnet_exception(line) {
             exception_under_header = true;
         }
+
+        if saw_header && (header_states_failure || exception_under_header) {
+            return true;
+        }
     }
 
     saw_header && (header_states_failure || exception_under_header)
 }

Note that a sample bound changes detection for a file whose first evidence sits beyond the window. If you prefer to keep the unbounded scan, apply the early-exit changes alone; they still remove the worst case for files that match early.

Also applies to: 311-317, 369-389

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@crates/cmtraceopen-parser/src/intune/device/windows/inventory/mod.rs` around
lines 183 - 207, Bound the scanning performed by detect_dialect: update
count_headers to stop immediately once its count reaches two and limit both
count_headers and has_rotation_failure_signature to a fixed early-line window
consistent with the existing sample limits. Preserve the current header
thresholds, filename-hint behavior, and rotation-failure detection while
ensuring all three scans avoid traversing the entire file.

Comment on lines +17 to +18
const LOGICAL_RECORD_DEBOUNCE: Duration = Duration::from_millis(250);
const MAX_PENDING_LOGICAL_RECORD_BYTES: usize = 1024 * 1024;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🗄️ Data Integrity & Integration | 🟠 Major | 🏗️ Heavy lift

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Description: Check whether any whole-file parse path bounds record size, and find the largest fixture payload.
set -euo pipefail

# Look for a size bound in the pure parser.
rg -n -C3 'max_pending_bytes|MAX_PENDING|len\(\) >' crates/cmtraceopen-parser/src/intune

# Look for comparable bounds elsewhere in the parser crate.
rg -n -C2 '1024 \* 1024|MiB|MAX_.*BYTES' crates/cmtraceopen-parser/src src-tauri/src

Repository: adamgell/cmtraceopen

Length of output: 50376


🏁 Script executed:

#!/bin/bash
set -euo pipefail

printf '%s\n' '--- whole-file parser symbols ---'
rg -n -C5 'parse_records|logical_framing|MAX_PENDING_LOGICAL_RECORD_BYTES|parse_errors' \
  crates/cmtraceopen-parser/src/intune/device/windows/inventory/mod.rs \
  src-tauri/src/watcher/tail.rs

printf '%s\n' '--- whole-file parser implementation ---'
sed -n '1,180p' crates/cmtraceopen-parser/src/intune/device/windows/inventory/mod.rs

printf '%s\n' '--- tail framing and emission paths ---'
sed -n '200,400p' src-tauri/src/watcher/tail.rs

Repository: adamgell/cmtraceopen

Length of output: 41269


🏁 Script executed:

#!/bin/bash
set -euo pipefail

printf '%s\n' '--- parse_records body ---'
sed -n '235,310p' crates/cmtraceopen-parser/src/intune/device/windows/inventory/mod.rs

printf '%s\n' '--- framing overflow tests ---'
sed -n '638,690p' crates/cmtraceopen-parser/src/intune/device/windows/inventory/mod.rs

printf '%s\n' '--- tail overflow tests ---'
sed -n '1400,1492p' src-tauri/src/watcher/tail.rs

printf '%s\n' '--- parser dispatch for inventory open path ---'
rg -n -C4 'parse_content_with_selection|parse_lines_with_selection|InventoryAdaptor|parse_content\(' \
  src-tauri/src crates/cmtraceopen-parser/src

Repository: adamgell/cmtraceopen

Length of output: 50376


🏁 Script executed:

#!/bin/bash
set -euo pipefail

python3 - <<'PY'
import re

LIMIT = 1024 * 1024
header = "[Thu Jul 30 13:05:02 2026][8604] - "
payload = "x" * (2 * LIMIT)

def whole_file(lines):
    entries = []
    errors = 0
    adaptor = re.compile(
        r"^\[[A-Z][a-z]{2} [A-Z][a-z]{2} +\d{1,2} "
        r"\d{2}:\d{2}:\d{2} \d{4}\]\[\d+\] - (?P<message>.*)$"
    )
    for line in lines:
        if adaptor.match(line):
            entries.append(line)
        elif entries:
            entries[-1] += "\n" + line
        else:
            entries.append(line)
    return entries, errors

def tail_frame(lines):
    completed = []
    pending = None
    overflows = 0
    adaptor = re.compile(
        r"^\[[A-Z][a-z]{2} [A-Z][a-z]{2} +\d{1,2} "
        r"\d{2}:\d{2}:\d{2} \d{4}\]\[\d+\] - "
    )
    for line in lines:
        if adaptor.match(line) and pending is not None:
            completed.append(pending)
            pending = None
        pending = line if pending is None else pending + "\n" + line
        while len(pending) > LIMIT:
            completed.append(pending[:LIMIT])
            pending = pending[LIMIT:]
            overflows += 1
    return completed, pending, overflows

lines = [header + "Oversized action.", payload]
opened, open_errors = whole_file(lines)
completed, pending, tail_errors = tail_frame(lines)

print({
    "opened_entries": len(opened),
    "opened_parse_errors": open_errors,
    "tailed_completed_entries": len(completed),
    "tailed_pending_bytes": len(pending or ""),
    "tailed_framing_parse_errors": tail_errors,
    "all_tailed_pieces_bounded": all(len(piece) <= LIMIT for piece in completed + ([pending] if pending else [])),
    "opened_message_bytes": len(opened[0]),
})
PY

Repository: adamgell/cmtraceopen

Length of output: 367


Align oversized-record handling across open and tail paths.

parse_records keeps an oversized Adaptor record as one entry with no parse error. Tailing splits it at MAX_PENDING_LOGICAL_RECORD_BYTES, creates multiple partial entries, and reports a parse error for each split. Apply the same limit to both paths, or define an explicit contract for oversized records.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src-tauri/src/watcher/tail.rs` around lines 17 - 18, Align oversized
Adaptor-record behavior between parse_records and the tailing path using
MAX_PENDING_LOGICAL_RECORD_BYTES. Update the tail logic to preserve oversized
records as a single entry without emitting split-related parse errors, or apply
the same limit and handling to parse_records; ensure both paths follow the
chosen explicit contract consistently.

@adamgell

adamgell commented Aug 3, 2026

Copy link
Copy Markdown
Owner Author

@copilot Is this ready?

Copilot AI commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

@copilot Is this ready?

Yes, with two items to confirm before merging.

Green:

  • All 17 CI checks pass on the current head (750472b): Rust MSRV ×2, Check & Test, E2E Playwright, TypeScript, ESP Diagnostics, Build ×3 (macOS/Linux/Windows), CodeQL ×4.
  • All three Copilot review threads are resolved (LogicalRecord framing, join-free parse_lines, is_reportable batch predicate).
  • The branch was 64 commits behind main; just merged cleanly (zero conflicts) in ce9d9d5.

Two open questions from the review (comment #5151731147) that weren't acted on because they change intended behavior — they need a decision, not a fix from me:

  1. has_rotation_failure_signature breadth. The function accepts any ISO-8601 header plus .NET exception evidence anywhere in the file, and rotation_header_states_failure matches starts_with("failed ") / starts_with("unhandled "). The concern (comment #5146080646) is that a line like 2026-07-30T10:15:00Z Failed to load configuration. in any unrelated ISO-8601 log could route the whole file to the Device Inventory parser. If the intended scope is only Device Inventory rotation logs, narrowing to rotation-specific phrases and requiring the exception to be a continuation of the header (not anywhere in the file) would make it precise. If the current breadth is intentional, the existing negative test should be extended to cover the starts_with arms.

  2. Continuation size asymmetry. Tailing bounds pending continuations at 1 MiB; parse_content (initial load) applies no bound. A record larger than 1 MiB produces different framing depending on whether the user tails or opens the file. This is noted as a known gap in the PR description (item 2) — confirming it's an acceptable gap for this PR is all that's needed.

If both are acceptable gaps for now, the PR is ready to merge.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (2)
CHANGELOG.md (2)

38-41: 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Move this entry into the existing ### Added section.

The 1.5.1 section already has ### Added at Line 7. This second heading places the Device Inventory entry after ### Dependencies and gives the changelog an incorrect structure.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@CHANGELOG.md` around lines 38 - 41, Move the Intune Device Inventory log
support entry into the existing ### Added section within the 1.5.1 changelog
section, and remove this duplicate ### Added heading so the entry appears before
### Dependencies.

40-40: 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Use the configured environment-variable path.

The entry hardcodes C:\Program Files, but crates/cmtraceopen-parser/src/collector/profile_data.json configures %ProgramFiles%\Microsoft Device Inventory Agent\Logs\*. Document %ProgramFiles%\Microsoft Device Inventory Agent\Logs or state that C:\Program Files is only the default location.

As per path instructions, verify documented paths against repository configuration.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@CHANGELOG.md` at line 40, Update the Intune Device Inventory log support
entry to document the configured %ProgramFiles%\Microsoft Device Inventory
Agent\Logs location, or explicitly identify C:\Program Files as only the
default; align the wording with the path configured in profile_data.json.

Source: Path instructions

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In `@CHANGELOG.md`:
- Around line 38-41: Move the Intune Device Inventory log support entry into the
existing ### Added section within the 1.5.1 changelog section, and remove this
duplicate ### Added heading so the entry appears before ### Dependencies.
- Line 40: Update the Intune Device Inventory log support entry to document the
configured %ProgramFiles%\Microsoft Device Inventory Agent\Logs location, or
explicitly identify C:\Program Files as only the default; align the wording with
the path configured in profile_data.json.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 60efec2f-9240-4286-b21c-e4657875b3a0

📥 Commits

Reviewing files that changed from the base of the PR and between 750472b and ce9d9d5.

📒 Files selected for processing (3)
  • CHANGELOG.md
  • src-tauri/tauri.conf.json
  • src-tauri/tests/parser_regression_corpus.rs

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

device Device management related enhancement New feature or request feature New feature intune Microsoft Intune related parser Log parser related test Testing related windows Windows platform related

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants