Skip to content

fix(parser): reject Unicode decimal fields - #502

Merged
adamgell merged 3 commits into
mainfrom
codex/issue-413-unicode-panics
Aug 5, 2026
Merged

adamgell merged 3 commits into
mainfrom
codex/issue-413-unicode-panics

Conversation

@adamgell

@adamgell adamgell commented Aug 5, 2026 •

Copy link
Copy Markdown
Owner

Summary

  • restrict numeric log grammar captures to ASCII digits
  • keep DNS wire-format parsing on UTF-8 character boundaries
  • cover public parser, ReportingEvents detection, DNS, and Intune event tracking surfaces

Verification

  • cargo test --test issue_413_unicode_panics
  • cargo test -p cmtraceopen-parser
  • cargo clippy -p cmtraceopen-parser --all-targets -- -D warnings
  • rustfmt --check on changed Rust files

Closes #413

Summary by CodeRabbit

  • Bug Fixes

    • Improved log parsing with consistent ASCII digit handling across timestamps, error codes, retries, inventory, and event fields.
    • Prevented crashes and infinite loops when processing malformed Unicode text in DNS names and numeric log fields.
    • Improved handling of invalid reporting-event data.
  • Tests

    • Added regression coverage for Unicode digits, malformed fields, Unicode DNS names, and Intune event tracking.

@github-actions github-actions Bot added bug Something isn't working enhancement New feature or request labels Aug 5, 2026
@coderabbitai

coderabbitai Bot commented Aug 5, 2026 •

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 268bb5ab-74ed-4324-ba4b-1aa9347d2978

📥 Commits

Reviewing files that changed from the base of the PR and between d89a1dc and 5bd464d.

📒 Files selected for processing (2)
  • crates/cmtraceopen-parser/src/parser/cmtlog.rs
  • crates/cmtraceopen-parser/tests/issue_413_unicode_panics.rs

📝 Walkthrough

Walkthrough

The parser now uses ASCII-only digit matching, handles malformed UTF-8 label lengths without invalid slicing, and adds regression tests for Unicode numeric fields, DNS labels, reporting events, and Intune tracking.

Changes

Unicode input hardening

Layer / File(s) Summary
ASCII digit patterns
crates/cmtraceopen-parser/src/parser/*.rs, crates/cmtraceopen-parser/src/intune/event_tracker.rs
Replaced Unicode-aware digit shorthands with explicit [0-9] classes in parser and Intune regular expressions.
DNS UTF-8 boundary handling
crates/cmtraceopen-parser/src/parser/dns_types.rs
The decoder checks character boundaries before slicing and advances unexpected text by UTF-8 character width.
Unicode regression coverage
crates/cmtraceopen-parser/tests/issue_413_unicode_panics.rs
Added tests for Unicode numeric fields, reporting events, DNS labels, and Intune event tracking.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Possibly related PRs

Suggested labels: parser, intune, test

🚥 Pre-merge checks | ✅ 4
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title follows Conventional Commits and accurately describes rejecting Unicode decimal fields in parser code.
Linked Issues check ✅ Passed The changes address issue #413 by restricting numeric grammar to ASCII digits, preventing CCM time-tail panics, and adding Unicode regression coverage.
Out of Scope Changes check ✅ Passed The regex updates, DNS boundary fix, and regression tests support the stated Unicode-input safety objectives and are not out of scope.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/issue-413-unicode-panics

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@crates/cmtraceopen-parser/tests/issue_413_unicode_panics.rs`:
- Around line 18-29: Update the tests in
crates/cmtraceopen-parser/tests/issue_413_unicode_panics.rs at lines 18-29 to
retain the no-panic parser calls while inspecting their results and asserting
that malformed Unicode decimal fields do not produce the affected parsed
records; also update lines 52-60 to inspect returned events and assert that
Unicode inventory counts do not produce an inventory event.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 997daa25-db2e-4a54-935d-c82bfabeed7a

📥 Commits

Reviewing files that changed from the base of the PR and between 1f025d0 and d89a1dc.

📒 Files selected for processing (7)
  • crates/cmtraceopen-parser/src/intune/event_tracker.rs
  • crates/cmtraceopen-parser/src/parser/ccm.rs
  • crates/cmtraceopen-parser/src/parser/dns_types.rs
  • crates/cmtraceopen-parser/src/parser/reporting_events.rs
  • crates/cmtraceopen-parser/src/parser/simple.rs
  • crates/cmtraceopen-parser/src/parser/timestamped.rs
  • crates/cmtraceopen-parser/tests/issue_413_unicode_panics.rs

@adamgell

adamgell commented Aug 5, 2026

Copy link
Copy Markdown
Owner Author

Independent critic verdict at d89a1dc694c023d35c10cd14b5364a44ffd926fe: REWORK.

Blocking omission: parser/cmtlog.rs still uses Unicode-capable \d for structural numeric CCM fields and defaults failed conversions. A public CmtLog input with time="14:30:00.١٢+000" remains structured and fabricates .000 instead of failing closed/preserving malformed input.

The existing CCM/simple/timestamped/reporting/DNS/event-tracker fixes otherwise passed the class audit. Rework is assigned to:

  • restrict CmtLog structural numeric grammar to ASCII;
  • add a public semantic regression for the reproduced input;
  • strengthen changed-parser regressions to assert output/fallback and valid Unicode payload preservation, not only “did not panic.”

#410/#414 greedy signless-tail semantics remain separate.

@adamgell

adamgell commented Aug 5, 2026

Copy link
Copy Markdown
Owner Author

Independent critic re-review: ACCEPT at frozen SHA 5bd464d3cdb462c267dc516cf43f422eae3779fd.

Residual risk is limited to that already-filed legacy signless-tail behavior, outside #413. Hosted CI is the remaining gate.

@adamgell

adamgell commented Aug 5, 2026

Copy link
Copy Markdown
Owner Author

Final integration critic: ACCEPT at frozen merge SHA bdad76b9bfe626819becc0f5628c98668ee5018b.

Residual integration risk is low. Restarted hosted CI is the final merge gate.

@adamgell
adamgell merged commit 5cc7da2 into main Aug 5, 2026
16 checks passed
@adamgell
adamgell deleted the codex/issue-413-unicode-panics branch September 25, 2026 13:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

CCM parser panics on non-ASCII decimal digits in a time tail

1 participant