fix(core): strip BOM from AI-generated content - #381
Closed
moizlatif7 wants to merge 1 commit into
Closed
Conversation
Changes the BOM preservation logic to only preserve BOMs from existing files, never from new AI-generated content. This fixes Issue #368 where Windows systems were adding UTF-8 BOMs to new files, breaking Linux scripts with shebangs. Changes: - Modified writeTextPreservingBom to strip BOM from new content - Updated 6 files across core and opencode packages - Changed logic from `source.bom || next.bom` to `source.bom` - Added test case to verify BOMs are stripped from AI content - Updated existing test to reflect new behavior Behavior: - Existing files with BOMs: BOM preserved (round-trip safe) - New files with BOM in content: BOM stripped (fixes #368) - AI-generated content with BOM: BOM stripped (fixes Windows issue) Fixes #368
Author
|
Closing this PR to redo the fix using proper ANR workflow with ANR change markers. Will reopen with correct branch naming (anr/368/*) and all required ANRCODE_CHANGE markers. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #368 by stripping UTF-8 BOMs from AI-generated content while preserving BOMs in existing files.
Problem
On Windows systems, OpenCode was adding UTF-8 BOMs (
0xEF 0xBB 0xBF) to newly created files when the AI's content contained a BOM character (\uFEFF). This broke Linux scripts because the shebang (#!/bin/bash) must be the very first bytes in the file.The logic was:
desiredBom = source.bom || next.bomsource.bom: Does existing file have BOM?next.bom: Does AI content have BOM? ← This was the problemSolution
Changed BOM handling to:
desiredBom = source.bomThis means:
Changes
Modified Files (6 packages):
packages/core/src/file-mutation.ts- Core V2 write logicpackages/core/src/patch.ts- Patch derivationpackages/core/src/tool/edit.ts- Core edit toolpackages/opencode/src/tool/write.ts- Write toolpackages/opencode/src/tool/edit.ts- Edit tool (2 locations)packages/opencode/src/patch/index.ts- Patch applicationTest Updates:
Testing
✅ All existing BOM-related tests pass
✅ New test verifies BOMs are stripped from AI content
✅ Full core test suite: 1075 tests pass
✅ Tool test suite: 335 tests pass
✅ Format/Patch tests: 31 tests pass
Verification
To verify the fix works:
Closes #368