Skip to content

fix: skip a UTF-8 BOM in the fast parser - #1041

Merged
motdotla merged 1 commit into
motdotla:masterfrom
Jaybhade:fix/fast-parser-bom
Aug 3, 2026
Merged

motdotla merged 1 commit into
motdotla:masterfrom
Jaybhade:fix/fast-parser-bom

Conversation

@Jaybhade

@Jaybhade Jaybhade commented Aug 2, 2026

Copy link
Copy Markdown
Contributor

parse(src, { fast: true }) silently drops the first variable when the .env file starts with a UTF-8 BOM. The classic parser reads the same file correctly, so this is a parity gap in the opt-in scanner rather than a change in documented behaviour.

Reproducing

$ printf '\xef\xbb\xbfGREETING=hello\nPORT=3000\n' > .env   # UTF-8 with signature
$ node -e "const d=require('dotenv'),fs=require('fs');const s=fs.readFileSync('.env','utf8');
  console.log('classic:', d.parse(s));
  console.log('fast:   ', d.parse(s, {fast:true}))"
classic: { GREETING: 'hello', PORT: '3000' }
fast:    { PORT: '3000' }

Through the CLI, the same file loses a variable with no warning at all — it just reports one fewer:

$ dotenv run --fast -- node -e "console.log(process.env.GREETING)"
◇ injected env (1) from .env
undefined

Why it happens

Between entries, parseFast skips only space (32), tab (9) and newline (10). A BOM is none of those, so the scanner treats  as the start of a key. It matches no KEY_CHAR, i === keyStart, and the line is discarded as invalid — taking the first variable with it.

parseRegex never had the problem because LINE opens with \s*, and JavaScript's \s includes .

This matters in practice because a BOM is what PowerShell's > / Out-File and Windows Notepad write by default, and fs.readFileSync(path, { encoding: 'utf8' }) preserves it. The .env looks completely normal in an editor.

The fix

One extra code in the whitespace-skip loop, so the scanner steps over a BOM exactly where the regex's \s* does — which covers both a leading BOM and one after a newline (concatenated files):

while (i < len && (c === 32 || c === 9 || c === 10 || c === 65279)) {

I deliberately kept this to the BOM rather than widening the loop to all of \s. Exotic spaces such as NBSP still differ between the two parsers ( KEY=value indented with U+00A0 parses classically but not with fast), but nothing writes those into a .env by accident, and matching them would cost more comparisons in the hot loop for no real-world gain. Happy to widen it if you'd rather the two be exactly equivalent.

Tests

Added to tests/test-parse-fast.js, using the existing assertSameParse helper:

  • seven parity cases — BOM before a key, before export, before a comment, a BOM-only source, after a blank line, and mid-file
  • one assertion pinning the concrete result, so the two parsers can't pass by both being wrong
  • a config({ fast: true }) test against a new tests/.env.bom fixture (real EF BB BF bytes), covering the readFileSync path rather than string literals only

npm test is green: 181 pass, 1 skip, 0 fail — build, standard, dts-check and TAP included. All eight new assertions fail on main before the change.

Performance

The scanner exists for speed, so I measured it rather than assuming. Median of 7 runs × 5000 parse(buf, { fast: true }) calls over tests/.env × 8, patched vs. unpatched:

median
before 341.68 ms
after 339.61 ms

Delta −0.6% on one run and −0.3% on a repeat — noise. The added comparison only executes on characters the loop was already inspecting.

The character scanner used by { fast: true } skipped only spaces, tabs and
newlines between entries, so a leading BOM was treated as the start of a key.
It matches no key character, the line was discarded as invalid, and the first
variable in the file silently went missing.

The classic regex parser consumes the BOM as part of its leading \s*, so the
two parsers disagreed on any .env saved with a UTF-8 signature — the default
encoding for PowerShell redirects and Windows Notepad.
@motdotla

motdotla commented Aug 3, 2026

Copy link
Copy Markdown
Owner

great catch. thank you @Jaybhade

@motdotla
motdotla merged commit 98c6030 into motdotla:master Aug 3, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants