Skip to content

fast parser diverges from the default parser (silent value loss) #1043

Closed

Description

@chuanghiduoc

name: fast parser diverges from the default parser (silent value loss with { fast: true })
about: parse(src, {fast:true}) returns different values than parse(src) for several inputs — including silently dropping values
title: fast parser diverges from the default parser (silent value loss)
labels: bug
assignees: ''


The opt-in fast parser (parse(src, { fast: true }), merged in #1010) disagrees with the default regex parser on several classes of input. Since fast is meant as a drop-in performance option, any divergence means users who flip the flag get different env values — in one case a value is silently replaced with an empty string.

Reproduced on master (after #1041 and #1042), Node 24.12.0, Windows:

1. Blank line before a quoted multiline value — value lost

require('dotenv').parse('A=\n\n"hello\nworld"')
// default: { A: 'hello\nworld' }
require('dotenv').parse('A=\n\n"hello\nworld"', { fast: true })
// fast:   { A: '' }        <- secret silently becomes empty string

The default LINE regex lets \s* inside the quoted-value alternative span newlines, so a quote on a later line is consumed as A's value. The fast scanner ends A at end-of-line and then fails to parse "hello\nworld" as a key line.

This also fires without blank lines: A=\n'b' gives {A: 'b'} vs {A: ''}.

2. Form feed / vertical tab / non-breaking space before a key — key dropped

require('dotenv').parse('\fA=1')            // { A: '1' }
require('dotenv').parse('\fA=1', {fast: true}) // {}
// same for \v and U+00A0

The default parser skips these because \s matches them; the fast scanner only skips space/tab/newline/BOM, so when the key scan finds no [A-Za-z0-9_.-] it discards the whole line. NBSP is common in .env files edited via copy-paste.

3. Trailing junk after a closing quote — different value

require('dotenv').parse('TOKEN="abc" oops')
// default: { TOKEN: '"abc" oops' }
require('dotenv').parse('TOKEN="abc" oops', { fast: true })
// fast:   { TOKEN: 'abc' }

Default keeps everything after = (the junk falls outside the match), fast returns the clean quoted content. Arguably fast's result is nicer here, but it differs.


Tests demonstrating all five failures (tests/test-fast-parity.js, tap):

not ok 2 - fast must match default            (blank line case)
not ok 4 - fast drops key after "\f"
not ok 6 - fast drops key after "\u000b"
    ok - nbsp case folded into whitespace test
not ok - fast must match default              (trailing junk case)
# { total: 10, pass: 5, fail: 5 }

Happy to send a PR aligning the scanner with the default parser on all three (or at minimum cases 1–2, where data is lost or keys vanish).

Activity

  1. motdotla commented on Aug 24, 2026

    @motdotla
    Owner

    @chuanghiduoc great catch. yes, please send a PR. greatly appreciated.

  2. chuanghiduoc commented on Aug 24, 2026

    @chuanghiduoc
    Author

    PR sent: #1044

  3. afonsojanu commented on Sep 3, 2026

    @afonsojanu

    There's a fourth divergence in this same family that none of the open PRs (#1044, #1046, #1047) currently cover: an unterminated double-quoted value doesn't get its \n/\r sequences unescaped in the fast parser, even though the classic parser does it regardless of whether a closing quote ever shows up.

    require('dotenv').parse('KEY="line one\nline two')
    // default: { KEY: '"line one\nline two' } (actual newline)
    require('dotenv').parse('KEY="line one\nline two', { fast: true })
    // fast: { KEY: '"line one\nline two' } (literal backslash-n, unchanged)

    The classic parser's rule is simpler than it looks: it unescapes \n and \r whenever the trimmed value's first character is a double quote, full stop, independent of whatever the outer-quote-stripping regex does or doesn't match afterward. An unterminated quote still starts with ", so it still qualifies, even though there's nothing on the other end to strip.

    Related, and also open in all three PRs as far as I can tell: two separate quoted segments on one line, like KEY="a" "b", where the classic parser's blanket "strip a leading/trailing quote character if both match" step ends up unwrapping across both segments (giving a" "b for that particular input) rather than stopping cleanly after the first pair. Checked this against all three branches and it reproduces identically on each.

    Not opening a PR since there's already solid coverage in flight for the cases originally listed here, but flagging these two so whichever branch lands can pick them up too if useful.

  4. motdotla commented on Sep 21, 2026

    @motdotla
    Owner

    working here: #1055

  5. motdotla commented on Sep 21, 2026

    @motdotla
    Owner

    going out as 18.0.2

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions