parser: allow decimal values in SET_VAR hints

[hawkingrei]

What problem does this PR solve?

Issue Number: close #64251

Problem Summary:

SET_VAR optimizer hints reject decimal and floating-point values, so a hint like /*+ SET_VAR(tidb_default_string_match_selectivity=0.3) */ cannot be parsed even though the target system variable accepts decimal values.

What changed and how does it work?

Allow decimal and floating-point numeric literals only while parsing the value slot of SET_VAR(... = value).

The hint scanner now tracks a small SET_VAR value-position state and returns a hint-only numeric literal token for decimal values in that position. Other hint arguments still reject decimal literals, preserving the existing grammar boundary.

Regenerate pkg/parser/hintparser.go and add parser/planner coverage for decimal SET_VAR values.

Check List

Tests

• Unit test
• Integration test
• Manual test (add detailed scripts or steps below)
• No need to test

  • I checked and no code files have been changed.

Manual test details:

git -c core.fsmonitor=false diff --check
./tools/check/failpoint-go-test.sh pkg/parser -run TestParseHint -count=1
./tools/check/failpoint-go-test.sh pkg/planner/core -run TestHintSuite -count=1

Side effects

• Performance regression: Consumes more CPU
• Performance regression: Consumes more Memory
• Breaking backward compatibility

Documentation

• Affects user behaviors
• Contains syntax changes
• Contains variable changes
• Contains experimental features
• Changes MySQL compatibility

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

Fix an issue that SET_VAR optimizer hints rejected decimal numeric values.

Summary by CodeRabbit

• New Features

  • SET_VAR hints now accept decimal numeric values with optional +/− signs (e.g., set_var(param=0.3, +0.3, -0.3).

• Bug Fixes

  • Improved handling so decimal literals in hints parse correctly; previously-invalid cases now produce appropriate errors.

• Tests

  • Added and updated tests to cover decimal hint parsing, error cases, and runtime behavior when applying decimal SET_VAR hints.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: 73cc67c3-a3be-4a31-82c0-241bf21699e3

📥 Commits

Reviewing files that changed from the base of the PR and between 13f45cb and 88ab574.

📒 Files selected for processing (5)

• pkg/parser/hintparser.go
• pkg/parser/hintparser.y
• pkg/parser/hintparser_test.go
• pkg/parser/hintparserimpl.go
• pkg/planner/core/hint_test.go

🚧 Files skipped from review as they are similar to previous changes (5)

• pkg/planner/core/hint_test.go
• pkg/parser/hintparser_test.go
• pkg/parser/hintparser.y
• pkg/parser/hintparser.go
• pkg/parser/hintparserimpl.go

---

📝 Walkthrough

Walkthrough

Adds lexer state to accept numeric literals in SET_VAR value positions, introduces a hintNumericLit token and signed-value grammar, regenerates parser tables and actions, and updates parser and planner tests to cover decimal hint values.

Changes

SET_VAR decimal hint parsing

Layer / File(s)	Summary
Grammar token & productions pkg/parser/hintparser.y	Introduce hintNumericLit token and extend Value production to accept hintNumericLit, + hintNumericLit, and - hintNumericLit (signed numeric literal handling).
Lexer state machine & token reclassification pkg/parser/hintparserimpl.go	Add hintSetVarValueState in hintScanner; track SET_VAR(name=...) progress and reclassify floatLit/decLit as hintNumericLit when in value positions; initialize the state before parsing.
Generated parser tables & semantic actions pkg/parser/hintparser.go	Regenerated parser constants/tables to include hintNumericLit, renumbered symbols/gotos, updated yyhintParseTab and yyhintReductions, yyError column change, and renumbered reduction actions that set ident, numeric string values, negation, scaled numbers, and boolean hint data.
Parser wiring / token flow pkg/parser/hintparserimpl.go, pkg/parser/hintparser.go	Wrap lexer token returns to update setVarValueState before the parser receives tokens so hintNumericLit tokens flow into the parser and are handled by the new productions.
Unit and integration tests pkg/parser/hintparser_test.go, pkg/planner/core/hint_test.go	Parser tests: add QB_NAME(1.5) negative case, change set_var(timestamp = 1.5) to parse successfully with Value: \"1.5\", and add sel=0.3, sel_plus=+0.3, sel_minus=-0.3 cases. Add planner test TestSetVarDecimalValueHintsWork validating runtime application and session binding persistence for decimal set_var hints.

sequenceDiagram
    participant Client
    participant Lexer as hintScanner
    participant Parser
    participant Planner
    participant SessionStore

    Client->>Parser: send SQL with /*+ set_var(...0.3...) */
    Parser->>Lexer: request next token stream
    Lexer->>Lexer: track setVarValueState (name, '=', optional sign)
    Lexer-->>Parser: emit `hintNumericLit("0.3")` when in value state
    Parser->>Parser: reduce `Value` using hintNumericLit (apply sign if '-')
    Parser->>Planner: produce hint AST with value "0.3" or "-0.3"
    Planner->>SessionStore: apply hint for this query / create binding
    SessionStore-->>Planner: confirm applied
    Planner-->>Client: return result reflecting overridden variable

Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Poem

🐰 A nibble of floats, now heard by the scanner,

0.3 hops in where integers were mannered.
The lexer keeps watch, the parser agrees,
Hints wear their decimals with elegant ease.
Hooray for the hop that fixed SET_VAR’s pleas!

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title 'parser: allow decimal values in SET_VAR hints' clearly and concisely summarizes the main change—enabling decimal numeric support in SET_VAR optimizer hints.
Description check	✅ Passed	The PR description follows the template with linked issue #64251, clear problem statement, technical explanation of changes, completed test checkboxes, release note, and manual test commands provided.
Linked Issues check	✅ Passed	The PR fully addresses issue #64251 requirements: allows decimal/floating-point values in SET_VAR hint value positions via lexer state tracking and grammar updates, with comprehensive test coverage added.
Out of Scope Changes check	✅ Passed	All changes are in-scope: parser grammar/lexer modifications for SET_VAR decimal support, regenerated parser tables, and targeted test coverage for the feature; no unrelated refactoring or external changes present.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

🧹 Nitpick comments (1)

pkg/parser/hintparser_test.go (1)

207-250: ⚡ Quick win

Add signed-decimal coverage for the new SET_VAR path.

The new parser path also adds +/- handling for decimal literals, but these cases only exercise unsigned decimals. A small parser case for set_var(...=+0.3) and set_var(...=-0.3) would lock down the new AfterSign lexer state and catch regressions there.

As per coding guidelines, "Keep test changes minimal and deterministic; avoid broad golden/testdata churn unless required."

Also applies to: 341-349

🤖 Prompt for AI Agents

Verify each finding against the current code and only fix it if needed.

In `@pkg/parser/hintparser_test.go` around lines 207 - 250, Add minimal test cases
exercising signed decimals through the new AfterSign lexer state by extending
the existing SET_VAR test row in hintparser_test.go: include two additional hint
invocations in the input string, e.g. set_var(sel=+0.3) and set_var(sel=-0.3),
and add corresponding entries to the expected output slice using ast.HintSetVar
with VarName "sel" (or distinct names like "sel_plus"/"sel_minus") and Value
"+0.3" and "-0.3" respectively; ensure you reference the same Test case that
contains the other SET_VAR entries so the AfterSign path is covered without
broad changes.

🤖 Prompt for all review comments with AI agents

Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In `@pkg/parser/hintparser_test.go`:
- Around line 207-250: Add minimal test cases exercising signed decimals through
the new AfterSign lexer state by extending the existing SET_VAR test row in
hintparser_test.go: include two additional hint invocations in the input string,
e.g. set_var(sel=+0.3) and set_var(sel=-0.3), and add corresponding entries to
the expected output slice using ast.HintSetVar with VarName "sel" (or distinct
names like "sel_plus"/"sel_minus") and Value "+0.3" and "-0.3" respectively;
ensure you reference the same Test case that contains the other SET_VAR entries
so the AfterSign path is covered without broad changes.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: 5d0c5ee8-9004-408a-a54c-510034fcc75e

📥 Commits

Reviewing files that changed from the base of the PR and between 0e5e814 and 491abbc.

📒 Files selected for processing (5)

• pkg/parser/hintparser.go
• pkg/parser/hintparser.y
• pkg/parser/hintparser_test.go
• pkg/parser/hintparserimpl.go
• pkg/planner/core/hint_test.go

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 76.5343%. Comparing base (00a509d) to head (88ab574).
⚠️ Report is 7 commits behind head on master.

Additional details and impacted files

@@               Coverage Diff                @@
##             master     #68128        +/-   ##
================================================
- Coverage   77.7015%   76.5343%   -1.1673%     
================================================
  Files          1991       1992         +1     
  Lines        552249     559800      +7551     
================================================
- Hits         429106     428439       -667     
- Misses       122223     131153      +8930     
+ Partials        920        208       -712     

Flag	Coverage Δ
integration	41.6666% <ø> (+1.8725%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components	Coverage Δ
dumpling	60.4888% <ø> (ø)
parser	∅ <ø> (∅)
br	49.9725% <ø> (-13.0354%)	⬇️

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[hawkingrei]

/check-issue-triage-complete

[D3Hunter]

Summary

• Total findings: 3
• Inline comments: 3
• Summary-only findings (no inline anchor): 0

Findings (highest risk first)

⚠️ [Major] (1)

1. Decimal SET_VAR hints lack persisted-binding upgrade coverage (pkg/parser/hintparser.y:310, pkg/parser/hintparserimpl.go:157, pkg/planner/core/hint_test.go:74)

🟡 [Minor] (1)

1. Context-sensitive SET_VAR lexer state has no intent comment (pkg/parser/hintparserimpl.go:40)

🧹 [Nit] (1)

1. hintNumericLit name is broader than the token it represents (pkg/parser/hintparser.y:54)

[hawkingrei]

Addressed the CodeRabbit signed-decimal nit in 13f45cb as well. The existing SET_VAR parser test now covers both set_var(sel_plus=+0.3) and set_var(sel_minus=-0.3).

[hawkingrei]

/retest

[ti-chi-bot]

[LGTM Timeline notifier]

Timeline:

• 2026-05-13 07:07:35.230979234 +0000 UTC m=+249423.763758543: ☑️ agreed by D3Hunter.
• 2026-05-13 07:36:10.290828228 +0000 UTC m=+251138.823607547: ☑️ agreed by qw4990.

[ti-chi-bot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: D3Hunter, qw4990, yudongusa

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

• pkg/parser/OWNERS [D3Hunter,yudongusa]
• pkg/planner/OWNERS [qw4990]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment