Skip to content

server: KILL SELECT SLEEP can trigger nil pointer recover in exec.Next #69352

Closed

Description

@together-wang

Bug Report

1. Minimal reproduce step (Required)

Start a local TiDB cluster with multiple TiDB instances:

tiup playground v8.5.7-pre \
  --db 3 --pd 1 --kv 1 --tiflash 0 \
  --without-monitor \
  --host 127.0.0.1 \
  --db.port 4000 \
  --pd.port 2379

The same issue can also be reproduced with v8.5.6.

Create the test database:

mysql -h127.0.0.1 -P4000 -uroot -e 'CREATE DATABASE IF NOT EXISTS test;'

Run a concurrent workload that repeatedly:

  1. Opens a target connection.
  2. Gets its connection ID with SELECT CONNECTION_ID().
  3. Starts SELECT SLEEP(30) on the target connection.
  4. From another connection, runs KILL <connection_id>.
  5. Waits for the sleep query to return.

Example command used locally:

go run repro.go \
  -dsn 'root@tcp(127.0.0.1:4000)/test' \
  -threads 200 \
  -duration 2m \
  -wait-sleep 200ms

The core reproduction logic is:

target, _ := db.Conn(ctx)
var connID int64
_ = target.QueryRowContext(ctx, "SELECT CONNECTION_ID()").Scan(&connID)

done := make(chan error, 1)
go func() {
    rows, err := target.QueryContext(context.Background(), "SELECT SLEEP(30)")
    if rows != nil {
        _ = rows.Close()
    }
    done <- err
}()

time.Sleep(200 * time.Millisecond)
_, _ = killer.ExecContext(ctx, fmt.Sprintf("KILL %d", connID))
<-done

2. What did you expect to see? (Required)

The killed SELECT SLEEP(30) should return a normal interruption/connection-close error to the client, and TiDB should not log an internal panic/recover.

3. What did you see instead (Required)

TiDB occasionally logs an internal nil pointer recover while handling the killed SELECT SLEEP(30):

[INFO] [conn.go:1222] ["command dispatched failed"]
[sql="SELECT SLEEP(30)"]
[err="runtime error: invalid memory address or nil pointer dereference
github.com/pingcap/errors.Trace
    .../github.com/pingcap/errors/.../juju_adaptor.go:15
github.com/pingcap/tidb/pkg/util.GetRecoverError
    .../tidb/pkg/util/util.go:290
github.com/pingcap/tidb/pkg/executor/internal/exec.Next.func1
    .../tidb/pkg/executor/internal/exec/executor.go:440
runtime.gopanic
runtime.panicmem
runtime.sigpanic
github.com/pingcap/tidb/pkg/executor/internal/exec.Next
    .../tidb/pkg/executor/internal/exec/executor.go:443
github.com/pingcap/tidb/pkg/executor.(*ExecStmt).next
    .../tidb/pkg/executor/adapter.go:1377
github.com/pingcap/tidb/pkg/executor.(*recordSet).Next
    .../tidb/pkg/executor/adapter.go:172
github.com/pingcap/tidb/pkg/server/internal/resultset.(*tidbResultSet).Next
    .../tidb/pkg/server/internal/resultset/resultset.go:72
github.com/pingcap/tidb/pkg/server.(*clientConn).writeChunks
github.com/pingcap/tidb/pkg/server.(*clientConn).writeResultSet
github.com/pingcap/tidb/pkg/server.(*clientConn).handleStmt
github.com/pingcap/tidb/pkg/server.(*clientConn).handleQuery
github.com/pingcap/tidb/pkg/server.(*clientConn).dispatch
github.com/pingcap/tidb/pkg/server.(*clientConn).Run
github.com/pingcap/tidb/pkg/server.(*Server).onConn
runtime.goexit"]

Local reproduction results:

  • v8.5.7-pre: 114,297 kill attempts in 2 minutes, 4 matching nil pointer recover logs.
  • v8.5.6: 114,958 kill attempts in 2 minutes, 1 matching nil pointer recover log.

The client-side invalid connection / unexpected EOF errors are expected for this workload. The unexpected part is the TiDB-side internal nil pointer recover.

4. What is your TiDB version? (Required)

Reproduced with tiup playground v8.5.7-pre:

Release Version: v8.5.7
Edition: Community
Git Commit Hash: ef954aa65764c336eebc831b14f803cbc70c6e42
Git Branch: HEAD
UTC Build Time: 2026-06-12 09:59:55
GoVersion: go1.25.10
Race Enabled: false
Check Table Before Drop: false
Store: tikv

Also reproduced with tiup playground v8.5.6:

Release Version: v8.5.6
Edition: Community
Git Commit Hash: ae18096e023780bb56bfce33698abec0d4640d0a
Git Branch: HEAD
UTC Build Time: 2026-04-14 07:11:54
GoVersion: go1.25.8
Race Enabled: false
Check Table Before Drop: false
Store: tikv

Additional context

This was found by the globalkill-global_kill test scenario, which stresses global kill behavior by repeatedly killing sessions that are executing SELECT SLEEP(30).

Related but not exact duplicates found during search:

Activity

  1. together-wang commented on Jun 22, 2026

    @together-wang
    Author

    /type bug

  2. together-wang commented on Jun 23, 2026

    @together-wang
    Author

    /severity major

  3. together-wang commented on Jun 23, 2026

    @together-wang
    Author

    /remove-label may-affects-7.5

  4. together-wang commented on Jun 23, 2026

    @together-wang
    Author

    /remove-label may-affects-8.1

  5. together-wang commented on Jun 23, 2026

    @together-wang
    Author

    /remove-label may-affects-8.5

  6. added a commit that references this issue on Jun 24, 2026
    b53834a
  7. added a commit that references this issue on Aug 20, 2026
    9bc03c8
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    contributionThis PR is from a community contributor.severity/majortype/bugThe issue is confirmed as a bug.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions