Skip to content

*: use meta service group, separate PD addrs from TiDB's required etcd addrs - #69152

Merged
ti-chi-bot[bot] merged 33 commits into
pingcap:masterfrom
ystaticy:yaojingyi/use_metaservice
Jun 25, 2026
Merged

ti-chi-bot[bot] merged 33 commits into
pingcap:masterfrom
ystaticy:yaojingyi/use_metaservice

Conversation

@ystaticy

@ystaticy ystaticy commented Jun 12, 2026 •

Copy link
Copy Markdown
Contributor

What problem does this PR solve?

Issue Number: ref #68338

Problem Summary:

What changed and how does it work?

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No need to test
    • I checked and no code files have been changed.

Side effects

  • Performance regression: Consumes more CPU
  • Performance regression: Consumes more Memory
  • Breaking backward compatibility

Documentation

  • Affects user behaviors
  • Contains syntax changes
  • Contains variable changes
  • Contains experimental features
  • Changes MySQL compatibility

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

*: use meta service group, separate PD addrs from TiDB's required etcd addrs

Summary by CodeRabbit

  • Refactor

    • Integrated keyspace-aware meta-service for PD address and metadata resolution, replacing older PD lookup paths and centralizing meta information caching.
    • Updated server/store startup and address resolution to use meta-service results, improving TLS and endpoint handling consistency.
  • Tests

    • Expanded test scaffolding to exercise meta-service scenarios and validate keyspace/meta address resolution.

ystaticy added 2 commits June 12, 2026 18:47
Signed-off-by: ystaticy <y_static_y@sina.com>
Signed-off-by: ystaticy <y_static_y@sina.com>
@ti-chi-bot ti-chi-bot Bot added do-not-merge/invalid-title do-not-merge/needs-linked-issue do-not-merge/needs-tests-checked release-note-none Denotes a PR that doesn't merit a release note. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Jun 12, 2026
@ystaticy ystaticy changed the title Yaojingyi/use metaservice *: use metaservice Jun 12, 2026
@coderabbitai

coderabbitai Bot commented Jun 12, 2026 •

Copy link
Copy Markdown

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

Adds metaservice-based PD discovery: EtcdBackend gains GetPDAddrs/MetaServiceInfo, TiKV driver and consumers are refactored to use metaservice info, infosync stores a metaservice client, and tests/mocks/BUILD deps are updated to match the new interface.

Changes

Metaservice PD discovery rollout

Layer / File(s) Summary
Backend contract and metaservice entrypoints
pkg/kv/BUILD.bazel, pkg/kv/kv.go, pkg/metaservice/etcd.go
EtcdBackend interface gains GetPDAddrs() and MetaServiceInfo(); pkg/metaservice exports NewEtcdMetaServiceClient; kv library BUILD deps updated.
TiKV store metaservice-backed implementation
pkg/store/driver/BUILD.bazel, pkg/store/driver/tikv_driver.go
OpenWithOptions now resolves PD via metaservice, constructs keyspace meta info, caches metaServiceInfo on tikvStore, and EtcdAddrs()/GetPDAddrs/MetaServiceInfo are rewritten accordingly.
Infosync client initialization
pkg/domain/infosync/BUILD.bazel, pkg/domain/infosync/info.go
Infosync imports metaservice, adds metaServiceClient to InfoSyncer, and initializes it with NewEtcdMetaServiceClient in GlobalInfoSyncerInit.
Consumers migrate to GetPDAddrs
pkg/executor/..., pkg/infoschema/tables.go, pkg/server/handler/tikvhandler/tikv_handler.go, pkg/store/helper/*
Executor, infoschema, server handler, and helper switch from EtcdAddrs() to GetPDAddrs(); corresponding test imports/BUILD deps updated.
Store and mockstore updates
pkg/store/mockstore/mockstorage/*, pkg/store/etcd_test.go, pkg/store/BUILD.bazel
Mock storage adds GetPDAddrs()/MetaServiceInfo(); EtcdAddrs() now reads group addresses from MetaServiceInfo(); mock BUILD deps updated.
Server build and keyspace metadata test
pkg/server/BUILD.bazel, pkg/server/server_test.go
Server tests add metaservice dep and a TestGetKeyspaceMetaServiceAddrs to validate address and TLS config passthrough.
Test mock conformance across packages
pkg/ddl/*, pkg/domain/*, pkg/session/*, pkg/executor/*, pkg/store/helper/*
Test mocks extended with GetPDAddrs() and MetaServiceInfo(); many go_test rules add //pkg/metaservice to deps.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related PRs

  • pingcap/tidb#68818: Introduces metaservice group APIs and helpers that this change consumes.

Suggested labels

size/XXL, ok-to-test

Suggested reviewers

  • D3Hunter
  • YangKeao
  • yudongusa

Poem

🐰 I found PD addrs in a hop and a bound,
Metaservice whispers the keyspace around.
Mocks now conform, tests sing along,
TiKV caches the chorus, sturdy and strong.
Hooray — the cluster map finally found!

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (2 warnings)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 44.44% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
Description check ⚠️ Warning The PR description is largely incomplete with missing critical sections: 'Problem Summary' and 'What changed and how does it work?' are empty, leaving the rationale and implementation details unclear despite substantial code changes. Fill in the 'Problem Summary' section explaining the issue being addressed, and complete 'What changed and how does it work?' with details about how metaservice is integrated and how PD addresses are now separated from TiDB's etcd requirements.
✅ Passed checks (3 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly summarizes the main objective: introducing metaservice for managing PD addresses separately from TiDB's etcd addresses. It accurately reflects the core changes across the codebase.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@tiprow

tiprow Bot commented Jun 12, 2026

Copy link
Copy Markdown

Hi @ystaticy. Thanks for your PR.

PRs from untrusted users cannot be marked as trusted with /ok-to-test in this repo meaning untrusted PR authors can never trigger tests themselves. Collaborators can still trigger tests on the PR using /test all.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 5

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@pkg/metaservice/etcd.go`:
- Around line 31-34: NewEtcdMetaServiceClient currently delegates to newClient
which treats a nil etcdCli as a hard failure, preventing PD-only setups from
exposing GetPDAddrs/GetPDHttpAddrs; change the construction so that
NewEtcdMetaServiceClient returns a usable ServiceClient when pdCli != nil even
if etcdCli == nil by updating newClient (or NewEtcdMetaServiceClient) to branch:
if etcdCli != nil build the full etcd+PD client as before, otherwise build a
PD-only client that implements the PD-related methods
(GetPDAddrs/GetPDHttpAddrs); ensure the PD-only client type satisfies
ServiceClient and that GlobalInfoSyncerInit can store it unconditionally.

In `@pkg/server/http_status.go`:
- Around line 591-598: The log message for the EtcdAddrs() error is
misleading—update the error log in the block handling "etcdAddr, err :=
ebd.EtcdAddrs()" to reflect that EtcdAddrs() failed (e.g., "failed to get etcd
addresses from EtcdBackend" or "EtcdAddrs() returned error") instead of "tikv
store not etcd backend"; keep the same conditional branch using intest.InTest
and use logutil.BgLogger().Panic / .Error with zap.Error(err) and the
ebd/EtcdAddrs() context.

In `@pkg/store/driver/tikv_driver.go`:
- Around line 257-266: OpenWithOptions currently writes a single metaServiceInfo
pointer into the tikvStore struct (metaServiceInfo) and
MetaServiceInfo()/EtcdAddrs() blindly return that cached pointer forever; change
this so the store does not freeze discovery: remove the long-lived cached
metaServiceInfo usage and instead store a meta service client/provider or a 작은
fetch function on tikvStore (e.g., metaServiceClient or metaServiceProvider) and
update MetaServiceInfo() to call that provider each time (or refresh on TTL) so
EtcdAddrs() always gets current endpoints; update OpenWithOptions to initialize
the client/provider rather than a static pointer and keep symbol references:
OpenWithOptions, tikvStore.metaServiceInfo, MetaServiceInfo(), and EtcdAddrs().
- Around line 221-229: The code dereferences metaServiceInfo.Group.Addrs without
checking for nil after calling metaservice.GetInfo, which can panic when no
group is resolved; update the block after metaservice.GetInfo to guard that
metaServiceInfo and metaServiceInfo.Group are non-nil and return a clear error
if missing before calling NewEtcdSafePointKVWithKeyspacePrefixIfNeeded (refer to
metaservice.GetInfo, metaServiceInfo.Group.Addrs,
NewEtcdSafePointKVWithKeyspacePrefixIfNeeded, and spkv); ensure the error
message references the keyspace (codec.GetKeyspaceMeta()) to make the failure
actionable.

In `@pkg/store/etcd_test.go`:
- Around line 36-47: The mockEtcdBackend stub returns the same slice for PD and
metaservice endpoints which hides regressions; change
mockEtcdBackend.GetPDAddrs() to return a pdAddrs slice and have
MetaServiceInfo() populate Group.Addrs with a distinct metaAddrs slice (both
stored on mockEtcdBackend fields) so tests can distinguish them, then update the
test assertion to call GetEtcdAddrs() and assert it returns metaAddrs (not
pdAddrs); update any references to pdAddrs/metaAddrs within
pkg/store/etcd_test.go accordingly.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: 81e2ab97-bbc7-464e-a441-08d529dcede9

📥 Commits

Reviewing files that changed from the base of the PR and between 74e231b and 52d6c43.

📒 Files selected for processing (32)
  • lightning/pkg/importer/BUILD.bazel
  • lightning/pkg/importer/import.go
  • pkg/ddl/BUILD.bazel
  • pkg/ddl/integration_test.go
  • pkg/ddl/owner_mgr_test.go
  • pkg/domain/BUILD.bazel
  • pkg/domain/domain_test.go
  • pkg/domain/infosync/BUILD.bazel
  • pkg/domain/infosync/info.go
  • pkg/executor/BUILD.bazel
  • pkg/executor/hot_regions_history_table_test.go
  • pkg/executor/infoschema_cluster_table_test.go
  • pkg/executor/split.go
  • pkg/infoschema/tables.go
  • pkg/kv/BUILD.bazel
  • pkg/kv/kv.go
  • pkg/metaservice/etcd.go
  • pkg/server/BUILD.bazel
  • pkg/server/handler/tikvhandler/tikv_handler.go
  • pkg/server/http_status.go
  • pkg/server/server_test.go
  • pkg/session/BUILD.bazel
  • pkg/session/bootstrap_test.go
  • pkg/store/BUILD.bazel
  • pkg/store/driver/BUILD.bazel
  • pkg/store/driver/tikv_driver.go
  • pkg/store/etcd_test.go
  • pkg/store/helper/BUILD.bazel
  • pkg/store/helper/helper.go
  • pkg/store/helper/helper_test.go
  • pkg/store/mockstore/mockstorage/BUILD.bazel
  • pkg/store/mockstore/mockstorage/storage.go

Comment thread pkg/metaservice/etcd.go Resolved
Comment thread pkg/server/http_status.go Resolved
Comment thread pkg/store/driver/tikv_driver.go Resolved
Comment thread pkg/store/driver/tikv_driver.go Resolved
Comment thread pkg/store/etcd_test.go Resolved
@codecov

codecov Bot commented Jun 12, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 6.30631% with 104 lines in your changes missing coverage. Please review.
✅ Project coverage is 76.1048%. Comparing base (42edcdf) to head (6145d0c).
⚠️ Report is 4 commits behind head on master.

Additional details and impacted files
@@               Coverage Diff                @@
##             master     #69152        +/-   ##
================================================
- Coverage   76.3204%   76.1048%   -0.2157%     
================================================
  Files          2041       2072        +31     
  Lines        561097     577803     +16706     
================================================
+ Hits         428232     439736     +11504     
- Misses       131964     136063      +4099     
- Partials        901       2004      +1103     
Flag Coverage Δ
integration 45.8636% <6.3063%> (+6.2356%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
dumpling 60.4610% <ø> (ø)
parser ∅ <ø> (∅)
br 64.1111% <30.7692%> (+1.3600%) ⬆️
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

ystaticy added 2 commits June 13, 2026 16:34
Signed-off-by: ystaticy <y_static_y@sina.com>
Signed-off-by: ystaticy <y_static_y@sina.com>
@ystaticy ystaticy changed the title *: use metaservice *: use metaservice, separate PD addrs from TiDB's required etcd addrs Jun 13, 2026
ystaticy added 13 commits June 13, 2026 17:05
Signed-off-by: ystaticy <y_static_y@sina.com>
Signed-off-by: ystaticy <y_static_y@sina.com>
Signed-off-by: ystaticy <y_static_y@sina.com>
Signed-off-by: ystaticy <y_static_y@sina.com>
Signed-off-by: ystaticy <y_static_y@sina.com>
Signed-off-by: ystaticy <y_static_y@sina.com>
Signed-off-by: ystaticy <y_static_y@sina.com>
Signed-off-by: ystaticy <y_static_y@sina.com>
Signed-off-by: ystaticy <y_static_y@sina.com>
Signed-off-by: ystaticy <y_static_y@sina.com>
Signed-off-by: ystaticy <y_static_y@sina.com>
Signed-off-by: ystaticy <y_static_y@sina.com>
Signed-off-by: ystaticy <y_static_y@sina.com>
@ystaticy

Copy link
Copy Markdown
Contributor Author

/retest-required

Signed-off-by: ystaticy <y_static_y@sina.com>
Comment thread pkg/metaservice/etcd.go Resolved
Comment thread pkg/metaservice/metamanager.go Resolved
Comment thread pkg/metaservice/metamanager.go Resolved
Comment thread pkg/metaservice/metamanager.go Resolved
@D3Hunter

Copy link
Copy Markdown
Contributor

rest lgtm

ystaticy added 3 commits June 22, 2026 17:35
Signed-off-by: ystaticy <y_static_y@sina.com>
Signed-off-by: ystaticy <y_static_y@sina.com>
Signed-off-by: ystaticy <y_static_y@sina.com>
@ti-chi-bot ti-chi-bot Bot added approved needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels Jun 24, 2026
@ti-chi-bot ti-chi-bot Bot added lgtm and removed needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels Jun 24, 2026
@ti-chi-bot

ti-chi-bot Bot commented Jun 24, 2026

Copy link
Copy Markdown

[LGTM Timeline notifier]

Timeline:

  • 2026-06-24 02:10:27.581381733 +0000 UTC m=+86311.244607246: ☑️ agreed by D3Hunter.
  • 2026-06-24 02:17:57.442750652 +0000 UTC m=+86761.105976165: ☑️ agreed by YangKeao.

Signed-off-by: ystaticy <y_static_y@sina.com>
@ystaticy

Copy link
Copy Markdown
Contributor Author

/retest-required

@ystaticy

Copy link
Copy Markdown
Contributor Author

/test pull-br-integration-test

1 similar comment
@ystaticy

Copy link
Copy Markdown
Contributor Author

/test pull-br-integration-test

@ystaticy

Copy link
Copy Markdown
Contributor Author

/retest-required

Signed-off-by: ystaticy <y_static_y@sina.com>
@ti-chi-bot ti-chi-bot Bot removed the approved label Jun 25, 2026
@ti-chi-bot

ti-chi-bot Bot commented Jun 25, 2026

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: D3Hunter, YangKeao, YuJuncen

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ti-chi-bot ti-chi-bot Bot added the approved label Jun 25, 2026
Comment thread br/pkg/task/restore.go Resolved
@D3Hunter D3Hunter mentioned this pull request Jun 25, 2026
1 of 13 tasks
Signed-off-by: ystaticy <y_static_y@sina.com>
@ti-chi-bot
ti-chi-bot Bot merged commit d5729fb into pingcap:master Jun 25, 2026
37 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved lgtm release-note Denotes a PR that will be considered when it comes time to generate release notes. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants