Skip to content

feat(grpcserver): gRPC adapter and thin standalone main (PR D) - #538

Open
steiler wants to merge 3 commits into
feat/core-rpcsfrom
feat/grpcserver-adapter
Open

steiler wants to merge 3 commits into
feat/core-rpcsfrom
feat/grpcserver-adapter

Conversation

@steiler

@steiler steiler commented Nov 9, 2026

Copy link
Copy Markdown
Collaborator

Stacked on feat/core-rpcs (PR C); retarget to main once C lands.

What

  • Adds pkg/grpcserver: sdcpb to pkg/api adapter for DataServer and SchemaServer on top of pkg/core, with ready gate, 30m timeout, request-uuid logging, Prometheus, TLS option and max recv size.
  • Keeps wire parity with pkg/server: same codes and messages, Aborted/Unavailable/ResourceExhausted/DeadlineExceeded mapping on the transaction path.
  • api.SchemaRef gains an optional Name; the hash only includes it when set.
  • main.go is a thin wrapper: retry listen/serve errors with backoff, signal -> cancel -> stop core on a fresh context, exit 1 when the core fails to start.
  • Adds core.ListDataStores, ListIntents, DatastoreExists, SchemaClient.

Known limitations

  • Update values other than json_val are rejected with InvalidArgument (the api types carry JSON only).
  • Blame, transaction and deviation values are returned as StringVal, not the original typed values.
  • go test -race flags an existing race between connectSBI and ConnectionState in pkg/datastore; not addressed here.

Not in this PR

Release tag (cut after merge) and Go 1.25 decision.

@steiler
steiler requested a review from a team as a code owner November 9, 2026 10:46
@steiler
steiler added this pull request to stack #536 November 9, 2026 15:02
@steiler
steiler force-pushed the feat/grpcserver-adapter branch 2 times, most recently from 2aa2eee to 9761a80 Compare November 9, 2026 15:56
steiler added a commit to sdcio/config-server that referenced this pull request Nov 10, 2026
Load the data-server config (DATA_SERVER_CONFIG, default
/config/data-server.yaml), build the core and gRPC adapter in main and
run both as manager Runnables, so :56000 is served by the controller
process. TargetManager and :9443 are now always built. readyz waits for
the core, and a health probe address is added (HEALTH_PROBE_PORT,
default :8090).

TargetManager takes a narrow DatastoreAdmin handle; DeleteDatastore runs
in-process against the core and treats api.ErrUnknownDatastore as gone.
LOCAL_DATASERVER is no longer read. The subscription reconciler gets the
nil TargetManager guard.

Pairs with: sdcio/data-server#538

Signed-off-by: Markus Vahlenkamp <markus.vahlenkamp@nokia.com>
steiler added a commit to sdcio/config-server that referenced this pull request Nov 10, 2026
TargetManager, TargetRuntime, the transactor, the deviation watcher and the
rollout transaction manager now call the embedded core through narrow
interfaces over pkg/api instead of a gRPC client. The target datastore
reconciler builds api.DatastoreSpec directly; the datastore hash and the
retry classification use api.ComputeCreateDSHash and api.IsRetryable, and
unknown datastores are matched with api.ErrUnknownDatastore.

Removed: DSConnManager, PhaseWaitingForDS, DSReady, the fake gRPC client,
the unused sdcctx package, GetLocalDataServerAddress and the dataserverpin
placeholder. The merged binary no longer opens the data-server metrics
listener on prometheus.address.

The api-server is a separate deployment without a core, so its target
helpers and dry-run keep the :56000 client.

Pairs with: sdcio/data-server#538, sdcio/data-server#539

Signed-off-by: Markus Vahlenkamp <markus.vahlenkamp@nokia.com>
steiler added a commit to sdcio/config-server that referenced this pull request Nov 10, 2026
TargetManager, TargetRuntime, the transactor, the deviation watcher and the
rollout transaction manager now call the embedded core through narrow
interfaces over pkg/api instead of a gRPC client. The target datastore
reconciler builds api.DatastoreSpec directly; the datastore hash and the
retry classification use api.ComputeCreateDSHash and api.IsRetryable, and
unknown datastores are matched with api.ErrUnknownDatastore.

Removed: DSConnManager, PhaseWaitingForDS, DSReady, the fake gRPC client,
the unused sdcctx package, GetLocalDataServerAddress and the dataserverpin
placeholder. The merged binary no longer opens the data-server metrics
listener on prometheus.address.

The api-server is a separate deployment without a core, so its target
helpers and dry-run keep the :56000 client.

Pairs with: sdcio/data-server#538, sdcio/data-server#539

Signed-off-by: Markus Vahlenkamp <markus.vahlenkamp@nokia.com>
@steiler
steiler force-pushed the feat/grpcserver-adapter branch 2 times, most recently from a3da59d to 5bb99e9 Compare November 11, 2026 10:41
@steiler
steiler removed this pull request from stack #536 November 11, 2026 10:52
@steiler
steiler added this pull request to stack #544 November 11, 2026 10:52
@steiler
steiler force-pushed the feat/grpcserver-adapter branch from 5bb99e9 to 0365ec2 Compare November 11, 2026 11:02
@steiler
steiler force-pushed the feat/grpcserver-adapter branch from 0365ec2 to 1f6c81e Compare November 11, 2026 12:37
Wire DataServer and SchemaServer through pkg/core with interceptors,
ready gate, metrics, and handler-level error parity. Standalone main
only retries listen failures and shuts down core on signal.

Signed-off-by: Markus Vahlenkamp <markus.vahlenkamp@nokia.com>
Keep schema names through api.SchemaRef, echo the configured target type
and drop the target port in GetDataStore, and validate CreateDataStore in
the original order with the original messages. Map resource exhaustion and
deadline errors, keep untyped invalid-argument errors on the transaction
path and reject non-JSON update values instead of dropping them.

Run retries listen and serve errors with a growing delay and returns the
core start failure; main exits non-zero on it and stops the core on a
fresh context. Dedupe handler logging and error mapping, document the
caller contracts and add wire, error-table and lifecycle tests.

Signed-off-by: Markus Vahlenkamp <markus.vahlenkamp@nokia.com>
Signed-off-by: Markus Vahlenkamp <markus.vahlenkamp@nokia.com>
@steiler
steiler force-pushed the feat/grpcserver-adapter branch from 1f6c81e to d5ca75a Compare November 11, 2026 12:49
@codecov

codecov Bot commented Nov 11, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 52.79879% with 312 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
pkg/grpcserver/convert.go 49.52% 97 Missing and 9 partials ⚠️
pkg/grpcserver/schema.go 19.14% 75 Missing and 1 partial ⚠️
pkg/grpcserver/server.go 61.53% 35 Missing and 5 partials ⚠️
pkg/grpcserver/datastore.go 61.90% 19 Missing and 5 partials ⚠️
pkg/core/list.go 0.00% 23 Missing ⚠️
main.go 0.00% 14 Missing ⚠️
pkg/grpcserver/deviation.go 55.17% 13 Missing ⚠️
pkg/grpcserver/intent.go 79.31% 3 Missing and 3 partials ⚠️
pkg/grpcserver/transaction.go 85.71% 3 Missing and 3 partials ⚠️
pkg/grpcserver/errors.go 92.85% 2 Missing ⚠️
... and 1 more

📢 Thoughts on this report? Let us know!

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant