rpcv2 integration tests: run the rpcv1 suite unchanged against rpcv2, four CI legs

[karthikiyer56]

Closes #883 (task 0 of #986).

What this PR does

• Runs the shared integration suite against either daemon. STELLAR_RPC_INTEGRATION_TESTS_DAEMON picks rpcv1 (default) or rpcv2.
• Adds two CI legs: stellar-rpc.yml now runs a protocol x daemon matrix (P27, P28) x (rpcv1, rpcv2).
• Keeps every assertion identical for both daemons.

Layout

package	contents	runs on
cmd/stellar-rpc/internal/integrationtest/	the harness (infrastructure/) and 13 daemon-neutral test files	both daemons
cmd/stellar-rpc/internal/rpcv1/integrationtest/	archive_test.go, backfill_test.go, ingest_loadtest_test.go, migrate_test.go, get_ledgers_datastore_test.go, health_retention_test.go, TestMain with the fake GCS server	rpcv1 legs
cmd/stellar-rpc/internal/rpcv2/integrationtest/	health_retention_test.go; the home for rpcv2-only tests	rpcv2 legs

Where the changes are

area	files	kind of change
Moves (git mv)	18 files identical, 14 with the import path and infrastructure.SkipLimitsUpgrade() only; infrastructure/docker/ byte-identical	mechanical
Harness, new	infrastructure/daemon.go, daemon_rpcv1.go, daemon_rpcv2.go, docker/rpcv2-integration-tests.toml, rpcv2_config_test.go, skip_limits.go	- the harness drives the daemon through a four-method interface - daemon_rpcv1.go is today's boot code moved behind it - daemon_rpcv2.go boots rpcv2 in-process: three captive-core ports from getFreeTCPPorts, the daemon's own listeners on port 0, the sample TOML minus the datastore tables, per-test values as flag overrides - rpcv2_config_test.go decodes the test TOML and the sample and requires them equal outside the three deliberate differences (no datastore tables, enable_debug, log level)
Harness, test.go	infrastructure/test.go	- waitForRPC returns when the daemon is healthy and caught up with the Core container - the limits upgrade also waits until the daemon has ingested the ledger that applied it, since preflight reads state from captive core, not the container - every address the harness dials is 127.0.0.1, never localhost - a daemon that exits before it is healthy fails the test at once - under rpcv2, every rpcv1-only TestConfig setting fails the test at startup instead of being ignored
rpcv2	cmd/stellar-rpc/internal/rpcv2/daemon.go, serve.go, startup.go, metrics_test.go	- Options and RunDaemonWithOptions: an exported entry point taking a logger, flag overrides, and an OnListen callback - OnListen reports the bound read and admin addresses once, so a caller may configure port 0 - the process metrics (log-line counters, soroban_rpc_build_info, Go and process collectors) come from host.RegisterProcessMetrics, so TestMetrics passes unchanged - nothing else under rpcv2/ changes
shared	cmd/stellar-rpc/internal/host/metrics.go	- RegisterProcessMetrics: the one copy of the process-metrics registration both daemons call, so the label set cannot drift between them
rpcv1	cmd/stellar-rpc/internal/rpcv1/daemon/metrics.go	- registerMetrics calls host.RegisterProcessMetrics; same families, same labels - the rpcv1-only tests change only their import path and GetDaemon() -> infrastructure.RPCv1Daemon(t, test) - in the harness, rpcv1 now submits through its own captive core (STELLAR_CAPTIVE_CORE_HTTP_PORT set, STELLAR_CORE_URL pointed at it), as it does by default in production
Tests, five edits and one comment	health_test.go, upgrade_test.go, metrics_test.go, get_ledgers_test.go (x2 files)	- TestHealth: retention-window line moves to a per-daemon test - TestUpgradeFrom20To21: latest ledger via getLatestLedger, not SQLite - TestMetrics: soroban_rpc_log_error_total read from /metrics, not the registry - TestGetLedgersFromDatastore: split into the rpcv1 package - TestGetLedgers: beyond-latest request asks for LatestLedger + 1000 - TestGetLedgers: a comment explains why OldestLedger - 4 wraps under rpcv2 and why the assertion still holds
CI and tooling	integration-tests.yml, stellar-rpc.yml, scripts/check-dependencies.bash, .golangci.yml, README.md	- daemon input; each leg runs the shared package plus that daemon's package - the dependency checker reads the protocol matrix - depguard lets the shared harness import both trees

Latent problems fixed along the way

problem	where it showed	fix
healthy does not mean caught up. A daemon that starts at genesis reports healthy as soon as its first ledger commits, because with 1 s test ledgers every ledger it still has to replay is inside the 30 s latency window. A test then submitted into a captive core still replaying, or read a protocol-0 ledger.	rpcv2 (TestGetTransactions, TestGetVersionInfoSucceeds, TestSendTransactionFailedInLedger)	waitForRPC also waits until the daemon's latest ledger reaches the Core container's ledger. Tests that delay the daemon on purpose (DelayDaemonForLedgerN) are exempt.
The limits upgrade was checked on the Core container's /sorobaninfo, but preflight reads state from the daemon's captive core, which closes the same ledger a moment later. A simulateTransaction sent in that gap ran with the old limits (Error(Budget, ExceededLimit)).	both daemons (TestGetLedgerEntriesSucceeds)	after the /sorobaninfo check, wait until the daemon has ingested the upgrade ledger.
Every harness client dialed localhost. Go tries the IPv6 side first, and on Docker Desktop for macOS a forwarder can hold the IPv6 side of a port whose IPv4 side the daemon or a container bound; the client connected to the forwarder and was reset for the whole 180 s health wait.	both daemons, local bench	every address the harness dials is 127.0.0.1.
A daemon that died before it was healthy failed the test only after the 180 s health wait. A CI leg runs two test binaries that could draw the same port range (pid % 10), so one test's RPC port could be another's captive-core peer port.	both daemons	waitForRPC watches the daemon's exit and fails the test at once with the exit error. Each test process claims one of 25 port ranges with an exclusive flock on a temp-dir lock file for its lifetime, so two binaries never share a range. rpcv2's own listeners bind port 0 and report their ports back, so only the three captive-core ports are chosen in advance. rpcv1's exit hook reports through the same exit channel.
TestGetLedgers asked for LatestLedger + 1 two calls after learning the latest ledger. On a 1 s ledger network that ledger sometimes existed by then, so the expected error did not come.	both daemons	the beyond-latest request asks for LatestLedger + 1000. Same assertion.
rpcv1 in the harness submitted transactions to the Core container, while rpcv1's own default (and rpcv2's only path) is the daemon's captive core.	rpcv1 harness	rpcv1 gets a captive-core admin HTTP port and STELLAR_CORE_URL points at it.
scripts/check-dependencies.bash read protocol and Core version pairs out of stellar-rpc.yml with an awk that only understood one literal protocol_version per job.	CI, dependency sanity checker	the awk also reads a protocol: [...] matrix list; the old and new workflow produce the same pairs.
The harness waited for Core's /info to pass the checkpoint ledger, but the container publishes that checkpoint to the history archive a few seconds later; until then the archive's root state says ledger 0 (new-hist writes it that way), which rpcv2 treats as a permanent startup failure.	rpcv2 (TestSendTransactionFailedInLedger, TestSimulateInvokeContractTransactionUseUpgradedAuth)	waitForArchive polls /.well-known/stellar-history.json until it reports the first checkpoint before any daemon starts.

| The rpcv1 harness ended a failed daemon start with a bare runtime.Goexit on the test goroutine. Go's test runner treats that as a panic and aborts the whole test binary. | rpcv1 harness, only when the daemon fails to build | the exit hook calls t.Fatal while the daemon is being built and t.Error plus Goexit from daemon goroutines. |

[Copilot]

Pull request overview

Moves daemon-neutral integration tests into a shared suite runnable against rpcv1 or rpcv2 and expands CI to four protocol/daemon combinations.

Changes:

• Adds a daemon abstraction and rpcv2 integration harness.
• Shares existing assertions while retaining daemon-specific tests.
• Adds rpcv2 metrics and matrix-based CI coverage.

Reviewed changes

Copilot reviewed 37 out of 55 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
.github/workflows/integration-tests.yml	Runs shared and daemon-specific suites.
.github/workflows/stellar-rpc.yml	Adds protocol/daemon matrix.
.golangci.yml	Permits shared harness imports.
README.md	Updates integration-test command.
cmd/stellar-rpc/internal/integrationtest/builtin_methods_disabled_test.go	Moves shared RPC-method test.
cmd/stellar-rpc/internal/integrationtest/cors_test.go	Moves shared CORS test.
cmd/stellar-rpc/internal/integrationtest/get_fee_stats_test.go	Moves shared fee test.
cmd/stellar-rpc/internal/integrationtest/get_ledger_entries_test.go	Moves shared entry tests.
cmd/stellar-rpc/internal/integrationtest/get_ledgers_test.go	Adds daemon-neutral ledger tests.
cmd/stellar-rpc/internal/integrationtest/get_network_test.go	Moves shared network test.
cmd/stellar-rpc/internal/integrationtest/get_transactions_test.go	Moves shared transaction-list test.
cmd/stellar-rpc/internal/integrationtest/get_version_info_test.go	Moves shared version test.
cmd/stellar-rpc/internal/integrationtest/health_test.go	Makes health assertions daemon-neutral.
cmd/stellar-rpc/internal/integrationtest/infrastructure/client.go	Moves client helpers.
cmd/stellar-rpc/internal/integrationtest/infrastructure/contract.go	Moves contract helpers and fixes paths.
cmd/stellar-rpc/internal/integrationtest/infrastructure/daemon.go	Defines daemon selection and interface.
cmd/stellar-rpc/internal/integrationtest/infrastructure/daemon_rpcv1.go	Adapts rpcv1 to the harness.
cmd/stellar-rpc/internal/integrationtest/infrastructure/daemon_rpcv2.go	Adds rpcv2 lifecycle integration.
cmd/stellar-rpc/internal/integrationtest/infrastructure/docker/README.md	Moves fixture documentation.
cmd/stellar-rpc/internal/integrationtest/infrastructure/docker/captive-core-integration-tests.cfg.tmpl	Moves captive-core template.
cmd/stellar-rpc/internal/integrationtest/infrastructure/docker/core-start.sh	Moves Core startup script.
cmd/stellar-rpc/internal/integrationtest/infrastructure/docker/docker-compose.rpc.yml	Moves released-rpc compose setup.
cmd/stellar-rpc/internal/integrationtest/infrastructure/docker/docker-compose.yml	Moves Core compose setup.
cmd/stellar-rpc/internal/integrationtest/infrastructure/docker/rpcv2-integration-tests.toml	Adds rpcv2 test configuration.
cmd/stellar-rpc/internal/integrationtest/infrastructure/docker/stellar-core-integration-tests.cfg	Moves Core configuration.
cmd/stellar-rpc/internal/integrationtest/infrastructure/docker/upgrades/enable.json	Moves enable-upgrade fixture.
cmd/stellar-rpc/internal/integrationtest/infrastructure/docker/upgrades/enable.xdr	Moves encoded enable fixture.
cmd/stellar-rpc/internal/integrationtest/infrastructure/docker/upgrades/testnet.p27.json	Moves P27 testnet limits.
cmd/stellar-rpc/internal/integrationtest/infrastructure/docker/upgrades/testnet.p27.xdr	Moves encoded P27 limits.
cmd/stellar-rpc/internal/integrationtest/infrastructure/docker/upgrades/testnet.p28.json	Moves P28 testnet limits.
cmd/stellar-rpc/internal/integrationtest/infrastructure/docker/upgrades/testnet.p28.xdr	Moves encoded P28 limits.
cmd/stellar-rpc/internal/integrationtest/infrastructure/docker/upgrades/unlimited.p27.json	Moves P27 unlimited limits.
cmd/stellar-rpc/internal/integrationtest/infrastructure/docker/upgrades/unlimited.p27.xdr	Moves encoded P27 fixture.
cmd/stellar-rpc/internal/integrationtest/infrastructure/docker/upgrades/unlimited.p28.json	Moves P28 unlimited limits.
cmd/stellar-rpc/internal/integrationtest/infrastructure/docker/upgrades/unlimited.p28.xdr	Moves encoded P28 fixture.
cmd/stellar-rpc/internal/integrationtest/infrastructure/rpcv2_config_test.go	Validates rpcv2 test configuration.
cmd/stellar-rpc/internal/integrationtest/infrastructure/skip_limits.go	Exposes limits-skip helper.
cmd/stellar-rpc/internal/integrationtest/infrastructure/test.go	Generalizes harness lifecycle and readiness.
cmd/stellar-rpc/internal/integrationtest/infrastructure/util.go	Moves shared utility helpers.
cmd/stellar-rpc/internal/integrationtest/metrics_test.go	Tests metrics through HTTP.
cmd/stellar-rpc/internal/integrationtest/simulate_transaction_test.go	Moves simulation tests.
cmd/stellar-rpc/internal/integrationtest/transaction_test.go	Moves transaction tests.
cmd/stellar-rpc/internal/integrationtest/upgrade_test.go	Removes rpcv1 database dependency.
cmd/stellar-rpc/internal/rpcv1/integrationtest/archive_test.go	Updates harness import.
cmd/stellar-rpc/internal/rpcv1/integrationtest/backfill_test.go	Uses explicit rpcv1 adapter.
cmd/stellar-rpc/internal/rpcv1/integrationtest/get_ledgers_datastore_test.go	Retains datastore-only coverage.
cmd/stellar-rpc/internal/rpcv1/integrationtest/health_retention_test.go	Adds rpcv1 retention assertion.
cmd/stellar-rpc/internal/rpcv1/integrationtest/ingest_loadtest_test.go	Updates harness import.
cmd/stellar-rpc/internal/rpcv1/integrationtest/main_test.go	Removes local limits helper.
cmd/stellar-rpc/internal/rpcv1/integrationtest/migrate_test.go	Updates harness import.
cmd/stellar-rpc/internal/rpcv2/daemon.go	Adds options entry point and process metrics.
cmd/stellar-rpc/internal/rpcv2/integrationtest/doc.go	Documents rpcv2-only tests.
cmd/stellar-rpc/internal/rpcv2/integrationtest/health_retention_test.go	Tests rpcv2 retention semantics.
cmd/stellar-rpc/internal/rpcv2/metrics_test.go	Tests rpcv2 process metrics.
scripts/check-dependencies.bash	Parses protocol matrices.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

Pull request overview

Copilot reviewed 37 out of 55 changed files in this pull request and generated 1 comment.

[Copilot]

Pull request overview

Copilot reviewed 37 out of 55 changed files in this pull request and generated no new comments.

Suppressed comments (3)

Previously missed (2) — in code that hasn't changed since the last review.

cmd/stellar-rpc/internal/integrationtest/infrastructure/daemon_rpcv1.go:35

• This changes rpcv1's harness settings by moving transaction submission from the validator Core container to rpcv1's captive Core. Issue #883 explicitly requires rpcv1 integration-test settings to remain unchanged/byte-identical and says daemon instantiation is the only allowed difference. Preserve the prior routing, or update the linked acceptance criteria before treating this PR as closing the issue.
  cmd/stellar-rpc/internal/integrationtest/infrastructure/daemon_rpcv2.go:112
• Setting only core_http_port makes rpcv2 derive core_url as http://localhost:<port> (rpcv2/config/config.go:608-613). sendTransaction therefore still dials localhost, reintroducing the IPv6/Docker-forwarder failure that the harness's new 127.0.0.1 addressing is meant to avoid. Set the submission URL explicitly to the same IPv4 port.

This issue also appears on line 152 of the same file.

cmd/stellar-rpc/internal/integrationtest/infrastructure/daemon_rpcv2.go:153

• This check is case-sensitive, but the captive-core collision quoted in util.go:28 is bind: Address already in use. That collision will bypass the retry path and fail the test immediately. Normalize the error before matching.

func isBindError(err error) bool {
	return err != nil && strings.Contains(err.Error(), "address already in use")

[Copilot]

Pull request overview

Copilot reviewed 37 out of 55 changed files in this pull request and generated 1 comment.

Suppressed comments (1)

cmd/stellar-rpc/internal/integrationtest/infrastructure/test.go:296

• This validation omits other rpcv1-only fields: UseReleasedRPCVersion bypasses daemon selection entirely, while SQLitePath and IgnoreLedgerCloseTimes are silently ignored by rpcv2. In particular, the shared BenchmarkGetLedgerEntries supplies SQLitePath, so running it with the rpcv2 selector does not use the requested storage location. Either map those settings to rpcv2 equivalents or reject them here as well.

[Copilot]

Pull request overview

Copilot reviewed 38 out of 55 changed files in this pull request and generated no new comments.

Suppressed comments (1)

cmd/stellar-rpc/internal/integrationtest/infrastructure/daemon_rpcv1.go:87

• Returning nil leaves rpcv1 outside the new early-exit/retry path. The exit hook calls t.Fatalf from a daemon goroutine; FailNow only terminates that calling goroutine, so the test's waitForRPC loop can still poll for the full 180 seconds and captive-core bind collisions are never retried. Please report rpcv1 exits through a channel here (and ensure the failed daemon is closed before restarting).

// The rpcv1 daemon reports a fatal exit through the logger's exit hook, which
// fails the test directly, so there is nothing to read here.
func (d *rpcv1Daemon) exited() <-chan error {
	return nil

[Copilot]

Pull request overview

Copilot reviewed 38 out of 55 changed files in this pull request and generated 1 comment.

[Copilot]

Pull request overview

Copilot reviewed 38 out of 55 changed files in this pull request and generated 1 comment.

[Copilot]

Pull request overview

Copilot reviewed 38 out of 55 changed files in this pull request and generated no new comments.

[Copilot]

Pull request overview

Copilot reviewed 42 out of 59 changed files in this pull request and generated no new comments.