Skip to content

Fix remaining Outerloop failures in DCP, CLI, and Dashboard tests - #20466

Merged
Ankit Jain (radical) merged 9 commits into
mainfrom
ankj/fix-remaining-outerloop-flakes
Sep 25, 2026
Merged

Ankit Jain (radical) merged 9 commits into
mainfrom
ankj/fix-remaining-outerloop-flakes

Conversation

@radical

@radical Ankit Jain (radical) commented Sep 25, 2026 •

Copy link
Copy Markdown
Member

Four repeat failures remained in the Outerloop workflow:

The SDK 'Aspire.AppHost.Sdk/14.0.0-ci' specified could not be found.
Watch task over Kubernetes ContainerExec resources terminated unexpectedly.
Polly.Timeout.TimeoutRejectedException: The operation didn't complete within the allowed timeout of '00:01:00'.
Expected rows not found: apiservice, webfrontend, cache
ScrollButtons_SmoothScrollHidesImmediately_AndReachesNewBottom
timed out after reaching the limit of 30000ms.

Root causes

LocalArchive test installs must preserve the package build identity. PR packages use a pr-N hive that matches the CLI's baked channel. Non-PR CI packages such as 14.0.0-ci use the local hive, but their CLI archive retains a baked daily identity. User configuration does not override the environment, install-sidecar, or assembly identity, so generated AppHosts selected daily feeds and could not resolve the CI SDK from the local hive.

DCP watch connections can stall while waiting for their initial HTTP response. Each connection attempt needs the API timeout so a stalled request is canceled and retried, while an established stream must remain active until its longer periodic restart. Factory or enumeration cancellation unrelated to that periodic restart must propagate instead of being silently retried.

The template tests matched .main-grid .resource-row. Fluent data-grid copies the configured row class onto optimized plain cells, so the locator returned both tr rows and td cells. A matched cell has no descendant grid cells, causing resource extraction to fail even though the dashboard displayed healthy resources.

The smooth-scroll test required observing an intermediate native animation frame. Windows headless Chromium can skip that frame under load even though the scroll request and scrollend behavior are correct.

Fix

LocalArchive installs now let the installer derive pr-N or local from the package version. Non-PR archives explicitly override ASPIRE_CLI_CHANNEL, ASPIRE_CLI_VERSION, and ASPIRE_CLI_PACKAGES so generated projects consume the local package directory. PR archives retain their automatically selected pr-N identity. Focused tests cover both package identities, both install command paths, and the exact local identity override.

DCP watches now apply the API timeout to each connection attempt and retry timeout failures until the periodic restart token is canceled. Once connected, the watch stream remains governed by the periodic restart token. PeriodicRestartAsyncEnumerable recreates factories canceled by that restart token, while cancellation from any other source propagates without retrying. Focused regressions require a second connection attempt after a deliberately stalled watch and cover both periodic-enumerable overloads.

Template resource checks now restrict the locator to .main-grid tr.resource-row and keep grid-cell lookups scoped to each matched row.

The Playwright test captures the requested smooth-scroll destination and dispatches scrollend explicitly after growing the content, preserving the product contract without depending on compositor timing.

Validation

  • CliInstallStrategyTests: 61 passed
  • Aspire.Deployment.EndToEnd.Tests build: succeeded
  • Aspire.Templates.Tests build: succeeded
  • KubernetesServiceTests and PeriodicRestartAsyncEnumerableTests: 14 passed
  • Stalled-watch retry regression: 10/10 repeated runs passed
  • Independent cancellation regressions: 20/20 repeated runs passed
  • DashboardInteractionsTests: 13 passed
  • Smooth-scroll regression: 10/10 repeated runs passed

Surprises

The intermittent terminal WebSocket failures are intentionally unchanged. They moved between operating systems and test cases across reruns, and no reproducible root cause was established.

@github-actions

This comment has been minimized.

@github-actions

Copy link
Copy Markdown
Contributor

🚀 Dogfood this PR with:

⚠️ WARNING: Do not do this without first carefully reviewing the code of this PR to satisfy yourself it is safe.

curl -fsSL https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.sh | bash -s -- 20466

Or

  • Run remotely in PowerShell:
iex "& { $(irm https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.ps1) } 20466"

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@karolz-ms

Karol Zadora-Przylecki (karolz-ms) commented Sep 25, 2026 •

Copy link
Copy Markdown
Contributor

KubernetesService and DCP watcher changes look good. CLI tests seem unhappy though.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

Copy link
Copy Markdown
Contributor

Retrying the failed CI jobs for this pull request from the CI run attempt. The rerun is being tracked in the rerun attempt.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

Copy link
Copy Markdown
Contributor

Retrying the failed CI jobs for this pull request from the CI run attempt. The rerun is being tracked in the rerun attempt.

@github-actions

This comment has been minimized.

@github-actions

Copy link
Copy Markdown
Contributor

Retrying the failed CI jobs for this pull request from the CI run attempt. The rerun is being tracked in the rerun attempt.

@github-actions

Copy link
Copy Markdown
Contributor

Retrying the failed CI jobs for this pull request from the CI run attempt. The rerun is being tracked in the rerun attempt.

@radical
Ankit Jain (radical) force-pushed the ankj/fix-remaining-outerloop-flakes branch from 69c271a to cf114f3 Compare September 25, 2026 18:12
@github-actions

This comment has been minimized.

LocalArchive E2E installs copied built packages into the local hive but
left the CLI on its baked channel, so aspire add could not resolve the
14.0.0-ci AppHost SDK.

DCP watches also inherited the one-minute non-streaming API timeout even
though the Kubernetes client buffers their long-lived response body.
The dashboard smooth-scroll test separately depended on observing an
intermediate compositor frame that Windows headless Chromium can skip.

Select the local hive after LocalArchive installs, keep watch requests
alive until their periodic restart, and drive the scrollend contract
deterministically in the Playwright test.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
LocalArchive installs can auto-detect a PR package suffix and place packages
in a pr-N hive, which conflicts with configuring the installed CLI to use
the local hive.

Long-lived watch factories can also be canceled by their periodic restart
token before returning an enumerable. That cancellation currently escapes
and terminates the outer watch instead of recreating it.

Force LocalArchive installs into the local hive and restart factories whose
restart interval expires. Cover both LocalArchive command paths and both
periodic-enumerable overloads.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The periodic factory restart handler currently treats every cancellation as
a restart while the outer token is active. A factory canceled for another
reason would be silently retried instead of propagating its failure.

Handle cancellation only when the periodic restart token fired, and verify
that unrelated cancellation propagates for both enumerable overloads.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
PR archive tests install pre-downloaded packages through LocalArchive.
Forcing those packages into the local hive breaks the CLI's baked pr-N
channel, so generated AppHosts cannot resolve the PR AppHost SDK.

Allow the installer to infer the hive from the package version and only
configure the global local channel for non-PR archives. Cover both PR
and CI package identities with focused tests.

Periodic restart handling also retried independent cancellation in the
struct factory and class enumerator paths. Require the periodic token to
be canceled before restarting, and verify unrelated cancellation is
propagated without retrying the factory.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
This was referenced Oct 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants