Various fixes for long-running tests by willdavsmith · Pull Request #8680 · radius-project/radius

willdavsmith · 2025-02-26T20:03:23Z

Description

Updating Kubernetes version in AKS cluster creation step
Changed functionality to delete all namespaces except specified: [Long Running Tests] Remove Hardcoded List of Test Namespaces in Cleanup Step #7966
Updated comment for long-running-test inputs to make it more clear

Type of change

This pull request fixes a bug in Radius and has an approved issue (issue link required).
This pull request adds or changes features of Radius and has an approved issue (issue link required).
This pull request is a minor refactor, code cleanup, test improvement, or other maintenance task and doesn't change the functionality of Radius (issue link optional).

Fixes: #issue_number

Contributor checklist

Please verify that the PR meets the following requirements, where applicable:

An overview of proposed schema changes is included in a linked GitHub issue.
- Yes
- Not applicable
A design document PR is created in the design-notes repository, if new APIs are being introduced.
- Yes
- Not applicable
The design document has been reviewed and approved by Radius maintainers/approvers.
- Yes
- Not applicable
A PR for the samples repository is created, if existing samples are affected by the changes in this PR.
- Yes
- Not applicable
A PR for the documentation repository is created, if the changes in this PR affect the documentation or any user facing updates are made.
- Yes
- Not applicable
A PR for the recipes repository is created, if existing recipes are affected by the changes in this PR.
- Yes
- Not applicable

Signed-off-by: willdavsmith <willdavsmith@gmail.com>

codecov · 2025-02-26T20:12:03Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 55.94%. Comparing base (fdc5877) to head (9e30547).
Report is 1 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #8680      +/-   ##
==========================================
- Coverage   55.95%   55.94%   -0.01%     
==========================================
  Files         604      604              
  Lines       41466    41466              
==========================================
- Hits        23203    23199       -4     
- Misses      16492    16495       +3     
- Partials     1771     1772       +1

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

radius-functional-tests · 2025-02-26T21:06:55Z

Radius functional test overview

🔍 Go to test action run

Name	Value
Repository	willdavsmith/radius
Commit ref	`5d176de`
Unique ID	funca7508d9228
Image tag	pr-funca7508d9228

Click here to see the list of tools in the current test run

gotestsum 1.12.0
KinD: v0.20.0
Dapr:
Azure KeyVault CSI driver: 1.4.2
Azure Workload identity webhook: 1.3.0
Bicep recipe location ghcr.io/radius-project/dev/test/testrecipes/test-bicep-recipes/<name>:pr-funca7508d9228
Terraform recipe location http://tf-module-server.radius-test-tf-module-server.svc.cluster.local/<name>.zip (in cluster)
applications-rp test image location: ghcr.io/radius-project/dev/applications-rp:pr-funca7508d9228
dynamic-rp test image location: ghcr.io/radius-project/dev/dynamic-rp:pr-funca7508d9228
controller test image location: ghcr.io/radius-project/dev/controller:pr-funca7508d9228
ucp test image location: ghcr.io/radius-project/dev/ucpd:pr-funca7508d9228
deployment-engine test image location: ghcr.io/radius-project/deployment-engine:latest

Test Status

⌛ Building Radius and pushing container images for functional tests...
✅ Container images build succeeded
⌛ Publishing Bicep Recipes for functional tests...
✅ Recipe publishing succeeded
⌛ Starting corerp-cloud functional tests...
⌛ Starting ucp-cloud functional tests...
✅ corerp-cloud functional tests succeeded
✅ ucp-cloud functional tests succeeded

kachawla

Have we spent time investigating it before marking it skipped? If we are skipping the test, can you make sure that a task to unskip it is created and assigned as well?

Signed-off-by: willdavsmith <willdavsmith@gmail.com>

willdavsmith · 2025-03-01T00:18:51Z


 # Delete all test namespaces.
 echo "delete all test namespaces"
-namespaces=$(kubectl get namespace |


IMO this system is too complicated. Instead I think it's better to specify the namespaces we want to keep and delete everything else. Looks like there's already been some discussion about this: #7966

willdavsmith · 2025-03-01T00:19:41Z

    name: aksClusterName
    location: location
-    kubernetesVersion: '1.28.5'
+    kubernetesVersion: '1.31.5'


I was unable to run this bicep deployment locally because of the outdated kubernetes version. This should make sure that next time we create a long-running test cluster we won't hit this issue.

Is a better option to query for the latest version and use that? I'm concerned about this becoming stale and having to think about updating this manually

I created #8901 to track version numbers for tools and platforms.

+1 for @nicolejms' suggestion. @willdavsmith we don't know when this bicep will be used next time to recreate the LRT cluster again, so making this update might not be very useful since it’s likely that this version will be outdated by the time it’s needed. Any thoughts on how you can make it fetch the latest version instead of hardcoding?

brooke-hamilton

🤖

brooke-hamilton · 2025-03-20T15:50:44Z

Have we spent time investigating it before marking it skipped? If we are skipping the test, can you make sure that a task to unskip it is created and assigned as well?

@willdavsmith, @kachawla, the skipped test referenced in this comment was reverted. Is this PR ready to approve?

radius-functional-tests · 2025-03-20T17:01:30Z

Radius functional test overview

🔍 Go to test action run

Name	Value
Repository	willdavsmith/radius
Commit ref	`165bba7`
Unique ID	func0ee6fc436a
Image tag	pr-func0ee6fc436a

Click here to see the list of tools in the current test run

gotestsum 1.12.0
KinD: v0.20.0
Dapr:
Azure KeyVault CSI driver: 1.4.2
Azure Workload identity webhook: 1.3.0
Bicep recipe location ghcr.io/radius-project/dev/test/testrecipes/test-bicep-recipes/<name>:pr-func0ee6fc436a
Terraform recipe location http://tf-module-server.radius-test-tf-module-server.svc.cluster.local/<name>.zip (in cluster)
applications-rp test image location: ghcr.io/radius-project/dev/applications-rp:pr-func0ee6fc436a
dynamic-rp test image location: ghcr.io/radius-project/dev/dynamic-rp:pr-func0ee6fc436a
controller test image location: ghcr.io/radius-project/dev/controller:pr-func0ee6fc436a
ucp test image location: ghcr.io/radius-project/dev/ucpd:pr-func0ee6fc436a
deployment-engine test image location: ghcr.io/radius-project/deployment-engine:latest

Test Status

⌛ Building Radius and pushing container images for functional tests...
✅ Container images build succeeded
⌛ Publishing Bicep Recipes for functional tests...
✅ Recipe publishing succeeded
⌛ Starting corerp-cloud functional tests...
⌛ Starting ucp-cloud functional tests...
✅ corerp-cloud functional tests succeeded
✅ ucp-cloud functional tests succeeded

kachawla · 2025-03-20T17:06:00Z

Have we spent time investigating it before marking it skipped? If we are skipping the test, can you make sure that a task to unskip it is created and assigned as well?

@willdavsmith, @kachawla, the skipped test referenced in this comment was reverted. Is this PR ready to approve?

I wasn't aware that this PR was entirely repurposed for something else. @willdavsmith - please respond to PR comments for heads up on such changes.

@brooke-hamilton - I won't be able to get to this PR today, but if it looks good then feel free to go ahead. The changes I had requested were on an older version of the PR, so you can dismiss that.

The PR has changed since I reviewed it.

radius-functional-tests · 2025-05-13T20:49:50Z

Radius functional test overview

🔍 Go to test action run

Name	Value
Repository	willdavsmith/radius
Commit ref	`9e30547`
Unique ID	func13d4a13d62
Image tag	pr-func13d4a13d62

Click here to see the list of tools in the current test run

gotestsum 1.12.0
KinD: v0.20.0
Dapr:
Azure KeyVault CSI driver: 1.4.2
Azure Workload identity webhook: 1.3.0
Bicep recipe location ghcr.io/radius-project/dev/test/testrecipes/test-bicep-recipes/<name>:pr-func13d4a13d62
Terraform recipe location http://tf-module-server.radius-test-tf-module-server.svc.cluster.local/<name>.zip (in cluster)
applications-rp test image location: ghcr.io/radius-project/dev/applications-rp:pr-func13d4a13d62
dynamic-rp test image location: ghcr.io/radius-project/dev/dynamic-rp:pr-func13d4a13d62
controller test image location: ghcr.io/radius-project/dev/controller:pr-func13d4a13d62
ucp test image location: ghcr.io/radius-project/dev/ucpd:pr-func13d4a13d62
deployment-engine test image location: ghcr.io/radius-project/deployment-engine:latest

Test Status

⌛ Building Radius and pushing container images for functional tests...
✅ Container images build succeeded
⌛ Publishing Bicep Recipes for functional tests...
✅ Recipe publishing succeeded
⌛ Starting corerp-cloud functional tests...
⌛ Starting ucp-cloud functional tests...
✅ ucp-cloud functional tests succeeded
✅ corerp-cloud functional tests succeeded

Skipping failing postgres test

a5e1f18

Signed-off-by: willdavsmith <willdavsmith@gmail.com>

willdavsmith requested review from a team as code owners February 26, 2025 20:03

willdavsmith had a problem deploying to publish-bicep February 26, 2025 20:03 — with GitHub Actions Error

bicep-types

5d176de

Signed-off-by: willdavsmith <willdavsmith@gmail.com>

willdavsmith temporarily deployed to publish-bicep February 26, 2025 20:04 — with GitHub Actions Inactive

lakshmimsft previously approved these changes Feb 26, 2025

View reviewed changes

willdavsmith temporarily deployed to functional-tests February 26, 2025 21:05 — with GitHub Actions Inactive

sk593 previously approved these changes Feb 26, 2025

View reviewed changes

kachawla previously requested changes Feb 27, 2025

View reviewed changes

Various LRT fixes

befed25

Signed-off-by: willdavsmith <willdavsmith@gmail.com>

willdavsmith dismissed stale reviews from sk593 and lakshmimsft via befed25 February 28, 2025 23:39

willdavsmith temporarily deployed to publish-bicep February 28, 2025 23:39 — with GitHub Actions Inactive

Fix issue

066a530

Signed-off-by: willdavsmith <willdavsmith@gmail.com>

willdavsmith temporarily deployed to publish-bicep March 1, 2025 00:14 — with GitHub Actions Inactive

willdavsmith changed the title ~~Skipping failing test: Test_TerraformRecipe_KubernetesPostgres~~ Various fixes for long-running tests Mar 1, 2025

willdavsmith commented Mar 1, 2025

View reviewed changes

willdavsmith requested review from kachawla, lakshmimsft and sk593 March 1, 2025 00:20

ytimocin approved these changes Mar 3, 2025

View reviewed changes

brooke-hamilton approved these changes Mar 12, 2025

View reviewed changes

brooke-hamilton mentioned this pull request Mar 19, 2025

REQUEST: Approvership for brooke-hamilton radius-project/community#80

Closed

4 tasks done

Merge branch 'main' into fix-lrt-22625

165bba7

brooke-hamilton temporarily deployed to publish-bicep March 20, 2025 15:44 — with GitHub Actions Inactive

brooke-hamilton temporarily deployed to functional-tests March 20, 2025 16:50 — with GitHub Actions Inactive

Merge branch 'main' into fix-lrt-22625

7ac7286

willdavsmith temporarily deployed to publish-bicep March 24, 2025 17:11 — with GitHub Actions Inactive

willdavsmith had a problem deploying to functional-tests April 23, 2025 17:10 — with GitHub Actions Failure

Merge branch 'main' into fix-lrt-22625

9e30547

willdavsmith temporarily deployed to functional-tests May 13, 2025 20:48 — with GitHub Actions Inactive

willdavsmith temporarily deployed to publish-bicep May 13, 2025 20:48 — with GitHub Actions Inactive

willdavsmith merged commit 6b7aa81 into radius-project:main May 13, 2025

kachawla mentioned this pull request May 16, 2025

LRT: Add test namespaces to cleanup script #9437

Closed

6 of 12 tasks

Conversation

willdavsmith commented Feb 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Type of change

Contributor checklist

Uh oh!

codecov Bot commented Feb 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

radius-functional-tests Bot commented Feb 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Radius functional test overview

Test Status

Uh oh!

kachawla left a comment

Choose a reason for hiding this comment

Uh oh!

willdavsmith Mar 1, 2025

Choose a reason for hiding this comment

Uh oh!

willdavsmith Mar 1, 2025

Choose a reason for hiding this comment

Uh oh!

nicolejms Mar 13, 2025

Choose a reason for hiding this comment

Uh oh!

brooke-hamilton Mar 18, 2025

Choose a reason for hiding this comment

Uh oh!

kachawla Mar 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

brooke-hamilton left a comment

Choose a reason for hiding this comment

Uh oh!

brooke-hamilton commented Mar 20, 2025

Uh oh!

radius-functional-tests Bot commented Mar 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Radius functional test overview

Test Status

Uh oh!

kachawla commented Mar 20, 2025

Uh oh!

radius-functional-tests Bot commented May 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Radius functional test overview

Test Status

Uh oh!

This branch was previously deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

willdavsmith commented Feb 26, 2025 •

edited

Loading

codecov Bot commented Feb 26, 2025 •

edited

Loading

radius-functional-tests Bot commented Feb 26, 2025 •

edited

Loading

kachawla Mar 24, 2025 •

edited

Loading

radius-functional-tests Bot commented Mar 20, 2025 •

edited

Loading

radius-functional-tests Bot commented May 13, 2025 •

edited

Loading