Making necessary changes in the running-functional-tests document. Also doing some cleanup. by ytimocin · Pull Request #8087 · radius-project/radius

ytimocin · 2024-11-25T06:44:55Z

Description

Making necessary changes in the running-functional-tests document. Also doing some cleanup:

Removed unnecessary environment variables from the workflows.

Type of change

This pull request is a minor refactor, code cleanup, test improvement, or other maintenance task and doesn't change the functionality of Radius (issue link optional).
Fixes: #issue_number

Contributor checklist

Please verify that the PR meets the following requirements, where applicable:

An overview of proposed schema changes is included in a linked GitHub issue.
A design document PR is created in the design-notes repository, if new APIs are being introduced.
If applicable, design document has been reviewed and approved by Radius maintainers/approvers.
A PR for the samples repository is created, if existing samples are affected by the changes in this PR.
A PR for the documentation repository is created, if the changes in this PR affect the documentation or any user facing updates are made.
A PR for the recipes repository is created, if existing recipes are affected by the changes in this PR.

github-actions · 2024-11-25T06:59:41Z

Unit Tests

3 610 tests ±0 3 608 ✅ ±0 6m 16s ⏱️ +12s
295 suites ±0 2 💤 ±0
1 files ±0 0 ❌ ±0

Results for commit da528bd. ± Comparison against base commit 7709a79.

♻️ This comment has been updated with latest results.

kachawla

Had left some comments on previous version of this PR here: #8069

ytimocin · 2024-11-25T18:32:57Z

What's the motivation for changing file name style?

I think the naming convention for sh files is underscores, dashes, or all words together. We use camel case for now. We use both underscores and dashes.

Ref: https://google.github.io/styleguide/shellguide.html#source-filenames.

I was creating new sh files for cloud and non-cloud, so I wanted to give them new names that would also be consistent with other names in our codebase.

That being said, we need to decide if we want to use underscores or dashes because we have both. We only had one camel-case and I changed that.

radius-functional-tests · 2024-11-26T18:24:39Z

Radius functional test overview

🔍 Go to test action run

Name	Value
Repository	radius-project/radius
Commit ref	`f8a1e10`
Unique ID	funcefc4ddb0b6
Image tag	pr-funcefc4ddb0b6

Click here to see the list of tools in the current test run

gotestsum 1.12.0
KinD: v0.20.0
Dapr:
Azure KeyVault CSI driver: 1.4.2
Azure Workload identity webhook: 1.3.0
Bicep recipe location ghcr.io/radius-project/dev/test/testrecipes/test-bicep-recipes/<name>:pr-funcefc4ddb0b6
Terraform recipe location http://tf-module-server.radius-test-tf-module-server.svc.cluster.local/<name>.zip (in cluster)
applications-rp test image location: ghcr.io/radius-project/dev/applications-rp:pr-funcefc4ddb0b6
dynamic-rp test image location: ghcr.io/radius-project/dev/dynamic-rp:pr-funcefc4ddb0b6
controller test image location: ghcr.io/radius-project/dev/controller:pr-funcefc4ddb0b6
ucp test image location: ghcr.io/radius-project/dev/ucpd:pr-funcefc4ddb0b6
deployment-engine test image location: ghcr.io/radius-project/deployment-engine:latest

Test Status

⌛ Building Radius and pushing container images for functional tests...
✅ Container images build succeeded
⌛ Publishing Bicep Recipes for functional tests...
✅ Recipe publishing succeeded
⌛ Starting corerp-cloud functional tests...
⌛ Starting ucp-cloud functional tests...
✅ corerp-cloud functional tests succeeded
✅ ucp-cloud functional tests succeeded

ytimocin · 2024-11-26T18:25:55Z

-          # AZURE_MSSQL_RESOURCE_ID
-          # AZURE_MSSQL_USERNAME
-          # AZURE_MSSQL_PASSWORD


I see these are not being used anymore.

ytimocin · 2024-11-26T18:26:27Z

          AWS_ACCESS_KEY_ID: ${{ secrets.FUNCTEST_AWS_ACCESS_KEY_ID }}
          AWS_SECRET_ACCESS_KEY: ${{ secrets.FUNCTEST_AWS_SECRET_ACCESS_KEY }}
          AWS_REGION: ${{ env.AWS_REGION }}
-          RADIUS_SAMPLES_REPO_ROOT: ${{ github.workspace }}/samples


This is only being used in the noncloud tests.

ytimocin · 2024-11-26T18:26:50Z

-          TEMP_CERT_DIR: ${{ steps.create-local-registry.outputs.temp-cert-dir }}
-          SSL_CERT_FILE: ${{ steps.create-local-registry.outputs.temp-cert-dir }}/certs/${{ env.LOCAL_REGISTRY_SERVER }}/client.crt


Not being used.

ytimocin · 2024-11-26T18:28:52Z

radius-functional-tests · 2024-12-03T14:00:14Z

Radius functional test overview

🔍 Go to test action run

Name	Value
Repository	radius-project/radius
Commit ref	`223c606`
Unique ID	funcec1695f50d
Image tag	pr-funcec1695f50d

Click here to see the list of tools in the current test run

gotestsum 1.12.0
KinD: v0.20.0
Dapr:
Azure KeyVault CSI driver: 1.4.2
Azure Workload identity webhook: 1.3.0
Bicep recipe location ghcr.io/radius-project/dev/test/testrecipes/test-bicep-recipes/<name>:pr-funcec1695f50d
Terraform recipe location http://tf-module-server.radius-test-tf-module-server.svc.cluster.local/<name>.zip (in cluster)
applications-rp test image location: ghcr.io/radius-project/dev/applications-rp:pr-funcec1695f50d
dynamic-rp test image location: ghcr.io/radius-project/dev/dynamic-rp:pr-funcec1695f50d
controller test image location: ghcr.io/radius-project/dev/controller:pr-funcec1695f50d
ucp test image location: ghcr.io/radius-project/dev/ucpd:pr-funcec1695f50d
deployment-engine test image location: ghcr.io/radius-project/deployment-engine:latest

Test Status

⌛ Building Radius and pushing container images for functional tests...
✅ Container images build succeeded
⌛ Publishing Bicep Recipes for functional tests...
✅ Recipe publishing succeeded
⌛ Starting ucp-cloud functional tests...
⌛ Starting corerp-cloud functional tests...
✅ ucp-cloud functional tests succeeded
✅ corerp-cloud functional tests succeeded

brooke-hamilton

I was able to get all five services running with a debugger following these instructions. 🎆

A helpful note in troubleshooting would be to say that after launching the debugger, five items should be visible in the VS Code call stack panel (in the debugging tab):

Launch Deployment Engine
Launch Dynamic RP
Launch Applications RP
Launch Controller
Launch UCP

Initially, I had a failure because I was deploying a kind cluster using the instructions in our docs site to map ports, but those ports were conflicting with the ports the debugger was attaching to, causing 503 errors. This became apparent when I realized that the Launch Applications RP debugging configuration was failing and would disappear from the list of processes.

brooke-hamilton · 2024-12-12T14:44:04Z

 5. Install C# VS Code extension - <https://marketplace.visualstudio.com/items?itemName=ms-dotnettools.csharp>.
 6. (Optional) Configure any cloud provider credentials you want to use for developing Radius.
-  
+


We should explain in step 1 that the Kubernetes cluster should be created with defaults, i.e., without the port mappings required in the docs because the debugger will attempt to attach to the same ports, and they will already be used by the cluster, resulting in 503 errors when running rad commands.

Can you explain more? This works fine for me with the current settings.

Our docs say to use the configuration below when creating a Kind cluster, but this configuration causes port conflicts when launching the debugger.

kind: Cluster apiVersion: kind.x-k8s.io/v1alpha4 nodes: - role: control-plane - role: worker extraPortMappings: - containerPort: 80 hostPort: 8080 listenAddress: "0.0.0.0" - containerPort: 443 hostPort: 8443 listenAddress: "0.0.0.0"

brooke-hamilton · 2024-12-12T14:49:35Z

+This example adds a `dev` workspace:

 ```yaml
 workspaces:


I was able to get the same result by adding the overrides: section to the existing default workspace, instead of adding a new workspace. It works either way, but do you think that modifying the existing workspace is a simpler way to understand it? In other words, override the ucp property of the current connection instead of creating a new workspace, like this. (I intend this comment as a discussion rather than a strong opinion on it.)

workspaces: default: default items: default: connection: context: kind-kind kind: kubernetes overrides: ucp: http://localhost:9000 environment: /planes/radius/local/resourceGroups/default/providers/Applications.Core/environments/default scope: /planes/radius/local/resourceGroups/default

We can do both ways. This doc was written before, so I am not going to touch that part of the information. We can update later if necessary.

brooke-hamilton · 2024-12-12T16:44:20Z

-### Run
+### Run Non-Cloud Functional Tests
+
+1. Required environment variables:


Where are the values for these found so that they can be set?

I added descriptions for these environment variables.

brooke-hamilton · 2025-01-14T14:33:02Z

+   2. **DOCKER_REGISTRY**: This is the container registry that you would be using for storing the test related images.
+   3. **BICEP_RECIPE_REGISTRY**: This is the container registry that you would be using for storing the Bicep recipes.


Are these URLs or registry names?

kachawla · 2025-01-14T17:28:52Z

+# and then switch to the environment using `rad env switch kind-radius`.
+
+rad group create default
+rad env create kind-radius


The kind-radius environment is created in kind-radius group here https://github.com/radius-project/radius/blob/main/.github/workflows/functional-test-noncloud.yaml#L293-L299. Any reason to create it in the default group locally?

kachawla · 2025-01-14T17:29:52Z

+# Make sure that you have default environment in the default group.
+# The way to check that is to run the following commands:
+# `rad group switch default`
+# `rad env list` and check if there is an environment named `default`.


If this is expected to be already existing, then the manual group creation on L30 shouldn't be needed

kachawla · 2025-01-14T17:38:15Z

+### Run Non-Cloud Functional Tests
+
+1. Required environment variables:
+   1. **TF_RECIPE_MODULE_SERVER_URL**: This is the URL for the Terraform Recipe Module Server. If you have run `make publish-test-terraform-recipes` you will see this URL at the end of that command.


Running make publish-test-terraform-recipes is a prerequisite so let's make that explicit. Feel free to modify the suggested phrasing. Also, let's call it out in the prerequisite above to copy the output for later use.

Suggested change

1. **TF_RECIPE_MODULE_SERVER_URL**: This is the URL for the Terraform Recipe Module Server. If you have run `make publish-test-terraform-recipes` you will see this URL at the end of that command.

1. **TF_RECIPE_MODULE_SERVER_URL**: This is the URL for the Terraform Recipe Module Server. You can find this URL in the output of `make publish-test-terraform-recipes` from step 7 of prerequisite above.

kachawla · 2025-01-14T17:40:51Z

+
+1. Required environment variables:
+   1. **TF_RECIPE_MODULE_SERVER_URL**: This is the URL for the Terraform Recipe Module Server. If you have run `make publish-test-terraform-recipes` you will see this URL at the end of that command.
+   2. **DOCKER_REGISTRY**: This is the container registry that you would be using for storing the test related images.


Is this required or do we have a default that we can point the users to?

kachawla · 2025-01-14T17:41:31Z

+1. Required environment variables:
+   1. **TF_RECIPE_MODULE_SERVER_URL**: This is the URL for the Terraform Recipe Module Server. If you have run `make publish-test-terraform-recipes` you will see this URL at the end of that command.
+   2. **DOCKER_REGISTRY**: This is the container registry that you would be using for storing the test related images.
+   3. **BICEP_RECIPE_REGISTRY**: This is the container registry that you would be using for storing the Bicep recipes.


Same here, is this required or optional? Do we have a default in either case?

kachawla · 2025-01-14T17:45:30Z

+   1. **TF_RECIPE_MODULE_SERVER_URL**: This is the URL for the Terraform Recipe Module Server. If you have run `make publish-test-terraform-recipes` you will see this URL at the end of that command.
+   2. **DOCKER_REGISTRY**: This is the container registry that you would be using for storing the test related images.
+   3. **BICEP_RECIPE_REGISTRY**: This is the container registry that you would be using for storing the Bicep recipes.
+   4. **RADIUS_SAMPLES_REPO_ROOT**: This should point to the root directory of the Samples repository in your local, if you want to run the functional tests for the samples.


if you want to run the functional tests for the samples

What happens if this is not set?

kachawla · 2025-01-14T17:49:40Z

-### Run
+### Run Non-Cloud Functional Tests
+
+1. Required environment variables:


Are all of these env variables required or some are optional?

kachawla · 2025-01-14T17:50:21Z

+   2. **DOCKER_REGISTRY**: This is the container registry that you would be using for storing the test related images.
+   3. **BICEP_RECIPE_REGISTRY**: This is the container registry that you would be using for storing the Bicep recipes.
+   4. **RADIUS_SAMPLES_REPO_ROOT**: This should point to the root directory of the Samples repository in your local, if you want to run the functional tests for the samples.
+2. Run:


Let's expand on this to give user some context on what's going to happen when they run this script.

kachawla · 2025-01-14T17:50:48Z

+       .{workspace}/radius/test/execute-functional-tests-noncloud.sh
+   ```
+
+When you're running locally with this configuration, the script is going to create a new Radius group and environment that the functional tests need to run. The same script is also going to make sure that the necessary environment variables are set. If everything is set, the script will run the commands:


nit: indentation

kachawla · 2025-01-14T17:52:02Z

+   5. **AWS_ACCESS_KEY_ID**
+   6. **AWS_SECRET_ACCESS_KEY**
+   7. **AWS_REGION**
+2. You also need to create AWS and Azure Credentials. Please refer to: <https://docs.radapp.io/reference/cli/rad_credential_register/>.


This should come before step 1 because we are asking the user to set values for these in the previous step.

kachawla · 2025-01-14T17:52:28Z

+   6. **AWS_SECRET_ACCESS_KEY**
+   7. **AWS_REGION**
+2. You also need to create AWS and Azure Credentials. Please refer to: <https://docs.radapp.io/reference/cli/rad_credential_register/>.
+3. Run:


Same as above regarding providing some context.

kachawla · 2025-01-14T17:54:16Z

 ```sh
-   make test-functional-corerp
-   make test-functional-msgrp
-   make test-functional-daprrp
-   make test-functional-datastoresrp
+    make test-functional-corerp-cloud
+    make test-functional-ucp-cloud
 ```

 You can also run/debug individual tests from VSCode.


You can also run/debug individual tests from VSCode.

This seems a bit abrupt and confusing at the end after user has already run the tests through command line. Should this option be provided before they run the execute script or earlier?

kachawla · 2025-01-14T17:56:32Z

+# `make test-functional-all-noncloud`
+
+usage() {
+  echo -e "$0 requires <resourcegroup_name>\n"


Where is resourcegroup_name defined?

kachawla · 2025-01-14T17:58:10Z

+# You can run them one by one or all at once.
+# The way to run them all at once is to run the following command:
+# `make test-functional-all-noncloud`


What should a user skip/update to follow this? Also should be cloud instead of noncloud.

kachawla · 2025-01-14T18:21:09Z

+# You can run them one by one or all at once.
+# The way to run them all at once is to run the following command:
+# `make test-functional-all-noncloud`


kachawla · 2025-01-14T18:22:09Z

+# Make sure that you are in the root directory of the repository.
+cd ../


Is this needed in the cloud tests script as well?

kachawla · 2025-01-14T18:25:33Z

+make test-functional-datastoresrp-noncloud
+make test-functional-kubernetes-noncloud
+# make test-functional-msgrp-noncloud
+make test-functional-samples-noncloud


Do we need users to run samples in local testing?

kachawla · 2025-01-14T18:25:54Z

+echo "Running Radius non-cloud functional tests."
+
+make test-functional-cli-noncloud
+# make test-functional-corerp-noncloud


Shouldn't be commented out?

…so doing some cleanup. Signed-off-by: ytimocin <ytimocin@microsoft.com>

kachawla

@ytimocin Is this ready for re-review since last set of comments?

github-actions · 2025-05-26T18:08:54Z

This pull request has been automatically marked as stale because it has been inactive for 90 days. Remove stale label or comment or this PR will be closed in 7 days.

ytimocin had a problem deploying to functional-tests November 25, 2024 06:45 — with GitHub Actions Failure

ytimocin temporarily deployed to publish-bicep November 25, 2024 06:45 — with GitHub Actions Inactive

ytimocin requested review from brooke-hamilton and vishwahiremat November 25, 2024 06:45

ytimocin mentioned this pull request Nov 25, 2024

Updating the Running Cloud Functional Tests and adding the Running Non-cloud Functional Tests docs #8069

Closed

6 tasks

ytimocin force-pushed the ytimocin/test-doc-update branch from 67cae74 to 7aeb2b6 Compare November 25, 2024 06:47

ytimocin had a problem deploying to functional-tests November 25, 2024 06:47 — with GitHub Actions Failure

ytimocin temporarily deployed to publish-bicep November 25, 2024 06:48 — with GitHub Actions Inactive

kachawla reviewed Nov 25, 2024

View reviewed changes

ytimocin force-pushed the ytimocin/test-doc-update branch from 7aeb2b6 to f8a1e10 Compare November 26, 2024 18:23

ytimocin marked this pull request as ready for review November 26, 2024 18:23

ytimocin requested review from a team as code owners November 26, 2024 18:23

ytimocin temporarily deployed to functional-tests November 26, 2024 18:23 — with GitHub Actions Inactive

ytimocin temporarily deployed to publish-bicep November 26, 2024 18:23 — with GitHub Actions Inactive

ytimocin commented Nov 26, 2024

View reviewed changes

ytimocin force-pushed the ytimocin/test-doc-update branch from f8a1e10 to 223c606 Compare December 3, 2024 01:45

ytimocin temporarily deployed to functional-tests December 3, 2024 01:45 — with GitHub Actions Inactive

ytimocin temporarily deployed to publish-bicep December 3, 2024 01:45 — with GitHub Actions Inactive

brooke-hamilton requested changes Dec 12, 2024

View reviewed changes

brooke-hamilton reviewed Dec 12, 2024

View reviewed changes

ytimocin force-pushed the ytimocin/test-doc-update branch from 223c606 to 10ec437 Compare January 2, 2025 17:03

ytimocin had a problem deploying to functional-tests January 2, 2025 17:03 — with GitHub Actions Failure

ytimocin had a problem deploying to publish-bicep January 2, 2025 17:03 — with GitHub Actions Error

ytimocin force-pushed the ytimocin/test-doc-update branch from 10ec437 to 8a17468 Compare January 2, 2025 17:04

ytimocin had a problem deploying to functional-tests January 2, 2025 17:04 — with GitHub Actions Failure

ytimocin force-pushed the ytimocin/test-doc-update branch from a617d63 to 24a4e7b Compare January 2, 2025 23:34

ytimocin had a problem deploying to functional-tests January 2, 2025 23:34 — with GitHub Actions Failure

ytimocin temporarily deployed to publish-bicep January 2, 2025 23:34 — with GitHub Actions Inactive

ytimocin force-pushed the ytimocin/test-doc-update branch from 24a4e7b to f72fa70 Compare January 13, 2025 16:08

ytimocin had a problem deploying to functional-tests January 13, 2025 16:08 — with GitHub Actions Failure

ytimocin temporarily deployed to publish-bicep January 13, 2025 16:08 — with GitHub Actions Inactive

ytimocin force-pushed the ytimocin/test-doc-update branch from f72fa70 to 1ee4a57 Compare January 13, 2025 16:25

ytimocin had a problem deploying to functional-tests January 13, 2025 16:25 — with GitHub Actions Failure

ytimocin temporarily deployed to publish-bicep January 13, 2025 16:25 — with GitHub Actions Inactive

ytimocin requested review from brooke-hamilton and kachawla January 13, 2025 16:25

brooke-hamilton reviewed Jan 14, 2025

View reviewed changes

kachawla reviewed Jan 14, 2025

View reviewed changes

Making necessary changes in the running-functional-tests document. Al…

da528bd

…so doing some cleanup. Signed-off-by: ytimocin <ytimocin@microsoft.com>

ytimocin force-pushed the ytimocin/test-doc-update branch from 1ee4a57 to da528bd Compare February 18, 2025 15:44

ytimocin had a problem deploying to functional-tests February 18, 2025 15:44 — with GitHub Actions Failure

ytimocin temporarily deployed to publish-bicep February 18, 2025 15:44 — with GitHub Actions Inactive

kachawla reviewed Feb 24, 2025

View reviewed changes

brooke-hamilton mentioned this pull request Mar 19, 2025

REQUEST: Approvership for brooke-hamilton radius-project/community#80

Closed

4 tasks done

github-actions Bot added the stale label May 26, 2025

github-actions Bot closed this Jun 2, 2025

		TEMP_CERT_DIR: ${{ steps.create-local-registry.outputs.temp-cert-dir }}
		SSL_CERT_FILE: ${{ steps.create-local-registry.outputs.temp-cert-dir }}/certs/${{ env.LOCAL_REGISTRY_SERVER }}/client.crt

		5. Install C# VS Code extension - <https://marketplace.visualstudio.com/items?itemName=ms-dotnettools.csharp>.
		6. (Optional) Configure any cloud provider credentials you want to use for developing Radius.

		2. DOCKER_REGISTRY: This is the container registry that you would be using for storing the test related images.
		3. BICEP_RECIPE_REGISTRY: This is the container registry that you would be using for storing the Bicep recipes.

	1. TF_RECIPE_MODULE_SERVER_URL: This is the URL for the Terraform Recipe Module Server. If you have run `make publish-test-terraform-recipes` you will see this URL at the end of that command.
	1. TF_RECIPE_MODULE_SERVER_URL: This is the URL for the Terraform Recipe Module Server. You can find this URL in the output of `make publish-test-terraform-recipes` from step 7 of prerequisite above.

		# Make sure that you are in the root directory of the repository.
		cd ../

Conversation

ytimocin commented Nov 25, 2024

Description

Type of change

Contributor checklist

Uh oh!

github-actions Bot commented Nov 25, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Unit Tests

Uh oh!

kachawla left a comment

Choose a reason for hiding this comment

Uh oh!

ytimocin commented Nov 25, 2024

Uh oh!

radius-functional-tests Bot commented Nov 26, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Radius functional test overview

Test Status

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

radius-functional-tests Bot commented Dec 3, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Radius functional test overview

Test Status

Uh oh!

brooke-hamilton left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

github-actions Bot commented Nov 25, 2024 •

edited

Loading

radius-functional-tests Bot commented Nov 26, 2024 •

edited

Loading

radius-functional-tests Bot commented Dec 3, 2024 •

edited

Loading