test: run the cloud-profiler system tests in google cloud build#8698
test: run the cloud-profiler system tests in google cloud build#8698danieljbruce wants to merge 8 commits into
Conversation
There was a problem hiding this comment.
Code Review
This pull request introduces helper scripts and templates under bin/move-to-gcb/ to automate the setup of Google Cloud Build (GCB) triggers for running system tests on client libraries, including a generated configuration for cloud-profiler. The review feedback suggests enhancing the configurability of these scripts by allowing the wrapper script generate_yaml_and_create_trigger.sh to accept and forward a custom GCP project ID (-p) to create_trigger.sh. Additionally, the reviewer recommends explicitly setting the _GCP_PROJECT_ID substitution parameter during trigger creation in create_trigger.sh to prevent falling back to the default project ID.
| usage() { | ||
| echo "Usage: $0 -l <library_name>" | ||
| echo " -l The name of the library (e.g., bigtable, spanner) [Required]" | ||
| exit 1 | ||
| } | ||
|
|
||
| while getopts "l:h" opt; do | ||
| case "${opt}" in | ||
| l) LIBRARY_NAME="${OPTARG}" ;; | ||
| h) usage ;; | ||
| *) usage ;; | ||
| esac | ||
| done | ||
|
|
||
| if [[ -z "${LIBRARY_NAME}" ]]; then | ||
| echo "Error: Library name (-l) is required." | ||
| usage | ||
| fi |
There was a problem hiding this comment.
The wrapper script generate_yaml_and_create_trigger.sh does not accept the -p (project ID) option, even though the underlying create_trigger.sh script supports it. This prevents users from specifying a custom GCP project ID when running the wrapper script.
| usage() { | |
| echo "Usage: $0 -l <library_name>" | |
| echo " -l The name of the library (e.g., bigtable, spanner) [Required]" | |
| exit 1 | |
| } | |
| while getopts "l:h" opt; do | |
| case "${opt}" in | |
| l) LIBRARY_NAME="${OPTARG}" ;; | |
| h) usage ;; | |
| *) usage ;; | |
| esac | |
| done | |
| if [[ -z "${LIBRARY_NAME}" ]]; then | |
| echo "Error: Library name (-l) is required." | |
| usage | |
| fi | |
| usage() { | |
| echo "Usage: $0 -l <library_name> [-p <project_id>]" | |
| echo " -l The name of the library (e.g., bigtable, spanner) [Required]" | |
| echo " -p The Google Cloud project ID" | |
| exit 1 | |
| } | |
| while getopts "l:p:h" opt; do | |
| case "${opt}" in | |
| l) LIBRARY_NAME="${OPTARG}" ;; | |
| p) PROJECT_ID="${OPTARG}" ;; | |
| h) usage ;; | |
| *) usage ;; | |
| esac | |
| done | |
| if [[ -z "${LIBRARY_NAME}" ]]; then | |
| echo "Error: Library name (-l) is required." | |
| usage | |
| fi |
There was a problem hiding this comment.
This was accidentally pushed and is no longer in the list of changes.
| # Create the build trigger | ||
| "${SCRIPT_DIR}/create_trigger.sh" -l "${LIBRARY_NAME}" |
There was a problem hiding this comment.
Forward the parsed PROJECT_ID to create_trigger.sh if it is provided.
| # Create the build trigger | |
| "${SCRIPT_DIR}/create_trigger.sh" -l "${LIBRARY_NAME}" | |
| # Create the build trigger | |
| if [[ -n "${PROJECT_ID}" ]]; then | |
| "${SCRIPT_DIR}/create_trigger.sh" -l "${LIBRARY_NAME}" -p "${PROJECT_ID}" | |
| else | |
| "${SCRIPT_DIR}/create_trigger.sh" -l "${LIBRARY_NAME}" | |
| fi |
There was a problem hiding this comment.
This was accidentally pushed and is no longer in the list of changes.
| # Execute the gcloud command with variable substitutions | ||
| gcloud builds triggers create github \ | ||
| --project="${PROJECT_ID}" \ | ||
| --name="${LIBRARY_NAME}-system-tests" \ | ||
| --region="global" \ | ||
| --description="CI build trigger for ${LIBRARY_NAME} system tests" \ | ||
| --repo-owner="googleapis" \ | ||
| --repo-name="google-cloud-node" \ | ||
| --pull-request-pattern="^main$" \ | ||
| --comment-control="COMMENTS_ENABLED_FOR_EXTERNAL_CONTRIBUTORS_ONLY" \ | ||
| --included-files="handwritten/${LIBRARY_NAME}/**" \ | ||
| --build-config="handwritten/${LIBRARY_NAME}/cloudbuild.yaml" |
There was a problem hiding this comment.
When creating the Cloud Build trigger, the _GCP_PROJECT_ID substitution should be explicitly set to the target PROJECT_ID. Otherwise, the trigger will fall back to the default value (long-door-651) defined in the cloudbuild.yaml file, even if the trigger is created in a different project.
| # Execute the gcloud command with variable substitutions | |
| gcloud builds triggers create github \ | |
| --project="${PROJECT_ID}" \ | |
| --name="${LIBRARY_NAME}-system-tests" \ | |
| --region="global" \ | |
| --description="CI build trigger for ${LIBRARY_NAME} system tests" \ | |
| --repo-owner="googleapis" \ | |
| --repo-name="google-cloud-node" \ | |
| --pull-request-pattern="^main$" \ | |
| --comment-control="COMMENTS_ENABLED_FOR_EXTERNAL_CONTRIBUTORS_ONLY" \ | |
| --included-files="handwritten/${LIBRARY_NAME}/**" \ | |
| --build-config="handwritten/${LIBRARY_NAME}/cloudbuild.yaml" | |
| # Execute the gcloud command with variable substitutions | |
| gcloud builds triggers create github \ | |
| --project="${PROJECT_ID}" \ | |
| --name="${LIBRARY_NAME}-system-tests" \ | |
| --region="global" \ | |
| --description="CI build trigger for ${LIBRARY_NAME} system tests" \ | |
| --repo-owner="googleapis" \ | |
| --repo-name="google-cloud-node" \ | |
| --pull-request-pattern="^main$" \ | |
| --comment-control="COMMENTS_ENABLED_FOR_EXTERNAL_CONTRIBUTORS_ONLY" \ | |
| --included-files="handwritten/${LIBRARY_NAME}/**" \ | |
| --build-config="handwritten/${LIBRARY_NAME}/cloudbuild.yaml" \ | |
| --substitutions=_GCP_PROJECT_ID="${PROJECT_ID}" |
There was a problem hiding this comment.
This was accidentally pushed and is no longer in the list of changes.
Description
This pull request adds a yaml file to instruct the Cloud Profiler system test CI check to work with the new Cloud Build trigger thereby making the new CI check effectively run our system tests. It turns out that no extra code modifications are required to allow the test to run in the new environment.
This pull request is very similar to #8697.
Impact
Leverages the strengths of running system tests in GCB rather than relying on kokoro for system tests.
Testing
This pull request tells the tests how to work with the new check to the continuous integration pipeline for the Cloud Profiler system tests thus improves the effectiveness of that test.
Next Steps