Repository navigation
Retry AWS SSI jobs when the installer cannot update apt sources - #8245
Merged
robertomonteromiguel merged 1 commit intoOct 30, 2026
Merged
Conversation
A transient Hash Sum mismatch on apt.datadoghq.com fails the install script with this message and exits 1, so GitLab never retries the job. Matching it as a known SSI installer exception exits 3 and reuses the existing job retry. Co-authored-by: Cursor <cursoragent@cursor.com>
Contributor
|
|
robertomonteromiguel
marked this pull request as ready for review
October 30, 2026 07:07
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 08fe2be00c
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
cbeauchesne
approved these changes
Oct 30, 2026
robertomonteromiguel
deleted the
ssi-installer-apt-sources-known-exception
branch
October 30, 2026 09:03
p-datadog
pushed a commit
that referenced
this pull request
Nov 1, 2026
* u/main: (82 commits) Auto-activate nodejs easy wins for asm-libraries (#8265) Auto-activate nodejs easy wins for apm-sdk-capabilities (#8264) Auto-activate rust easy wins for apm-sdk-capabilities (#8217) Auto-activate golang easy wins for apm-sdk-capabilities (#8263) Auto-activate dotnet easy wins for apm-sdk-capabilities (#8262) test(ffe): cover Go and Java customer exposure hooks (#8094) Move the debugger scenarios to the mocked backend, with remote config sent through it (#8253) test(otel): observe the SDK's default OTLP protocol instead of hardcoding it (#8256) Revert "[debugger] Test that debugger metrics use the debugger telemetry namespace" (#8257) Check the mocked backend network aliases on GitLab (#8212) Agent on the mocked backend: set DD_SITE instead of each intake URL (#8254) [debugger] Test that debugger metrics use the debugger telemetry namespace (#8052) Fail SSI installs when component installation fails (#8246) Collect HotSpot hs_err logs from Java AWS SSI weblogs (#8249) FFE direct shutdown: detect the priming capture without the watchdog latency (#8250) Move DEBUGGER_TELEMETRY to the mocked backend, which serves remote config (#8239) PHP parametric base image (#8172) AI Guard: tests for backend-provided extra_tags on the ai_guard span (#8130) Cbeauchesne/rename dead and legacy features (#8240) Retry AWS SSI jobs when the installer cannot update apt sources (#8245) ...
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
AWS SSI jobs fail for good when the Datadog apt repository is briefly inconsistent. The install script prints
Failed to update the sources after adding the Datadog repository.(typically after aHash Sum mismatchonapt.datadoghq.com) and the provision step exits 1. GitLab only retries exit code 3, so the job is not retried.Seen on
HOST_AUTO_INJECTION_INSTALL_SCRIPT_PROFILING/Ubuntu_24_10_amd64(GitLab job 2191149577): the installer died duringapt-get update, before any package was installed.Changes
ssi_installer_apt_sources_update_failedtoutils/virtual_machine/aws_infra_exceptions.json.ssi_installer_prefix marks exceptions that come from the SSI installer, so later installer flakes can be added next to it._handle_provision_errorexit 3..base_job_onboarding_system_testsalready retries that code up to 2 times, which recreates the VM.Workflow
system-tests-reviewers: ask to one of youre co-worker familiar with the tested feature.system-tests-core(slack)🚀 Once your PR is reviewed and the CI green, you can merge it!
🛟 #apm-shared-testing 🛟
Test plan
Failed to update the sources after adding the Datadog repository.is present in the PulumiCommandErrorof a failed install-script job (it is, in theUbuntu_24_10_amd64.logof job 2191149577).Known infrastructure exception (ssi_installer_apt_sources_update_failed)and GitLab retries the job (exit code 3).Made with Cursor