Skip to content

GCSHook: Log NotFound error instead of raise on blob delete - #2

Draft
tuzonghua wants to merge 1175 commits into
mainfrom
google/gcsdeleteobjectsoperator_allow_skip_error
Draft

tuzonghua wants to merge 1175 commits into
mainfrom
google/gcsdeleteobjectsoperator_allow_skip_error

Conversation

@tuzonghua

Copy link
Copy Markdown
Owner

Instead of raising a 404/NotFound error when calling GCSHook.delete(), match the behavior of delete_bucket and log the error.


Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)

  • Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
  • For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
  • When adding dependency, check compliance with the ASF 3rd Party License Policy.
  • For significant user-facing changes create newsfragment: {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

@tuzonghua
tuzonghua force-pushed the google/gcsdeleteobjectsoperator_allow_skip_error branch from e4f649e to b610dc5 Compare February 24, 2026 23:10
bugraoz93 and others added 29 commits March 26, 2026 10:30
v0.12.6 is the first that has wheels for 3.13 and 3.14.

Docker image building should be slightly faster as a result.
* Use compat sdk conf import in Informatica provider

* Remove common-compat "use next version" comment
* Use compat sdk conf import in Google Gen AI operators

* Remove common-compat "use next version" comment
* Compat sdk conf follow-up for multiple providers

* Remove common-compat "use next version" comments
…pache#62083)

* Add initial Portuguese (pt) translation agent skill (apache#62001)

* Update pt.md and remove SKILL.md

* Fix markdownlint MD032: add blank lines around lists + confirm license header (apache#62001)

* Remove long ASF license header as per request

* Fix insert-license CI error

---------

Co-authored-by: Jason(Zhe-You) Liu <68415893+jason810496@users.noreply.github.com>
* simplify the bug report template

Changes involve:
- Merging how to reproduce and what happened as these two pieces of information are naturally intertwined and a good reproduction walkthrough usually tells the story of what went wrong along the way.
- apache airflow version is usually a handy information while filing the bug. Instead of giving dropdowns, the user let to mention the airflow version would cover redundant ask of filling the version if it belongs to 3x versions.

* pre-commit fixes

updating the missing line in the bug template

fixing yaml linting issues

* simplify the bug report template

Changes involve:
- Merging how to reproduce and what happened as these two pieces of information are naturally intertwined and a good reproduction walkthrough usually tells the story of what went wrong along the way.
- apache airflow version is usually a handy information while filing the bug. Instead of giving dropdowns, the user let to mention the airflow version would cover redundant ask of filling the version if it belongs to 3x versions.

* pre-commit fixes

updating the missing line in the bug template

fixing yaml linting issues

* reverting linting changes unrelated to the template edit

* add placeholder and update terminology

* retain the airflow version type and structure

Since the issue template is static in nature, hiding the other versions is not allowed.
Retaining the current element type till the alternate options are decided

* add placeholder text for airflow version input and update verbiage.

- updated the airflow version field to have placeholder that hints the reporter run the airflow version command and paste the output.
- minor verbiage update for issue description

* remove quotes for the placeholder
…ITLOperator (apache#64108)

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
…ode (apache#64154)

* refactor: Fixed output encoding with WinRMTrigger
…e#64242)

* Pass parameters to k8s methods conditionally to fix mypy
…alls (apache#64216)

When `assume_role_method` is set to `assume_role_with_web_identity`, the
STS client used to fetch credentials was created without the connection's
botocore config. This meant proxy settings, timeouts, and other config
from `config_kwargs` in the connection extra were silently ignored.

The `assume_role` and `assume_role_with_saml` paths correctly pass
`self.config` to the STS client, but the web identity path passed a raw
`base_session.create_client` as `client_creator` to botocore's
`AssumeRoleWithWebIdentityCredentialFetcher`, which never received the
connection config.

This wraps `client_creator` to merge the connection's botocore config
into any config that botocore passes when creating the STS client,
ensuring proxy and other settings are respected.
…he#63979)

* Improve Playwright test patterns in VariablePage (apache#63965)

- Replace CSS :has-text() with locator.filter({ hasText }) in rowByKey
- Replace CSS attribute selector with getByRole('checkbox') in selectRow
- Replace page.waitForFunction() DOM queries with locator-based
  waiting (Promise.race of noData text vs first table row)
- Replace CSS input[type='checkbox'] with getByRole('checkbox')
  in selectAllCheckbox

Aligns with Playwright best practices per apache#63036.

* Revert checkbox selectors — Chakra hidden input incompatible with getByRole

getByRole('checkbox') resolves to Chakra UI's hidden <input> which
is not visible/stable, causing TimeoutError. Keep original CSS
selectors for checkbox interactions until Chakra components expose
proper accessible roles.

* Use expect().toBeVisible() with .or() combinator

Replace Promise.race + waitFor() with Playwright's built-in
.or() combinator for assertion-based waiting. Verified locally
with 5/5 pass.

---------

Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
Document that providers can be suspended from CI builds and releases
when their dependencies block upgrades, including per-Python-version
suspension. Stewards are responsible for resolving issues and ensuring
green CI before unsuspension.
…pache#64245)

The zip import error fix (apache#63617) changed the public signature of
`DagFileProcessorManager.deactivate_deleted_dags` from
`(bundle_name, present: set[DagFileInfo])` to
`(bundle_name, observed_filelocs: set[str])`, breaking subclass
overrides. Restore the original signature and compute observed
filelocs internally.

Also widen `DagModel.deactivate_deleted_dags` `rel_filelocs` type
from `set[str]` to `Collection[str]` to accept both list and set
callers.
…he#64077)

* Add LLMFileAnalysisOperator and @task.llm_file_analysis to the common-ai provider

# Conflicts:
#	uv.lock

* Fix mypy issues

* Update utils

* Update return model

* Fix spells

* fix up read

* document prefix lookup operation
* avoid passing parsed input back to component

* on change, update component and debounce utc parsing

* typo, linting fixes

* longer type delay, removed redundant isValid check
…ng (apache#64182)

The PoolBar component links to the task instances page using
SearchParamsKeys.STATE ('state') and SearchParamsKeys.POOL ('pool'),
but the TaskInstances page reads filters from SearchParamsKeys.TASK_STATE
('task_state') and SearchParamsKeys.POOL_NAME_PATTERN ('pool_name_pattern').

This mismatch causes clicking a pool slot segment (running, queued, etc.)
to navigate to the task instances page without any filters being applied,
showing all task instances instead of those filtered by the clicked state
and pool.

Fix by using the correct search parameter keys (TASK_STATE and
POOL_NAME_PATTERN) and slot.slotType instead of slot.color for the
state filter value.
…te (apache#64244)

* Fix LLMApprovalMixin to enforce allow_modifications in execute_complete
Co-authored-by: Oleg Kachur <kachur@google.com>
Catch TaskAlreadyRunningError from the supervisor and raise Celery
Ignore() to prevent the broker redelivery from being recorded as a
task failure.

related: apache#58441
gopidesupavan and others added 29 commits April 4, 2026 10:05
* Fix bulk task instance RBAC checks across DAGs

* Update tests

* fix up tests

* Fixup tests

* Resolve comments
---
updated-dependencies:
- dependency-name: aiohttp
  dependency-version: 3.13.4
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [google-cloud-aiplatform](https://github.com/googleapis/python-aiplatform) from 1.99.0 to 1.133.0.
- [Release notes](https://github.com/googleapis/python-aiplatform/releases)
- [Changelog](https://github.com/googleapis/python-aiplatform/blob/main/CHANGELOG.md)
- [Commits](googleapis/python-aiplatform@v1.99.0...v1.133.0)

---
updated-dependencies:
- dependency-name: google-cloud-aiplatform
  dependency-version: 1.133.0
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
This is when the `boto3` extra was removed.
Temporary switch to the latest commit of infrastructure-actions
allowlist-check until apache/infrastructure-actions#662 is merged,
which will provide a proper tagged release.
Bumps the github-actions-updates group with 3 updates: [actions/setup-go](https://github.com/actions/setup-go), [github/codeql-action](https://github.com/github/codeql-action) and [astral-sh/setup-uv](https://github.com/astral-sh/setup-uv).


Updates `actions/setup-go` from 6.3.0 to 6.4.0
- [Release notes](https://github.com/actions/setup-go/releases)
- [Commits](actions/setup-go@4b73464...4a36011)

Updates `github/codeql-action` from 4.32.6 to 4.35.1
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](github/codeql-action@0d579ff...c10b806)

Updates `astral-sh/setup-uv` from 7.6.0 to 8.0.0
- [Release notes](https://github.com/astral-sh/setup-uv/releases)
- [Commits](astral-sh/setup-uv@37802ad...cec2083)

---
updated-dependencies:
- dependency-name: actions/setup-go
  dependency-version: 6.4.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: github-actions-updates
- dependency-name: github/codeql-action
  dependency-version: 4.35.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: github-actions-updates
- dependency-name: astral-sh/setup-uv
  dependency-version: 8.0.0
  dependency-type: direct:production
  update-type: version-update:semver-major
  dependency-group: github-actions-updates
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
apache#64674)

* Add SlackAPIConversationsHistoryOperator for conversations.history API

* Add SlackAPIConversationsHistoryOperator to Slack example DAG and docs

* ci: rerun

* ci: rerun
Reset the OTel SDK's Once() guard on _METER_PROVIDER_SET_ONCE before
calling set_meter_provider() in get_otel_logger(). When a forked child
process re-initializes Stats (detected via PID mismatch in stats.py),
the inherited Once._done = True flag prevents the new MeterProvider from
being registered. The child falls back to the parent's stale provider
whose PeriodicExportingMetricReader thread is dead after fork, causing
task-level metrics like ti.finish to be silently dropped.

The fix resets _done and _METER_PROVIDER before each set_meter_provider()
call. On first initialization (no fork), _done is already False so this
is a no-op. On re-initialization after fork, it allows the new provider
to be set correctly.

Closes: apache#64690
…avior (apache#64699)

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…64719)

BulkAction enum values were rendered as "BulkAction.UPDATE" / "BulkAction.DELETE"
instead of "update" / "delete" in authorization error messages because Python's
str enum __format__ returns the enum repr, not the value. Use .value to get the
plain string.
* Load hook metadata from YAML without importing Hook class

* Add hook-name to all provider.yaml connection-types

* Add hook-name to connection types and regenerate get_provider_info.py

* Fix ruff import order in connections.py

* fix: import ProvidersManager at top level per review

* Fix provider connection hook display names

* Add iter_connection_type_hook_ui_metadata for connection UI hook metadata
Verify rustup-init binary with SHA256 checksum instead of curl-pipe-sh

Download the rustup-init binary directly and verify its SHA256 checksum
before execution, instead of piping the shell installer script through sh.

Pin rustup-init to version 1.29.0 with hardcoded SHA256 checksums for
amd64 and arm64, matching the existing cosign verification pattern.
This prevents a compromised server from serving a tampered binary with
a matching checksum.
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…#62849)

* Fix dict args in structlog positional formatting

When a dict was passed as a positional argument to a log message
(e.g. log.warning('message %s', {'a': 10})), both the structlog
bound logger and the stdlib logging path would try named substitution
first, causing TypeError for positional format specifiers like %s.

Fix both paths to match CPython's stdlib logging behavior: try
positional formatting (msg % args) first, fall back to named
substitution (msg % args[0]) only on TypeError/KeyError.

- In _make_airflow_structlogger.meth(): try event % args first,
  fall back to named substitution on failure
- Add positional_arguments_formatter() to replace structlog's built-in
  PositionalArgumentsFormatter, which has the same ordering bug for
  stdlib logging records

Fixes apache#62201

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: add missing blank line before parametrize decorator

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
…63949)

When execution_api_server_url was moved from [workers] to [core], the transition lacked a backward-compatibility deprecation mapping in parser.py.

As a result, Airflow 3 SDK ignores user configurations still placed under [workers], falling back to an often unresolvable Default URL derived from api.base_url. This caused Task SDK pods to crash on startup with an httpcore.ConnectError and prevented the task's hostname from being properly reported, resulting in Invalid URL log fetch errors in the UI.

This commit adds execution_api_server_url to deprecated_options to restore proper fallback behavior.
…pache#64322)

* fix(scheduler): skip asset-triggered dags without SerializedDagModel in dags_needing_dagruns

Remove those dag_ids from the in-memory candidate set until serialization
exists; retain AssetDagRunQueue rows and emit DEBUG logs. Add unit tests and
a bugfix newsfragment.

* fix(scheduler): prevent premature asset-triggered DagRuns when SerializedDagModel is unavailable

* test(dag): persist DagModel before AssetDagRunQueue in unit tests

Split DagModel and AssetDagRunQueue inserts and flush after DagModel so foreign-key order matches production DB constraints in TestDagModel.

* Apply suggestions from code review

Co-authored-by: Wei Lee <weilee.rx@gmail.com>

* refactor(dag): clarify ADRQ skip log and condense serialized-DAG guard

Combine the missing-from-serialized set check with a walrus assignment and improve the debug message when DagRun creation is skipped for DAGs with queued asset events but no SerializedDagModel row.

* test(models): align caplog assertions with updated serialized dag warnings

* test(dag): align ADRQ missing-serialized log assertion with message text

* Apply suggestion from @jscheffl

Co-authored-by: Jens Scheffler <95105677+jscheffl@users.noreply.github.com>

* chore(newsfragments): remove 64322.bugfix.rst

---------

Co-authored-by: Wei Lee <weilee.rx@gmail.com>
Co-authored-by: Jens Scheffler <95105677+jscheffl@users.noreply.github.com>
…ibutors (apache#64516)

* Add UI demo/screenshot check to auto-triage for area:UI PRs

Add a deterministic check to `breeze pr auto-triage` that flags
`area:UI` PRs from non-collaborator contributors when their PR
description lacks screenshots or demo videos.

The check runs as part of the existing deterministic checks in
`_assess_pr_deterministic()` alongside CI failure, merge conflict,
and unresolved comment checks. PRs flagged only for missing UI demo
receive a comment (not draft conversion), using a new soft-violation
branch in `_compute_default_action`.

Detection patterns cover GitHub drag-and-drop uploads (`<img>` tags,
`user-attachments/assets/` URLs), markdown image syntax, and direct
media file URLs (png, jpg, gif, mp4, etc).

* Clarify that demo should be in PR description, not comments

* Remove soft-violation special-case in _compute_default_action

Per reviewer feedback from @pierrejeambrun and @potiuk: the
separate elif branch for soft-only violations (e.g. missing UI
demo) defaulting to COMMENT is unnecessary — maintainers can
override DRAFT to COMMENT interactively. A future enhancement
will let each check define its own suggested action severity.

---------

Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
…e#63882)

* Fix start_date in example DAGs to avoid TZ conversion overflow

* Update airflow-core/src/airflow/example_dags/example_inlet_event_extra.py

Co-authored-by: yuseok89 <yuseok89@gmail.com>

* Update airflow-core/src/airflow/example_dags/example_inlet_event_extra.py

Co-authored-by: yuseok89 <yuseok89@gmail.com>

* Update airflow-core/src/airflow/example_dags/example_outlet_event_extra.py

Co-authored-by: yuseok89 <yuseok89@gmail.com>

* Update airflow-core/src/airflow/example_dags/example_outlet_event_extra.py

Co-authored-by: yuseok89 <yuseok89@gmail.com>

* Apply suggestion from @yuseok89

Co-authored-by: yuseok89 <yuseok89@gmail.com>

---------

Co-authored-by: yuseok89 <yuseok89@gmail.com>
…pache#62030)

* Add workers.celery.schedulerName & workers.kubernetes.schedulerName

* Deprecate section
Bumps [litellm](https://github.com/BerriAI/litellm) from 1.82.6 to 1.83.0.
- [Release notes](https://github.com/BerriAI/litellm/releases)
- [Commits](https://github.com/BerriAI/litellm/commits)

---
updated-dependencies:
- dependency-name: litellm
  dependency-version: 1.83.0
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
- Quote user-controlled values in _build_ssh_command with shlex.quote
- Validate strict_host_key_checking against allowlist of valid SSH values
- URL-encode username and auth token when embedding in repo URLs
- Fix logic bug: `not X or not Y` → `not X and not Y` for git@/https:// check
- Limit str.replace to first occurrence
Instead of raising a 404/NotFound error when calling `GCSHook.delete()`,
match the behavior of `delete_bucket` and log the error.
Makes the `GCSHook.delete` `ignore_error` boolean available to
ignore errors when a blob doesn't exist.
@tuzonghua
tuzonghua force-pushed the google/gcsdeleteobjectsoperator_allow_skip_error branch from ca7cdbd to 0451700 Compare April 6, 2026 13:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.