Skip to content

fix(postprocessing): skip UploadReady events without an upload id - #3607

Open
junkerderprovinz wants to merge 1 commit into
opencloud-eu:mainfrom
junkerderprovinz:postprocessing-skip-uploadready-without-upload
Open

junkerderprovinz wants to merge 1 commit into
opencloud-eu:mainfrom
junkerderprovinz:postprocessing-skip-uploadready-without-upload

Conversation

@junkerderprovinz

Copy link
Copy Markdown

Description

The posix driver publishes UploadReady for every non-empty file it assimilates from disk, so search, activitylog and clientlog see files that were copied into the storage directly. Those events have no UploadID because there is no upload session. Postprocessing looked the empty ID up in its store, which fails with nats: invalid key, and logged errors for every such file: two lines on 8.0.1, three on main because getPP logs as well.

Postprocessing now returns early for an UploadReady without UploadID, like it already does for a PostprocessingStepFinished from an on-demand scan. The event is still acked. The other consumers don't read UploadID and are unchanged.

A real upload can't hit the new branch. Its UploadReady comes from decomposedfs after PostprocessingFinished and carries the ID that postprocessing used as its store key, so an empty ID never matches a stored upload.

Related Issue

Motivation and Context

The errors look like failed uploads and bury real ones. On 8.0.1, 300 files copied into a personal space gave 600 error lines at the next start.

How Has This Been Tested?

  • test environment: unit tests built with Go 1.26.4 for linux/amd64 and run in an Alpine container. Server built from this branch in the golang:1.26 image and run on data from opencloudeu/opencloud-rolling:latest (8.0.1), posix driver.
  • test case 1: new unit test sends an UploadReady without UploadID and expects no error. Without the change it fails with event error: cannot get upload.
  • test case 2: new unit test checks that a failed upload is still marked as finished, so events with an ID still reach the store.
  • test case 3: the server built from this branch handled 600 UploadReady events for assimilated files without errors (reproduction in postprocessing logs "cannot get upload" for every non-empty file the posix driver assimilates #3606). The files were listed in the space, and a WebDAV upload still went through postprocessing.

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Technical debt
  • Tests only (no source changes)

Checklist:

  • Code changes
  • Unit tests added
  • Acceptance tests added
  • Documentation added

Claude Code helped with the investigation, the reproduction and the fix. I reviewed the change and take responsibility for it.

Assisted-by: Claude Code:claude-opus-5-5

@codacy-production

Copy link
Copy Markdown

Up to standards ✅

🟢 Issues 0 issues

Results:
0 new issues

View in Codacy

🟢 Metrics 5 complexity

Metric Results
Complexity 5

View in Codacy

NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

postprocessing logs "cannot get upload" for every non-empty file the posix driver assimilates

1 participant