Skip to content

ci: retain API unit test reports - #42583

Merged
lyzno1 merged 1 commit into
mainfrom
ci/api-unit-test-timings
Sep 20, 2026
Merged

lyzno1 merged 1 commit into
mainfrom
ci/api-unit-test-timings

Conversation

@hyoban

@hyoban hyoban commented Sep 20, 2026

Copy link
Copy Markdown
Member

Summary

Retain separate JUnit reports for parallel API unit tests and serial controllers for seven days. Upload reports after failures when available, providing testcase identities and durations for CI validation. Unbuffer Python output for timely logs. Test scheduling and coverage remain unchanged in this layer.

Fixes #42582

Stack layer 1/5. Next: #42585 (remove real waits).

Validation

  • JUnit reporting was validated in the baseline CI; collection timing and extra slow-phase output from that experiment have been removed.
  • Ten focused tests passed serially and with two xdist workers; both JUnit files contained ten cases with durations.
  • Ruff, targeted Pyrefly, and Actionlint checks passed.

Checklist

  • This change requires a documentation update, included: Dify Document
  • I understand that this PR may be closed in case there was no previous discussion or issues. (This doesn't apply to typos!)
  • I've verified the change and added or updated tests where meaningful regression risk justifies coverage.
  • I've updated the documentation accordingly.
  • I ran make lint && make type-check (backend) and vp staged (frontend) to appease the lint gods

@github-actions

github-actions Bot commented Sep 20, 2026

Copy link
Copy Markdown
Contributor

Pyrefly Type Coverage

Metric Base PR Delta
Type coverage 64.94% 64.95% +0.00%
Strict coverage 64.55% 64.55% +0.00%
Typed symbols 47,120 47,129 +9
Untyped symbols 25,591 25,592 +1
Modules 3415 3416 +1

@codecov

codecov Bot commented Sep 20, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 87.56%. Comparing base (3a0acfe) to head (5f4e55f).

Additional details and impacted files
@@            Coverage Diff             @@
##             main   #42583      +/-   ##
==========================================
- Coverage   87.56%   87.56%   -0.01%     
==========================================
  Files        5480     5480              
  Lines      317375   317375              
  Branches    63615    63615              
==========================================
- Hits       277896   277894       -2     
- Misses      34114    34117       +3     
+ Partials     5365     5364       -1     
Flag Coverage Δ
api 87.23% <ø> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@github-actions

github-actions Bot commented Sep 20, 2026

Copy link
Copy Markdown
Contributor

Pyrefly Diff

base → PR
--- /tmp/pyrefly_base.txt	2026-09-20 10:02:10.722655841 +0000
+++ /tmp/pyrefly_pr.txt	2026-09-20 10:02:02.786603643 +0000
@@ -147,13 +147,13 @@
 ERROR Missing argument `using_ugc` in function `dify_vdb_lindorm.lindorm_vector.LindormVectorStore.__init__` [missing-argument]
   --> providers/vdb/vdb-lindorm/tests/integration_tests/test_lindorm.py:41:41
 ERROR Object of class `ModuleType` has no attribute `BulkIndexError` [missing-attribute]
-  --> providers/vdb/vdb-lindorm/tests/unit_tests/test_lindorm_vector.py:39:5
-ERROR Object of class `ModuleType` has no attribute `bulk` [missing-attribute]
   --> providers/vdb/vdb-lindorm/tests/unit_tests/test_lindorm_vector.py:40:5
+ERROR Object of class `ModuleType` has no attribute `bulk` [missing-attribute]
+  --> providers/vdb/vdb-lindorm/tests/unit_tests/test_lindorm_vector.py:41:5
 ERROR Object of class `ModuleType` has no attribute `OpenSearch` [missing-attribute]
-  --> providers/vdb/vdb-lindorm/tests/unit_tests/test_lindorm_vector.py:42:5
-ERROR Object of class `ModuleType` has no attribute `helpers` [missing-attribute]
   --> providers/vdb/vdb-lindorm/tests/unit_tests/test_lindorm_vector.py:43:5
+ERROR Object of class `ModuleType` has no attribute `helpers` [missing-attribute]
+  --> providers/vdb/vdb-lindorm/tests/unit_tests/test_lindorm_vector.py:44:5
 ERROR Object of class `ModuleType` has no attribute `MoVectorClient` [missing-attribute]
   --> providers/vdb/vdb-matrixone/tests/unit_tests/test_matrixone_vector.py:31:5
 ERROR Object of class `ModuleType` has no attribute `client` [missing-attribute]
@@ -6685,13 +6685,13 @@
 ERROR Object of class `NoneType` has no attribute `get_data` [missing-attribute]
    --> tests/unit_tests/services/test_audio_service.py:828:16
 ERROR Argument `BillingInfo` is not assignable to parameter `result` with type `dict[Unknown, Unknown]` in function `TestBillingServiceSubscriptionInfoDataType._assert_billing_info_types` [bad-argument-type]
-    --> tests/unit_tests/services/test_billing_service.py:1875:41
+    --> tests/unit_tests/services/test_billing_service.py:1898:41
 ERROR Argument `BillingInfo` is not assignable to parameter `result` with type `dict[Unknown, Unknown]` in function `TestBillingServiceSubscriptionInfoDataType._assert_billing_info_types` [bad-argument-type]
-    --> tests/unit_tests/services/test_billing_service.py:1884:41
+    --> tests/unit_tests/services/test_billing_service.py:1907:41
 ERROR Argument `BillingInfo` is not assignable to parameter `result` with type `dict[Unknown, Unknown]` in function `TestBillingServiceSubscriptionInfoDataType._assert_billing_info_types` [bad-argument-type]
-    --> tests/unit_tests/services/test_billing_service.py:1897:41
+    --> tests/unit_tests/services/test_billing_service.py:1920:41
 ERROR Argument `BillingInfo` is not assignable to parameter `result` with type `dict[Unknown, Unknown]` in function `TestBillingServiceSubscriptionInfoDataType._assert_billing_info_types` [bad-argument-type]
-    --> tests/unit_tests/services/test_billing_service.py:1908:41
+    --> tests/unit_tests/services/test_billing_service.py:1931:41
 ERROR Argument `FakeRepo` is not assignable to parameter `workflow_run_repo` with type `APIWorkflowRunRepository | None` in function `services.retention.workflow_run.clear_free_plan_expired_workflow_run_logs.WorkflowRunCleanup.__init__` [bad-argument-type]
    --> tests/unit_tests/services/test_clear_free_plan_expired_workflow_run_logs.py:130:49
 ERROR Class member `FailingRepo.delete_runs_with_related_by_ids` overrides a member in a parent class but is missing an `@override` decorator [missing-override-decorator]

@hyoban hyoban changed the title ci: report API unit test and collection timings ci: reduce API unit test collection and retry overhead Sep 20, 2026
@hyoban
hyoban force-pushed the ci/api-unit-test-timings branch from 8ac9e44 to c91cd6c Compare September 20, 2026 10:12
@hyoban
hyoban added this pull request to stack #42588 September 20, 2026 10:12
@hyoban hyoban changed the title ci: reduce API unit test collection and retry overhead ci: report API unit test and collection timings Sep 20, 2026
@hyoban
hyoban removed this pull request from stack #42588 September 20, 2026 11:48
@hyoban
hyoban added this pull request to stack #42594 September 20, 2026 11:48
@hyoban
hyoban force-pushed the ci/api-unit-test-timings branch from c91cd6c to 4e0bcf8 Compare September 20, 2026 12:30
@hyoban hyoban changed the title ci: report API unit test and collection timings ci: retain API unit test reports Sep 20, 2026
@hyoban
hyoban force-pushed the ci/api-unit-test-timings branch from 4e0bcf8 to 5f4e55f Compare September 20, 2026 12:44
@hyoban
hyoban marked this pull request as ready for review September 20, 2026 12:44
@lyzno1
lyzno1 added this pull request to the merge queue Sep 20, 2026
Merged via the queue into main with commit 57b9d73 Sep 20, 2026
45 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

API unit test CI lacks timings for slow-test investigation

2 participants