Skip to content

[ci-scan] Hang: System.Collections.Tests on osx.15.amd64.open #129385

Closed as not planned
Closed as not planned
[ci-scan] Hang: System.Collections.Tests on osx.15.amd64.open#129385
Labels
Known Build ErrorUse this to report build issues in the .NET Helix tabagentic-workflowsarea-System.Collections
Milestone
Future

Description

@github-actions
github-actions
bot
opened on Jun 14, 2026

Build Information

Build: https://dev.azure.com/dnceng-public/public/_build/results?buildId=1462616
Build error leg or test failing: osx-x64 Release Libraries_Release_CoreCLR - System.Collections.Tests

Error Details

System.Collections.Tests times out on the osx.15.amd64.open Helix queue. The test starts running (discovers 7118 test cases, begins parallel execution on 6 threads) but does not complete within the Helix work item timeout. Exit code -3 (SIGQUIT/killed).

Discovering: System.Collections.Tests (method display = ClassAndMethod, method display options = None)
Discovered:  System.Collections.Tests (found 7118 of 9212 test cases)
Starting:    System.Collections.Tests (parallel test collections = on [6 threads], stop on fail = off)
['System.Collections.Tests' END OF WORK ITEM LOG: Command timed out, and was killed]

Error Message

true only for clear infra flakes. ExcludeConsoleLog skips helix log scanning. -->

{
  "ErrorMessage": "System.Collections.Tests' END OF WORK ITEM LOG: Command timed out, and was killed",
  "ErrorPattern": "",
  "BuildRetry": false,
  "ExcludeConsoleLog": false
}

Impact on platforms

  • runtime-extra-platforms (def 154) / osx-x64 Release Libraries_Release_CoreCLR / Helix queue: osx.15.amd64.open / exit code: -3 (timeout killed)

First build it occurred

Confirmed in follow-up build: https://dev.azure.com/dnceng-public/public/_build/results?buildId=1463147 (same test timed out, along with 5 other test assemblies on the same queue)

Duplicate search

No existing KBE matches this signature.

Filed by ci-failure-scan (detection only). ci-failure-fix walks open [ci-scan] KBEs and either opens a small fix PR or comments here to loop in owners — it never disables the test.

Note

🔒 Integrity filter blocked 4 items

The following items were blocked because they don't meet the GitHub integrity level.

  • #124417 search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
  • #125436 search_pull_requests: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
  • #125670 search_pull_requests: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
  • #90312 search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".

To allow these resources, lower min-integrity in your GitHub frontmatter:

tools:
  github:
    min-integrity: approved  # merged | approved | unapproved | none

Generated by CI Outer-Loop Failure Scanner · ● 72.4M · ◷

Report

Summary

24-Hour Hit Count 7-Day Hit Count 1-Month Count
0 0 0

Activity

  1. github-actions
    added
    blocking-clean-ciBlocking PR or rolling runs of 'runtime' or 'runtime-extra-platforms'
    Known Build ErrorUse this to report build issues in the .NET Helix tab
    agentic-workflows
    on Jun 14, 2026
  2. dotnet-policy-service
    added
    untriagedNew issue has not been triaged by the area owner
    on Jun 14, 2026
  3. jeffschwMSFT
    added
    area-System.Collections
    on Jun 16, 2026

  4. dotnet-policy-service commented on Jun 16, 2026

    @dotnet-policy-service
    dotnet-policy-servicebot
    on Jun 16, 2026 – with Dotnet Policy Service
    Contributor

    Tagging subscribers to this area: @dotnet/area-system-collections
    See info in area-owners.md if you want to be subscribed.

  5. github-actions commented on Jun 19, 2026

    @github-actions
    github-actionsbot
    on Jun 19, 2026 – with GitHub Actions
    ContributorAuthor

    Workflow artifact: ci-fix
    Artifact kind: handoff
    Linked KBE: #129385

    Note

    AI/Copilot-generated triage note.

    This Known Build Error has no producible automated code change (reason: test timeout/hang in System.Collections.Tests on osx — no specific test failure or code defect is identifiable from the timeout, and the root cause may be platform infrastructure), so I could not open even a best-effort PR. Looping in owners so it can be fixed forward rather than muted.

    Root cause (best analysis)

    System.Collections.Tests hangs/times out on osx.15.amd64.open Helix queue. The Helix work item System.Collections.Tests exceeds its timeout (typically 15 minutes), reporting:

    The test execution exceeded a custom timeout and was killed.

    The test suite runs 7118+ test cases across 6 threads. The timeout occurs without identifying a specific hanging test, which makes it difficult to determine whether this is:

    1. A platform-specific deadlock in a test (e.g., concurrent collection test with platform-dependent threading behavior on macOS 15)
    2. A macOS 15 system-level issue (e.g., memory pressure, thread scheduling differences)
    3. An infrastructure issue with the osx.15.amd64.open Helix queue

    The failure is intermittent with low frequency.

    Evidence

    Suggested reviewers / area contacts

    • Area owners (area-System.Collections): @dotnet/area-system-collections
    • Lead: @jeffhandley

    If the hang recurs, attaching a diagnostic dump (createdump) or enabling ETW/EventPipe tracing on the Helix work item would help identify the specific hanging test and thread state.

    Filed by ci-failure-fix, which attempts validated fixes for [ci-scan] Known Build Errors and otherwise loops in owners. Comment here or on the workflow file to suggest changes; ci-failure-scan-feedback reads in-scope feedback daily and opens (or updates) a PR with prompt edits.

    Note

    🔒 Integrity filter blocked 2 items

    The following items were blocked because they don't meet the GitHub integrity level.

    To allow these resources, lower min-integrity in your GitHub frontmatter:

    tools:
  github:
    min-integrity: approved  # merged | approved | unapproved | none

    Generated by CI Outer-Loop Failure Fixer · ● 17.2M · ◷

  6. github-actions commented on Aug 9, 2026

    @github-actions
    github-actionsbot
    on Aug 9, 2026 – with GitHub Actions
    ContributorAuthor

    Workflow artifact: ci-fix
    Artifact kind: handoff
    Linked KBE: #129385

    Note

    AI/Copilot-generated triage note.

    This Known Build Error has no producible automated code change (reason: a test-run timeout/hang with no identified faulting code — an infra/hang investigation rather than a localizable source bug), so I could not open even a best-effort PR. Looping in owners so it can be fixed forward rather than muted.

    Root cause (best analysis)

    System.Collections.Tests times out (Helix work item killed, exit code -3) on the osx.15.amd64.open queue:

    Discovered:  System.Collections.Tests (found 7118 of 9212 test cases)
Starting:    System.Collections.Tests (parallel test collections = on [6 threads] ...)
['System.Collections.Tests' END OF WORK ITEM LOG: Command timed out, and was killed]

    Execution begins (7118 cases discovered, 6-thread parallel run starts) but never completes within the work-item timeout. The KBE notes the same queue also killed other assemblies (System.Text.RegularExpressions.Tests, plus 5 others in a follow-up build), which points at a queue/machine-level slowdown or a hang rather than a defect isolated to System.Collections.Tests. With no faulting test, stack, or assertion to localize — and the multi-assembly pattern on the same osx.15.amd64.open queue — this is an infrastructure/hang investigation, not a source diff. No non-disabling code change is producible.

    Evidence

    Suggested reviewers / area contacts

    • Area owner (area-System.Collections): @jeffhandley, @dotnet/area-system-collections

    A fix likely involves determining whether the osx.15.amd64.open queue is under-provisioned/overloaded (multiple assemblies timing out together) versus a genuine hang in a specific System.Collections test, e.g. by capturing a dump on timeout, before deciding on a runtime vs. infra remedy.

    Warning

    Firewall blocked 1 domain

    The following domain was blocked by the firewall during workflow execution:

    • awmgmcpg

    To allow these domains, add them to the network.allowed list in your workflow frontmatter:

    network:
  allowed:
    - defaults
    - "awmgmcpg"

    See Network Configuration for more information.

    Generated by CI Outer-Loop Failure Fixer · 339.4 AIC · ⌖ 21 AIC · ⊞ 17.1K · ◷

  7. steveisok
    removed
    untriagedNew issue has not been triaged by the area owner
    on Aug 11, 2026
  8. steveisok
    added this to the Future milestone on Aug 11, 2026
  9. jeffschwMSFT
    removed
    blocking-clean-ciBlocking PR or rolling runs of 'runtime' or 'runtime-extra-platforms'
    on Aug 14, 2026

  10. jeffschwMSFT commented on Aug 14, 2026

    @jeffschwMSFT
    jeffschwMSFT
    on Aug 14, 2026
    Member

    removing blocking-clean-ci as it has not failed in 30 days

    24-Hour Hit Count 7-Day Hit Count 1-Month Count
    0 0 0

  11. jkotas
    closed this as not plannedon Aug 14, 2026
  12. github-actions
    locked and limited conversation to collaborators on Sep 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Known Build ErrorUse this to report build issues in the .NET Helix tabagentic-workflowsarea-System.Collections

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions