Skip to content

Stabilize MSBuild invocation tests in LLM environments - #56263

Draft
JeremyKuhne with Copilot wants to merge 3 commits into
mainfrom
copilot/fix-ci-flake-mac-arm64
Draft

JeremyKuhne with Copilot wants to merge 3 commits into
mainfrom
copilot/fix-ci-flake-mac-arm64

Conversation

Copilot AI commented Sep 11, 2026 •

Copy link
Copy Markdown
Contributor

Ambient LLM environment variables can append -tlp:DISABLENODEDISPLAY, causing exact MSBuild invocation assertions to fail intermittently.

  • Invocation assertions

    • Normalize the environment-dependent terminal logger argument before comparing command shapes.
    • Apply normalization consistently across exact build, clean, MSBuild, publish, and store assertions.
  • LLM detection coverage

    • Share the complete detector environment-variable list.
    • Clear and restore all detector inputs so positive and negative cases are isolated from the host environment.

Production CLI behavior is unchanged.

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
3 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

Co-authored-by: JeremyKuhne <8184940+JeremyKuhne@users.noreply.github.com>
Copilot AI changed the title [WIP] Fix CA CLI invocation tests flake on Mac arm64 Helix Stabilize MSBuild invocation tests in LLM environments Sep 11, 2026
Copilot AI requested a review from JeremyKuhne September 11, 2026 14:35

@JeremyKuhne JeremyKuhne left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot #56258 (comment) has a good question. Check to see if there is an avoidable (or isolatable) environment change to the variable.

Co-authored-by: JeremyKuhne <8184940+JeremyKuhne@users.noreply.github.com>

Copilot AI commented Sep 11, 2026

Copy link
Copy Markdown
Contributor Author

@copilot #56258 (comment) has a good question. Check to see if there is an avoidable (or isolatable) environment change to the variable.

Addressed in ee042c1. The avoidable change came from two parallel in-process detector tests, not Helix setup. They now run with DoNotParallelize, and I removed the argument-normalization workaround so invocation assertions remain strict.

@mmitche

mmitche commented Sep 11, 2026

Copy link
Copy Markdown
Member

@JeremyKuhne I think I might have a better fix..one sec.

@mmitche

mmitche commented Sep 14, 2026

Copy link
Copy Markdown
Member

@JeremyKuhne #56275

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

CA CLI invocation tests flake on Mac arm64 Helix due to -tlp:DISABLENODEDISPLAY leaking through AdjustMSBuildForLLMs

3 participants