ci: stabilize iOS simulator startup before functional tests

[KazuCocoa]

The iOS job currently relies on simulator-action boot completion and leaves Simulator UI startup to the first Appium session. In run 34810153703, the first iOS fixture failed because the simulator did not finish booting within 120 seconds, while subsequent fixtures ran.

Align startup with the existing WebDriverAgent workflow and XCUITest driver workflow:

• Open the selected Xcode's Simulator UI before preparing the simulator, and list devices/runtimes for diagnostics.
• Adapt the driver's dependency-free idle helper to wait for background services immediately before functional tests. Keep the upstream defaults: three consecutive samples below 20% aggregate child-process CPU, sampled every two seconds, with a 150-second best-effort limit. A three-minute step timeout bounds unexpected hangs.
• Add dependency-free CLI tests and run them in the iOS job.

Validation: all five helper tests pass (selected-device isolation, aggregate CPU and consecutive-sample reset, timeout, disappearing simulator, missing UDID); Node syntax check, actionlint, and git diff --check pass. The complete iOS suite remains for PR CI. The observed timeout motivates this alignment, but does not establish CPU contention as its root cause; improved flake rate needs repeated hosted-runner results. The helper preserves upstream warning-and-continue behavior when the simulator disappears or does not settle.

[Copilot]

🟢 Approval recommended

The remaining comment is a non-blocking nit regarding CPU-contention wording.

Pull request overview

This pull request stabilizes iOS simulator startup before functional tests by launching Simulator, waiting for background services to settle, and targeting the prepared simulator explicitly.

Changes:

• Adds simulator startup diagnostics and UI initialization.
• Adds dependency-free idle checks and CLI tests.
• Propagates the simulator UDID into iOS capabilities.

File summaries

File	Summary
test/integration/helpers/Caps.cs	Adds optional simulator UDID capability support.
.github/workflows/functional-test.yml	Integrates diagnostics, startup stabilization, idle checks, and helper tests.
.github/scripts/wait-for-simulator-idle.test.mjs	Tests helper behavior and edge cases.
.github/scripts/wait-for-simulator-idle.mjs	Waits for simulator background processes to settle; includes a non-blocking rationale-wording nit.

Review details

Suppressed comments (1)

.github/scripts/wait-for-simulator-idle.mjs:10

• The PR description explicitly says the reported timeout does not establish CPU contention as its root cause, but this new comment states that CPU contention has been observed to cause the 100+ second tap. Please avoid attributing the failure to CPU contention here (or cite separate evidence) so the rationale matches the stated scope.

// from that burst has been observed to turn a single native tap into a 100+ second operation on CI.

• Files reviewed: 3/3 changed files
• Comments generated: 0
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

🟢 Approval recommended

No unresolved review issues were identified, and the described validation passes.

Review details

• Files reviewed: 3/3 changed files
• Comments generated: 0 new
• Review effort level: Lite

[Dor-bl]

⏱️ CI Run Investigation: Why ios-tests Took ~48 Minutes (Run 34855842533)

The ios-tests job completed in 47m 49s, compared to historical runs of ~22–25 minutes. Approximately ~25 minutes of unnecessary overhead was introduced in this run.

---

📊 Step Duration Breakdown

Step	Duration	Status	Notes
Setup iOS Simulator	22m 45s	⚠️ Retry Loop	Normally takes ~1m 40s; failed 3 bootstatus attempts
Build solution & Restore	~35s	✅ Success	Expected
Finalize simulator boot	2m 33s	⚠️ Timeout	Hit full 150s timeout (CPU stayed >20%)
Run iOS functional tests	20m 17s	❌ Failed 1 test	Normal test suite execution time (~20m)
Complete job (cleanup)	~25s	✅ Success	Terminated orphan Simulator process

---

🔍 Root Causes

1. Start Simulator UI before Setup iOS Simulator caused a 22.5-minute boot hang

In .github/workflows/functional-test.yml:

- name: Start Simulator UI
  run: open -Fn "$(xcode-select --print-path)/Applications/Simulator.app"

- name: Setup iOS Simulator
  id: simulator
  uses: futureware-tech/simulator-action@v5
  ...

• What happened:
  1. open -Fn .../Simulator.app launched the GUI Simulator.app without passing any device UDID.
  2. macOS Simulator.app started up and defaulted to launching its own default device, locking simulator daemons and CoreSimulator resources.
  3. futureware-tech/simulator-action then ran and attempted to boot the specific target simulator (iPhone 16 / 18.5) and execute xcrun simctl bootstatus <UDID>.
  4. Because Simulator.app was running in conflicting state, xcrun simctl bootstatus hung until simulator-action's 360-second (6 minute) timeout tripped.
  5. simulator-action retried across 4 attempts:
     • Attempt 1: 15:28:05 → 15:34:07 (6m 02s) ❌
     • Attempt 2: 15:35:47 → 15:41:47 (6m 00s) ❌
     • Attempt 3: 15:42:17 → 15:48:18 (6m 01s) ❌
     • Attempt 4: 15:48:31 → 15:49:23 (52s) ✅
  • Result: 22m 45s wasted in retries (the exact same 27m retry loop occurred in run 34855549707). In previous runs without Start Simulator UI (e.g., run 34813368448), Setup iOS Simulator took only 1m 42s.

2. Finalize simulator boot CPU threshold (20%) is unreachable on GitHub Actions macOS runners

In .github/scripts/wait-for-simulator-idle.mjs:

const CPU_THRESHOLD_PERCENT = Number(process.env.WAIT_SIM_IDLE_CPU_THRESHOLD ?? 20);

• What happened:
  On GitHub-hosted macos-15 runners, an active iOS 18.5 simulator with SpringBoard and background services consistently consumes 45%–94% CPU (averaging ~55–60%).
• Result: The child-process aggregate CPU never reached < 20%. The script always ran for the entire MAX_WAIT_MS (150s), warned Simulator ... did not settle within 150s (last cpu=62%); proceeding anyway, and exited. This adds 2m 33s of unconditional delay to every job.

3. Test Failure: WebviewTest.GetPageTestCase

• Failure: OpenQA.Selenium.UnknownErrorException : No such context found. at _driver.Context = webviewContext.
• Cause: Clicking "Web View" requires a brief delay for the web context to register in XCUITest. _driver.Contexts only had ["NATIVE_APP"] when queried immediately, making webviewContext null. This is an independent assertion timing issue.

---

🛠️ Actions Required

1. Move or remove Start Simulator UI:

   • Do not launch Simulator.app prior to simulator-action.
   • If pre-launching the GUI is desired so Appium does not do it on the first test session, open it after Setup iOS Simulator and explicitly target the booted device's UDID:

     - name: Setup iOS Simulator
       id: simulator
       uses: futureware-tech/simulator-action@v5
       with:
         model: ${{ env.IOS_DEVICE_NAME }}
         os_version: ${{ env.IOS_VERSION }}
         wait_for_boot: true
         shutdown_after_job: false
     
     - name: Start Simulator UI
       run: open -a "$(xcode-select --print-path)/Applications/Simulator.app" --args -CurrentDeviceUDID ${{ steps.simulator.outputs.udid }}

     (Alternatively, remove Start Simulator UI altogether, as futureware-tech/simulator-action boots the simulator and Appium/XCUITest attaches cleanly via UDID).

2. Adjust or bypass Finalize simulator boot idle check:

   • Adjust the threshold and max wait so it does not block for 2.5 minutes if the simulator is already responsive, e.g.:

     env:
       WAIT_SIM_IDLE_CPU_THRESHOLD: '65'
       WAIT_SIM_IDLE_MAX_WAIT_MS: '30000'

   • Or rely directly on xcrun simctl bootstatus, which already ensures system services are booted.

3. (Optional) Add polling in WebviewTest.cs:

   • Poll with a small timeout (e.g. DefaultWait) for _driver.Contexts.Any(c => c.Contains("WEBVIEW")) before switching context.

[Dor-bl]

The iOS-tests job completed in 47m 49s, compared to historical runs of ~22–25 minutes. Approximately ~25 minutes of unnecessary overhead was introduced in this run.

See my comment for more details.

[Dor-bl]

@KazuCocoa Can you please check the merge conflicts?