Repository navigation
[No QA] Synthetic target for Melvin agent-device mobile smoke test - #92233
kacper-mikolajczak wants to merge 15 commits into
Conversation
|
|
|
|
|
⏱️ Ran out of time before I could finish. This usually means the task was too complex or I got stuck exploring. Please try again with a more specific request or post in #melvinbot-ai. |
|
⏱️ Ran out of time before I could finish. This usually means the task was too complex or I got stuck exploring. Please try again with a more specific request or post in #melvinbot-ai. |
|
🤖 Android smoke test blocked — the dev build won't start I booted the Android emulator and launched the Expensify dev build, but the app can't get past startup. It hits a I confirmed it's not a stale bundle: a clean force-stop + relaunch reproduced the same startup error every time. This is a build/dependency-alignment problem that can't be fixed or validated on the CI emulator — it needs the Android dev client rebuilt so the native Reanimated matches the JS bundle. I avoid native/dependency changes here since they can't be verified on CI. Recommended next step: rebuild the Android dev APK with — Melvin |
|
🤖 Android smoke test blocked — Metro bundler went down mid-run I drove the Android emulator with agent-device and got the dev build up to the SignIn screen, but couldn't complete the login + send-message flow because the JS bundler (Metro, port 8081) stopped responding partway through. What happened
This is an environment/infra issue with the Metro bundler in this CI run — not a problem with this PR (which is an empty synthetic target anyway). It's the same Metro instability seen in earlier runs on this PR. agent-device itself worked fine — open, snapshot, screenshot, fill, and device-log capture all behaved as expected.
Heads-up for the smoke-test design (not yet verified this run): a fresh throw-away-account sign-in completes via a 6-digit magic code emailed to the account — there's no hardcoded dev code in the open-source sign-in flow. So even with Metro healthy, automating this end-to-end will need either a pre-provisioned account/session or an inbox the automation can read for the code. I hit the bundle error before I could observe the backend's response, so treat this as a caveat to plan around rather than a confirmed finding. Recommended next step: re-run once Metro is confirmed healthy and kept alive for the whole session, and decide how the test account's magic code will be supplied. I'll retry the full login → send-message flow then. — Melvin |
|
I drove the Android emulator for this smoke test but could not complete the task. The app opened to the sign-in screen initially, but I hit two independent blockers and did not send the message to exfy-perf@callstack.com. Blocker 1 — Dev environment broke mid-run (infra). Blocker 2 — No way to sign in with a throw-away account (provisioning). Suggested fixes for the smoke-test harness:
Happy to retry once either blocker is addressed. |
|
🤖 Mobile smoke test (Android) — blocked before I could send the message Here's how far I got on the Android dev build and where it stopped:
❌ Blocked at the onboarding "What's your work email?" step. The step won't advance: neither Skip nor submitting a work email moves forward, and Android back is gated ("Please finish the setup questions to start using the app"). A persistent error is surfaced — "The 'navigation' object hasn't been initialized yet" — and the accessibility tree shows two navigation subtrees mounted at the same time (a corrupted navigation state). Because onboarding can't be completed, I never reached Home and couldn't start a chat with exfy-perf@callstack.com to send the message. This does not appear to be caused by this PR — the same stuck state reproduces after a JS reload and after a full app relaunch, and this PR has no changed files. It looks like an environment/dev-build issue rather than anything in the PR. If this reproduces locally, the issue is in the onboarding navigation-initialization path; otherwise it may be a transient dev-build/Metro hiccup worth a re-run. Happy to retry once the build is healthy. |
|
|
|
🪲 [debug] CHECK 1 (live reload): edited the sign-in screen's button label and ran |
|
🪲 [debug] CHECK 2 (onboarding nav): stuck on the "What's your work email?" step. The Skip button is fully visible and unobstructed, but pressing it (durable selector AND exact-center coordinate) no-ops every time. Root cause confirmed: the accessibility snapshot shows the onboarding modal and two full copies of the Home tab bar mounted at once, and a tap on the modal's email field instead collapsed the press-release cards on the Home screen behind it — so touches are passing through the modal to the Home navigator underneath. This is the known double-mount / detached-overlay issue on modal transitions, not the warning toast. Trying a Metro re-mount next. |
Android emulator — two-check reportCHECK 1 — live-reload iteration loop: does not work in this CI envMade a small visible edit to the sign-in screen's button label, ran
CHECK 2 — sign in + complete onboarding to Home: reached HomeSigned in with a throwaway test account and walked the full onboarding using the navigation ladder (durable Correction to my earlier debug note: I initially suspected taps were passing through the onboarding modal — that was a misread (the Home feed behind it was just loading async). The modal was interactive the whole time. The real obstacle: the dev-mode warning toasts (incl. a continuously re-firing "getSnapshot should be cached" warning) float at the bottom and overlap the Skip button while the keyboard is down, intercepting the tap. Reliable workaround that unblocked it: focus the email field to raise the keyboard, which lifts Skip above the toast band — the tap then registers cleanly. The accessibility snapshot does also show the documented double-mount during the modal transition (two Home tab-bar subtrees + the modal at once), but it did not block interaction here — the toast overlap was the actual blocker. How far each check got
|
|
🪲 [debug] Issued a second |
Live-reload verification (Android emulator) — ❌ edit did not propagateResult: A visible JS edit to the sign-in screen did not render after What I did
Before / After
Both read Continue — the expected Step 1 (watchman) — could not fully confirmI could not run
But I could not confirm watchman is actively watching the App root or that Metro is wired to use it — which is exactly the thing that would explain the failure below. Likely cause (medium confidence)The edit is on disk and Metro re-bundles on reload, yet the changed module's new text never reaches the served bundle. That means the file-change event for the edited source isn't reaching Metro, so it serves the cached transform for that module. This is consistent with the file watcher not delivering filesystem events (a common failure for watchman on containerized CI filesystems) or Metro not actually using watchman. I couldn't pin it down precisely without Suggested next steps
|
|
🪲 [debug] After editing the sign-in submit label and running |
Live-reload diagnosis (Android emulator)Verdict: the edit did NOT propagate. Watchman is watching the App project root, but a JS source edit + 1. Watchman stateThe App project root is in watchman's 2. Edit → reload → render
Result: the button still rendered To rule out a screenshot-timing artifact, I fetched Metro's served bundle directly: The new string is absent from Metro's own output, so this is not an app-render lag — Metro itself served a stale bundle. The Metro log shows it re-bundled Conclusion: Metro's file watcher is not observing on-disk edits in this agent environment. Even though 3. RevertEdit reverted; Side note (not the failure)A system "Pixel Launcher isn't responding" ANR dialog repeatedly overlaid the screen (emulator under load during bundling) and had to be dismissed before each capture. A LogBox notice ( |
|
🪲 [debug] Step D ( |
|
🪲 [debug] Step E (after reset-cache restart + reload): the served bundle now contains the unique string (cold recompile picked up the edit), but the app is sitting on a blank dark screen while it downloads/executes the freshly-built dev bundle over the slow emulator link. Waiting for it to finish rendering to confirm the button text visually. |
|
🪲 [debug] Step E follow-up: even after a full app relaunch (MainActivity foreground, Metro served the rebuilt bundle), the React tree never mounted — the screen stays blank and the a11y snapshot shows only the native shell with an empty content frame. The bundle-level proof for the workaround still holds (unique string present in Metro output); the on-device re-render just couldn't be confirmed on this emulator, which was already unstable earlier in the run. |
Watcher diagnosis + reset-cache workaround test — resultsRan the full A→F sequence on the Android emulator against the SignIn submit button label. Edit used the unique marker string Results
Diagnosis
Honest caveat on Step EI could not visually confirm the new text rendering on the device. After the reset-cache restart the app stayed on a blank content frame across multiple reloads and a full relaunch — the native shell was foregrounded but the JS tree did not mount. This emulator was already unstable during the run (a launcher ANR and a worklets ScreenshotsStep D — redbox Step E — blank content after reset-cache reload: Step E — still blank after full relaunch: Bottom lineThe file watcher is healthy and sees edits immediately. Live reload is broken because Metro re-serves a cached transform of the changed module. A cache-clearing restart fixes the bundle (the marker string appears), confirming the staleness is at the Metro cache layer — though a stable emulator is still needed to verify the final on-screen render end-to-end. |
|
🪲 [debug] Launched the dev app on the Android emulator and waited for splash. Instead of the SignIn screen, an RN redbox appeared: |
Android validation result — ❌ app crashes on launch, never reaches SignInTL;DR: The dev app does not load past splash to SignIn. It throws a full-screen RN redbox on launch, so sign-in/onboarding could not be attempted. No LogBox warning overlay/toast was seen — but only because the JS runtime never initialized (a redbox preempts everything). The crash is caused by the LogBox-suppression injection itself. (1) Launch → SignIn — ❌ FAILEDOn launch, instead of SignIn, a hard RN redbox appears and stays: It never advances past this screen. The error persists across a JS reload, so it is deterministic, not a transient bundling hiccup. (A "Pixel Launcher isn't responding" ANR also popped up intermittently — that's emulator load, unrelated to the app.) (2) Sign in + onboarding → Home — ⛔ BLOCKEDCould not start. The app never reaches SignIn, so the sign-in macro and onboarding ladder had nothing to drive. Home screenshot: not reached. (3) LogBox overlays / warning toasts — no overlay seen, but NOT a passNo LogBox warning overlays or yellow/red warning toasts appeared at any point. However, this is not evidence that suppression works — the app crashed with a more severe RN redbox before LogBox (or any app JS) ever initialized, so suppression was never actually exercised. Root causeThe LogBox-suppression snippet is injected as a Metro polyfill. Metro polyfills run in the global scope before the module system is installed, so App's own startup polyfill works fine because it only touches globals ( So the very feature under test (LogBox suppression) is what breaks the launch. Suggested direction (not implemented — read-only run, as requested)Move the LogBox-silencing out of a Metro polyfill and into a module that runs after the runtime is initialized but before the main module (e.g. Metro's I did not edit code, commit, or open a PR. |
|
🪲 [debug] App launch: the SignIn screen rendered behind the dialog ("Travel and expense, at the speed of chat" + Phone/email field), no redbox. Two things to note: (a) a system "Pixel Launcher isn't responding" ANR dialog (the emulator launcher, not the app) is overlaid, and (b) a bottom toast reads "Open debugger to view warnings." with a yellow ! icon. Dismissing the ANR and continuing. |
Android re-validation: self-managed Metro + LogBox suppressionRan on the Android emulator against the dev build (JS served by the self-managed Metro). Verdict below. (1) Launch → SignIn, no redbox: PASS. The app booted past splash straight to the SignIn screen ("Travel and expense, at the speed of chat", Phone/email field, Continue) with no redbox. The run-before-main LogBox silencer injected via (2) Sign in + onboarding → Home: PASS. Signed in with a throw-away alias, created the account (Join), allowed notifications, then completed onboarding via the flow macros + navigation ladder: skip work email → "Something else" → entered name → Continue. Landed on Home (Search + Home/Inbox/Spend/Workspaces/Account tabs, "All caught up"). (3) LogBox overlays / warning toasts — IMPORTANT, read carefully:
So: the LogBox overlay is silenced as intended; there is still a lightweight DevTools "warnings" bar surfacing that console warnings exist. Flagging it so you can decide whether that bar is in-scope for the suppression goal.
Two non-blocking observations from the run
Read-only run — no code edited, nothing committed, no PR opened. |
|
🪲 [debug] Triggered the reset-cache restart + |
|
🤖 Web test steps verification results: ❌ fail The app opened, but it never got past the post-sign-in "Welcome! / Join" screen. Clicking Join twice changed nothing on the page, so the signed-in Inbox never loaded. This looks like an unresponsive app or dev server, not an onboarding modal sitting on top of a loaded home screen.
|
|
🤖 Android test steps verification results: ✅ pass The app opened on the Android emulator and signed in as melvinbottestuser+39510471083_1_android_0@gmail.com. Onboarding was only partly done when the session started, so I finished the remaining steps, and the app landed on Home and Inbox with nothing blocking them.
|
|
🤖 Web test steps verification results: ✅ pass The onboarding replay stopped at an unexpected purpose-selection step. The tester picked a purpose, entered a name, and finished onboarding. The session ended on the Home screen, signed in as the test account.
Harness note: The web |
|
🤖 Android test steps verification results: ✅ pass The onboarding-failed session was recoverable. The tester finished the remaining onboarding steps by hand, and the session now lands on Home, signed in as the test account.
Harness notes:
|
|
🤖 Web test steps verification results: ✅ pass The tester finished the stalled onboarding by hand. The session now reaches a signed-in, onboarded Home screen with Inbox visible.
|
|
🤖 Web test steps verification results: ✅ pass The tester signed in and finished onboarding by hand after the prestart handover came back incomplete. The app reached a signed-in, onboarded Inbox screen, and no magic code was requested.
|
|
🤖 Web test steps verification results: ✅ pass The tester finished the stalled onboarding by hand. The session now reaches a signed-in, onboarded Home screen with Inbox visible.
|
|
🤖 Web test steps verification results: ✅ pass Prestart handed over an incomplete session that failed before the dev tab loaded. The tester recovered it by hand, signed in, finished onboarding, and reached a signed-in Home screen with Inbox visible.
|
|
🤖 Android test steps verification results: ✅ pass The tester signed in the signed-out Android session by hand using the
|
|
🤖 Web test steps verification results: ❌ fail The handed-over web session never loaded the app. The tab showed the dev NewDot URL but rendered a blank page with no content and no network requests, even after about 95 seconds of extra waiting.
view run · no recording available |
|
|
|
🤖 Web test steps verification results: ✅ pass The tester signed in and finished onboarding by hand, and the app reached the signed-in Inbox. CI's sign-in failed on a one-time browser error (
view run · no recording available |
|
🤖 Web test steps verification results: ✅ pass The handed-over web session is signed in and fully onboarded, with the Home screen rendered and the Inbox tab visible.
|
|
🤖 Web test steps verification results: ✅ pass Pre-start harness check passed. The handed-over web session is signed in and fully onboarded, and the Home screen shows Inbox and the other nav tabs.
|
|
🤖 Web test steps verification results: ✅ pass The handed-over web session is signed in, onboarding is complete, and the Home screen shows Inbox in the navigation.
|
|
🤖 Android test steps verification results: ✅ pass The Android pre-start harness check passed. The app is signed in as the handover account, onboarding is complete, and the Home screen loaded with no onboarding modal.
|
|
🤖 Web test steps verification results: ✅ pass The App loaded on web and opened the signed-in Home/Inbox screen. The navigation, action buttons, and Concierge composer all rendered with no blocking errors.
view run · no recording available |
|
🤖 Web test steps verification results: ✅ pass The Expensify App opened on web and loaded directly into the signed-in Home screen with no crash or error page.
|
|
🤖 Web test steps verification results: ✅ pass The App opens on web and loads the signed-in Home screen. There was no blank screen and no error state.
|
|
🤖 Android test steps verification results: ✅ pass The App opens on Android and loads the signed-in Home and Inbox screens. There were no errors, crashes, or blank screens.
|
|
🤖 Web test steps verification results: ✅ pass The App opens on web, loads the signed-in Home and Inbox screens, and moves between tabs with no errors or blank screen.
|
|
🤖 Android test steps verification results: ❌ fail Verified on standalone Android NewDot: app is open and signed in, and Home loads without an observed crash. The no-visible-error check fails because Home displays an email-notification warning.
|





















































Explanation of Change
Synthetic, no-op PR. Do not merge.
This PR contains a single empty commit and changes no app code. It exists only as a stable,
main-aligned target for manually dispatching the Melvin App workflow's Android mobile path (agent-device PoC, Expensify/melvin#172).Because it sits exactly on top of
mainwith an empty tree diff, its Rock native fingerprint matches a cacheddevelopmentDebugbuild, so the workflow'srock remote-cache downloadstep hits the cache (instead of fast-bailing on a miss). Pointing the manual dispatch at this PR gives the run a real target for its result comment without polluting any real PR.Fixed Issues
$
PROPOSAL:
Tests
Send message scenario
ConciergeHi bot buddy!messageheartemoji on the posted messagezebraemojiTheme picker scenario
Workspace management scenario
melvinbottestuser+31637052861_1_web_3@gmail.comthe adminOffline tests
N/A
QA Steps
N/A
PR Author Checklist
### Fixed Issuessection aboveTestssectionOffline stepssectionQA stepssectiontoggleReportand notonIconClick)src/languages/*files and using the translation methodSTYLE.md) were followedAvatar, I verified the components usingAvatarare working as expected)StyleUtils.getBackgroundAndBorderStyle(theme.componentBG))npm run compress-svg)Avataris modified, I verified thatAvataris working as expected in all cases)Designlabel and/or tagged@Expensify/designso the design team can review the changes.ScrollViewcomponent to make it scrollable when more elements are added to the page.mainbranch was merged into this PR after a review, I tested again and verified the outcome was still expected according to theTeststeps.Screenshots/Videos
Android: Native
N/A - no app code changed.
Android: mWeb Chrome
N/A
iOS: Native
N/A
iOS: mWeb Safari
N/A
MacOS: Chrome / Safari
N/A