Skip to content

[Due for payment 2026-11-03] Ad-hoc test builds ignore MOBILE_EXPENSIFY_MAIN when the App PR description has a MOBILE-EXPENSIFY entry #104731

Closed

Description

@MariaHCD

Created by Claude on behalf of @MariaHCD.

Problem

When you trigger the Build and deploy apps for testing workflow (testBuild.yml) with MOBILE_EXPENSIFY_MAIN: true, the input is ignored if the App PR description contains a MOBILE-EXPENSIFY: <url> entry. The build uses the head commit of the referenced Mobile-Expensify PR instead of the tip of Mobile-Expensify main.

MOBILE_EXPENSIFY_MAIN is set explicitly by whoever triggers the run, so it should take precedence over a value inferred from the PR description. This was raised in #103625 (comment).

Root cause

testBuild.yml resolves the Mobile-Expensify ref like this:

MOBILE_EXPENSIFY_REF: ${{ needs.resolveRefs.outputs.MOBILE_EXPENSIFY_REF || (inputs.MOBILE_EXPENSIFY_MAIN && 'main' || '') }}

MOBILE_EXPENSIFY_MAIN is only a fallback that applies when resolveRefs returns nothing. resolveBuildRefs.yml does not receive MOBILE_EXPENSIFY_MAIN. It resolves the ref in this order:

  1. The MOBILE_EXPENSIFY_PULL_REQUEST_URL input.
  2. A MOBILE-EXPENSIFY: https://github.com/Expensify/Mobile-Expensify/pull/<n> match in the App PR body.
  3. Empty, which means the pinned submodule commit.

So any MOBILE-EXPENSIFY: line in the PR body silently overrides the checkbox.

Related problem: merged Mobile-Expensify PRs resolve to a stale commit

resolveBuildRefs.yml resolves the PR number to a ref with gh pr view <n> --json headRefOid. For a merged PR, that is the last commit on the PR branch, not the current state of main. People commonly leave a MOBILE-EXPENSIFY: line in the description after the Mobile-Expensify PR merges. Every later ad-hoc build then checks out an outdated Mobile-Expensify commit without saying so. That is the situation the MOBILE_EXPENSIFY_MAIN checkbox was meant to work around.

The PR number also feeds MOBILE_EXPENSIFY_PR, so the "Built from ... Mobile-Expensify PR ..." comment reports the stale PR rather than what the user asked for.

Evidence

From App PR 103625:

  • 2026-10-09: the PR body had MOBILE-EXPENSIFY: https://github.com/Expensify/Mobile-Expensify/pull/14176.
  • 2026-10-09 20:47 UTC: Mobile-Expensify PR 14176 was merged. Head commit 53c8328f, merge commit c3baeb0b.
  • 2026-10-20 to 2026-10-21: four ad-hoc builds ran (39442988450, 39629890401, 39691886868 and one rerun). Android failed in each. Every build comment says "Built from App PR Bump react-native-wallet to 0.1.41 #103625 Mobile-Expensify PR Expensify/Mobile-Expensify#14176".
  • 2026-10-22 06:47 UTC: the reviewer asked for a build with MOBILE_EXPENSIFY_MAIN: true.
  • 2026-10-22 11:24 UTC: run 39770249581 was triggered with MOBILE_EXPENSIFY_MAIN: true. The build comment again says "Mobile-Expensify PR Expensify/Mobile-Expensify#14176", and Android failed again. The checkbox had no effect.
  • 2026-10-22 11:48 UTC: the PR body was edited to Related Mobile-Expensify PR (already merged): ..., which no longer matches the regex. Only after this edit would the checkbox take effect.

The input descriptions don't agree with each other. MOBILE_EXPENSIFY_PULL_REQUEST_URL says it "Overrides MOBILE-EXPENSIFY set in App's PR description". MOBILE_EXPENSIFY_MAIN says nothing about precedence, which suggests it is unconditional.

Proposed solution

  1. Make MOBILE_EXPENSIFY_MAIN override the PR description.
    • Pass MOBILE_EXPENSIFY_MAIN into resolveBuildRefs.yml.
    • When it is true, skip the PR-body scan and return an empty MOBILE_EXPENSIFY_PR, so the build comment doesn't name a PR that wasn't used.
    • The new resolution order: explicit MOBILE_EXPENSIFY_PULL_REQUEST_URL, then MOBILE_EXPENSIFY_MAIN, then the MOBILE-EXPENSIFY: entry in the PR body, then the pinned submodule commit.
    • If MOBILE_EXPENSIFY_PULL_REQUEST_URL and MOBILE_EXPENSIFY_MAIN are both set, fail fast with a clear error instead of silently picking one.
  2. Handle merged Mobile-Expensify PRs in the PR-body scan. If the referenced PR is merged, ignore it and fall back to the next option, with a ::notice:: in the run log. A merged PR's head commit is never a better target than main or the pinned submodule.
  3. Update the input descriptions in testBuild.yml and the header comment in resolveBuildRefs.yml to document the full precedence order.
  4. Make the build comment show the Mobile-Expensify source (custom PR, main, or pinned submodule), so a mismatch shows up in the PR thread.

Option 1 is the core fix. Reordering the expression in testBuild.yml alone is not enough, because MOBILE_EXPENSIFY_PR would still come from the PR body and the build comment would mislabel the build.

Acceptance criteria

  • MOBILE_EXPENSIFY_MAIN: true builds against Mobile-Expensify main even when the App PR body contains MOBILE-EXPENSIFY: <url>.
  • Setting both MOBILE_EXPENSIFY_PULL_REQUEST_URL and MOBILE_EXPENSIFY_MAIN: true fails with a clear error.
  • A MOBILE-EXPENSIFY: entry pointing to a merged PR is not used as the build ref.
  • The build comment correctly reports which Mobile-Expensify source was used.
  • Input descriptions document the precedence order.
Issue OwnerCurrent Issue Owner: @mountiny

Activity

  1. changed the title [-]Ad-hoc test builds ignore MOBILE_EXPENSIFY_MAIN when the App PR description has a MOBILE-EXPENSIFY entry[/-] [+]Ad-hoc test builds ignore `MOBILE_EXPENSIFY_MAIN` when the App PR description has a `MOBILE-EXPENSIFY` entry[/+] on Oct 22, 2026
  2. self-assigned this
    on Oct 22, 2026
  3. added theissue type on Oct 22, 2026
  4. changed the title [-]Ad-hoc test builds ignore `MOBILE_EXPENSIFY_MAIN` when the App PR description has a `MOBILE-EXPENSIFY` entry[/-] [+][Due for payment 2026-11-03] Ad-hoc test builds ignore `MOBILE_EXPENSIFY_MAIN` when the App PR description has a `MOBILE-EXPENSIFY` entry[/+] on Oct 27, 2026
  5. melvin-bot commented on Oct 27, 2026

    @melvin-bot

    @ZhenjaHorbach

    The solution for this issue has been 🚀 deployed to production 🚀 in version 9.5.23-3 and is now subject to a 7-day regression period 📆. Here is the list of pull requests that resolve this issue:

    If no regressions arise, payment will be issued on 2026-11-03. 🎊

    The following checklist (instructions) will need to be completed before the issue can be closed. Please copy/paste the Contributor+ Checklist from here into a new comment on this GH and complete it. If you have the K2 extension, you can simply click: [this button]. If no checklist is needed for this issue, you can click: [no checklist button]

  6. MariaHCD commented on Oct 29, 2026

    @MariaHCD
    ContributorAuthor

    Not overdue

  7. ZhenjaHorbach commented on Oct 30, 2026

    @ZhenjaHorbach
    Contributor

    It's a pipeline issue
    So I think we don't need a checklist here

  8. 5 remaining items

  9. melvin-bot commented on Nov 2, 2026

    @melvin-bot

    Payment Summary

    Resolving PRs:

    BugZero Checklist (@mallenexpensify)

    • I have confirmed assignees, roles, and Upwork contracts look correct
    • I have paid out Upwork contracts / manual NewDot requests
    • [BugZero Assignee] I have created a GH issue for creating/updating the regression test once above steps have been agreed upon
  10. melvin-bot commented on Nov 5, 2026

    @melvin-bot

    @mallenexpensify Uh oh! This issue is overdue by 2 days. Don't forget to update your issues!

  11. mallenexpensify commented on Nov 6, 2026

    @mallenexpensify
    Contributor

    Payment Summary

    Contributor+: @ZhenjaHorbach due $100 via NewDot

    No tests in reviewers checklist and [No QA] so 50%

  12. ZhenjaHorbach commented on Nov 6, 2026

    @ZhenjaHorbach
    Contributor

    Payment Summary

    Contributor+: @ZhenjaHorbach due $100 via NewDot

    No tests in reviewers checklist and [No QA] so 50%

    @mallenexpensify
    I was a contributor here, and it was not quite a trivial issue😅
    So there should be a full price, I hope

  13. mallenexpensify commented on Nov 6, 2026

    @mallenexpensify
    Contributor

    @ZhenjaHorbach this is an edge case I'm not familiar with.
    @mountiny the PR was authored by @ZhenjaHorbach , it's [No QA] and didn't need Screenshots/Videos tests. Do you think this should be 100% payment or 50%? (I'll be using this as an example to add to a rubric.md file I'm building too)

  14. mountiny commented on Nov 7, 2026

    @mountiny
    Contributor

    We can do a full price in this case since it's updating the workflows for deploy. There is nothing to test in the App

  15. added and removed on Nov 8, 2026
  16. mallenexpensify commented on Nov 9, 2026

    @mallenexpensify
    Contributor

    Payment Summary

    Contributor+: @ZhenjaHorbach due $175 via NewDot

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

Awaiting PaymentAuto-added when associated PR is deployed to productionDailyKSv2

Type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions