Skip to content

Investigate workflow job failing on main: e2ePerformanceTests / Find the baseline and delta refs, and check for an existing build artifact for that commit #54478

Closed

Description

@github-actions

🚨 Failure Summary 🚨:

⚠️ Action Required ⚠️:

🛠️ A recent merge appears to have caused a failure in the job named e2ePerformanceTests / Find the baseline and delta refs, and check for an existing build artifact for that commit.
This issue has been automatically created and labeled with Workflow Failure for investigation.

👀 Please look into the following:

  1. Why the PR caused the job to fail?
  2. Address any underlying issues.

🐛 We appreciate your help in squashing this bug!

Activity

  1. marcaaron commented on Dec 23, 2024

    @marcaaron
    Contributor

    Looks like some kind of Http error on GitHubs end possibly

  2. hannojg commented on Dec 30, 2024

    @hannojg
    Contributor

    hm, for some reason when that PR got merged the job tried to look for a PR with your user instead of using the PR number. Maybe there is something broken about that logic (I think the PR number should've been used)

  3. hannojg commented on Dec 30, 2024

    @hannojg
    Contributor

    While the above error might have been a GitHub hiccup, it seems that the e2e tests are still broken though.
    All runs are currently failing after ~ 1h45m on AWS.
    Checking whats wrong with that …

  4. hannojg commented on Dec 30, 2024

    @hannojg
    Contributor

    @perunt it seems that the tests are failing due to this error in the linking test:

    CleanShot 2024-12-30 at 11 59 26

    @kirillzyusko I know that we have a mechanism for marking a test as failed in a way that it won't fail the whole pipeline, but just the faulty test. Can you please let us know what the API for that is? Then @perunt can open a PR to fix that not the whole pipeline fails.
    After that we should probably investigate why that error is happening though.

  5. hannojg commented on Dec 30, 2024

    @hannojg
    Contributor
  6. kirillzyusko commented on Dec 30, 2024

    @kirillzyusko
    Contributor

    Yes, we have a fault tolerance mechanism, but it's used to safely handle flaky tests I believe.

    But this test explicitly fails with error:

    E2EClient.submitTestResults({
        branch: Config.E2E_BRANCH,
        error: 'Linked message not found',
        name: `${name} test can't find linked message`,
    }).then(() => E2EClient.submitTestDone());

    And for such cases we don't have safe failure handling, because they are clearly indicate that it's something wrong with the test itself (i. e. it's not flakiness) 🤷‍♂️

    And it forces us to investigate a reason instead of silently ignoring them 👀

  7. melvin-bot commented on Jan 23, 2025

    @melvin-bot

    This issue has not been updated in over 15 days. @hannojg eroding to Monthly issue.

    P.S. Is everyone reading this sure this is really a near-term priority? Be brave: if you disagree, go ahead and close it out. If someone disagrees, they'll reopen it, and if they don't: one less thing to do!

  8. hannojg commented on Feb 17, 2025

    @hannojg
    Contributor

    This is fixed by now, can we please close this @mountiny (I don't have the permission for some reason)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions