Skip to content

Find way to make flaky tests less flaky on CI #3587

Description

@etpinard

Since about a week ago, the test-jasmine CI test container has been failing roughly (my estimate) 1 out of two runs. Of those failure most are due to

retry npm run test-jasmine -- --tags=flaky --skip-tags=noCI

We should try to find a solution.

Potential solutions

  1. split off that
retry npm run test-jasmine -- --tags=flaky --skip-tags=noCI

into a new test container (named test-jasmine3)

  1. use shard_jasmine_tests.js util as in our @gl tests:

SHARDS=($(node $ROOT/tasks/shard_jasmine_tests.js --tag=gl))
for s in ${SHARDS[@]}; do
retry npm run test-jasmine -- "$s" --tags=gl --skip-tags=noCI
done

to split off the @flaky tests into multiple runs.

  1. combine 1) and 2)

@antoinerg @archmoj wanna give this a shot?

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions