GoalBe able to perform multiple retries of an entire test suite until all tests succeed.
Instead of configuring retries allowed for every test within a test suite, as defined by Configure Test Retries: Global Configuration
How to provide test suite retries?
If any test fails within a test suite, then start over and retry the entire test suite again until all tests within a test suite succeed or stop when the entire test suite retry attempts exceed the requested retry attempts.
In the following code, configures retry attempts applicable for every test within a test suite.
The desired goal for this test suite, test 'TEST counter' is expected to fail until incrementing _counter equals _runModeRetries and only then is it expected to succeed. Repeat the entire test suite means re-run every test prior to test 'TEST counter' until it succeeds.
However, what happens is that only test 'TEST counter' is retried _runModeRetries times, and _counter is not incremented because test 'TEST increment' is called only once.
Why do I want this?
I have test suites that have a series of tests the required to run in sequence, and if a test fails, then retries require restarting the sequence. For example, _counter can only be incremented if test 'TEST increment' is called again with a full test suite retry.
How can I do test suite retries?
let _counter = 0; const _runModeRetries = 3; context('CONTEXT Cypress Retries', { retries: { runMode: _runModeRetries, openMode: 0 } }, () => { it('TEST increment', () => { _counter++; expect(_counter).to.be.a('number').gt(0); }); it('TEST true', () => { expect(true).to.be.a('boolean').to.be.true; }); it('TEST false', () => { expect(false).to.be.a('boolean').to.be.false; }); it('TEST counter', () => { if (_counter < _runModeRetries) { assert.fail(); } else { assert.isTrue(true); } }); } );