Assertion planning

Assertion planning is a technique to eliminate tests with false-positive outcomes.

We expect the someOperation method to raise an exception in the following test. However, our test will still be green if no exception is ever raised.

This is a classic case of a false-positive test, as the assertion block is never executed.

test('a false-positive test', async ({ assert }) => {
  try {
    await someOperation()
  } catch (error) {
    assert.equal(error.message, 'The expected error message')
  }
})

To prevent this behavior, you can use assertion planning. In the following example, we tell the assert module to expect exactly one assertion (using the assert.plan(1) method) by the end of the test.

If no assertions were made (meaning, the someOperation method didn't raise an exception), mark the test as failed.

test('a false-positive test', async ({ assert }) => {
  assert.plan(1)

  try {
    await someOperation()
  } catch (error) {
    assert.equal(error.message, 'The expected error message')
  }
})

Usage with @japa/assert

The @japa/assert plugin allows for assertion planning using the assert.plan method.

test('a false-positive test', async ({ assert }) => {
  assert.plan(1)

  try {
    await someOperation()
  } catch (error) {
    assert.equal(error.message, 'The expected error message')
  }
})

If fewer or more assertions are made, then it will make the test fail with the following error message.

Usage with @japa/expect

The @japa/expect plugin uses the Jest expect expect.assertions method.

test('a false-positive test', async ({ expect }) => {
  expect.assertions(1)

  try {
    await someOperation()
  } catch (error) {
    expect(error.message).toEqual('The expected error message')
  }
})

If fewer or more assertions are made, then it will make the test fail with the following error message.