[Feature]: Add config setting to fail if the app writes to the console.

[coryhouse]

It's a good idea to keep the console clean. So, I'd like to assure that my app never writes to the console when any pages are loaded by Playwright.

I'm doing this right now by calling this util function at the top of each test:

// test-utils.ts
import { Page } from "@playwright/test";

// To keep the console clean, throw an error to fail the test immediately if a console message occurs.
export function throwOnConsole(page: Page) {
  page.on("console", (message) => {
    // Optionally, can check for only errors like this: if (message.type() === "error") {
    throw new Error(message.text());
  });
}

Ideally, I could just configure a setting that does this. The setting could let me specify which types of console logs should fail my tests.

Perhaps like this:

  failOnConsoleLog: ["error", "log", "warning"]

[yury-s]

You can implement this today by either adding console listener in beforeEach and then checking that there were no console messages in afterEach or override page fixture to check the same:

import { test as base } from '@playwright/test';

export const test = base.extend({
  page: async ({ baseURL, page }, use) => {
    const messages = [];
    page.on('console', m => messages.push(m)); 
    await use(page);
    expect(messages).toEqual([]);
  },
});

[evnp]

The problem with the suggested solution here is that you must wait for the entire test to complete (pass or fail) before afterEach is run and your error-detection takes place. This can take a while for complex tests especially if waits and retry logic are involved.

@coryhouse's solution works for us and makes the test fail immediately with useful error output, sometimes saving many minutes of developer time (that would be spent waiting on tests). It would be great to have something like this as a config option though.

[karlhorky]

@yury-s what are your thoughts on the comment by @evnp on the limitations of your suggested approaches?

Would the Playwright team re-consider adding this as an option? (integrated in Playwright core, failing immediately)

This would be super useful to throw errors on things like CSP errors (Content Security Policy) - which currently do not fail the test if an inline <script> or other resource cannot be loaded (even with bypassCSP: false)

[karlhorky]

Another pattern that seems like it would also be good to have a configuration pattern for would be "failing on error", which can be achieved with this:

page.on('pageerror', (error) => {
  throw error;
});

References:

• [BUG] throwing from pageerror callback leads to unpredictable behavior #28056

Would also be great to have this be added as a Playwright configuration option like failOnPageError or failOnUnhandledException, also commented over here:

• [Question] Fail on unhandled exception? #16355 (comment)

[corneliusroemer]

I think one needs to modify the import to add type after import to @coryhouse's solution like so:

import type { Page } from '@playwright/test';

export function throwOnConsole(page: Page) {
    page.on('console', (message) => {
        throw new Error(message.text());
    });
}

[nboisteault]

So this issue should be reopened as there is no config settings today?