[Bug]: snapshotPathTemplate doesnt work when multiple projects

[debs-obrien]

Version

1.51.0-alpha-2025-01-22

Steps to reproduce

1. add this to your config:

snapshotPathTemplate: '__snapshots__/{testFilePath}/{arg}{ext}',

2. Make sure chromium, firefox and webkit are selected in vscode sidebar under projects
3. create the following test:

test('test with name', async ({ page }) => {  
  await page.goto('https://debs-obrien.github.io/playwright-movies-app/movie?id=533535&page=1');
  await expect(page.getByRole('main')).toMatchAriaSnapshot({ name: 'movie.yml'});
});

5. run test using the play button in vscode extension
6. run test again

Expected behavior

all tests should pass once the snapshot is created. it is an aria snapshot and therefore should work across browsers

Actual behavior

firefox passes, chromium and webkit fail

Additional context

Environment

System:
    OS: macOS 15.2
    CPU: (10) arm64 Apple M1 Max
    Memory: 236.09 MB / 64.00 GB
  Binaries:
    Node: 20.11.1 - /usr/local/bin/node
    Yarn: 1.22.19 - /usr/local/bin/yarn
    npm: 10.2.4 - /usr/local/bin/npm
    pnpm: 8.5.0 - /usr/local/bin/pnpm
  IDEs:
    VSCode: 1.96.4 - /usr/local/bin/code
  Languages:
    Bash: 3.2.57 - /bin/bash
  npmPackages:
    @playwright/test: ^1.51.0-alpha-2025-01-22 => 1.51.0-alpha-2025-01-22

[pavelfeldman]

Investigation note: when capturing full snapshots, they are different for Firefox. Looks like pseudo element visibiltiy is different, which results in different snapshots:

Running on Firefox against Chromium baseline

[pavelfeldman]

Reduced test case:

test('preudo attr', async ({ page }) => {
  await page.setContent(`
    <style>
    .stars:before { display: block; content: attr(data-forhalf); }
    </style>
    <div class="stars" data-forhalf="★">★</div>
  `);
  await expect(page.locator('body')).toMatchAriaSnapshot(`- text: ★ ★`);
});

content: attr is essential