Fix flaky playwright tests #4003
Annotations
6 errors, 3 warnings, and 1 notice
Run Playwright tests:
playwright/e2e/share-dialog/share-dialog.spec.ts#L60
1) [Chrome] › share-dialog/share-dialog.spec.ts:49:9 › Share dialog › should share an event @screenshot
Error: expect(locator).toHaveScreenshot(expected)
265 pixels (ratio 0.01 of all image pixels) are different.
Expected: /home/runner/work/element-web/element-web/playwright/snapshots/share-dialog/share-dialog.spec.ts/share-dialog-event-linux.png
Received: /home/runner/work/element-web/element-web/playwright/test-results/share-dialog-share-dialog-Share-dialog-should-share-an-event-Chrome/share-dialog-event-actual.png
Diff: /home/runner/work/element-web/element-web/playwright/test-results/share-dialog-share-dialog-Share-dialog-should-share-an-event-Chrome/share-dialog-event-diff.png
Call log:
- expect.toHaveScreenshot(share-dialog-event.png) with timeout 5000ms
- verifying given screenshot expectation
- waiting for getByRole('dialog', { name: 'Share Room Message' })
- locator resolved to <div role="dialog" class="mx_ShareDialog" data-focus-lock-disabled="false" aria-describedby="mx_Dialog_content" aria-labelledby="mx_BaseDialog_title">…</div>
- taking element screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- attempting scroll into view action
- waiting for element to be stable
- 265 pixels (ratio 0.01 of all image pixels) are different.
- waiting 100ms before taking screenshot
- waiting for getByRole('dialog', { name: 'Share Room Message' })
- locator resolved to <div role="dialog" class="mx_ShareDialog" data-focus-lock-disabled="false" aria-describedby="mx_Dialog_content" aria-labelledby="mx_BaseDialog_title">…</div>
- taking element screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- attempting scroll into view action
- waiting for element to be stable
- captured a stable screenshot
- 265 pixels (ratio 0.01 of all image pixels) are different.
58 | const dialog = page.getByRole("dialog", { name: "Share Room Message" });
59 | await expect(dialog.getByRole("checkbox", { name: "Link to selected message" })).toBeChecked();
> 60 | await expect(dialog).toMatchScreenshot("share-dialog-event.png", {
| ^
61 | // QRCode and url changes at every run
62 | mask: [page.locator(".mx_QRCode"), page.locator(".mx_ShareDialog_top > span")],
63 | });
at /home/runner/work/element-web/element-web/playwright/e2e/share-dialog/share-dialog.spec.ts:60:30
|
Run Playwright tests:
playwright/e2e/share-dialog/share-dialog.spec.ts#L60
1) [Chrome] › share-dialog/share-dialog.spec.ts:49:9 › Share dialog › should share an event @screenshot
Retry #1 ───────────────────────────────────────────────────────────────────────────────────────
Error: expect(locator).toHaveScreenshot(expected)
265 pixels (ratio 0.01 of all image pixels) are different.
Expected: /home/runner/work/element-web/element-web/playwright/snapshots/share-dialog/share-dialog.spec.ts/share-dialog-event-linux.png
Received: /home/runner/work/element-web/element-web/playwright/test-results/share-dialog-share-dialog-Share-dialog-should-share-an-event-Chrome-retry1/share-dialog-event-actual.png
Diff: /home/runner/work/element-web/element-web/playwright/test-results/share-dialog-share-dialog-Share-dialog-should-share-an-event-Chrome-retry1/share-dialog-event-diff.png
Call log:
- expect.toHaveScreenshot(share-dialog-event.png) with timeout 5000ms
- verifying given screenshot expectation
- waiting for getByRole('dialog', { name: 'Share Room Message' })
- locator resolved to <div role="dialog" class="mx_ShareDialog" data-focus-lock-disabled="false" aria-describedby="mx_Dialog_content" aria-labelledby="mx_BaseDialog_title">…</div>
- taking element screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- attempting scroll into view action
- waiting for element to be stable
- 265 pixels (ratio 0.01 of all image pixels) are different.
- waiting 100ms before taking screenshot
- waiting for getByRole('dialog', { name: 'Share Room Message' })
- locator resolved to <div role="dialog" class="mx_ShareDialog" data-focus-lock-disabled="false" aria-describedby="mx_Dialog_content" aria-labelledby="mx_BaseDialog_title">…</div>
- taking element screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- attempting scroll into view action
- waiting for element to be stable
- captured a stable screenshot
- 265 pixels (ratio 0.01 of all image pixels) are different.
58 | const dialog = page.getByRole("dialog", { name: "Share Room Message" });
59 | await expect(dialog.getByRole("checkbox", { name: "Link to selected message" })).toBeChecked();
> 60 | await expect(dialog).toMatchScreenshot("share-dialog-event.png", {
| ^
61 | // QRCode and url changes at every run
62 | mask: [page.locator(".mx_QRCode"), page.locator(".mx_ShareDialog_top > span")],
63 | });
at /home/runner/work/element-web/element-web/playwright/e2e/share-dialog/share-dialog.spec.ts:60:30
|
Run Playwright tests:
playwright/e2e/share-dialog/share-dialog.spec.ts#L60
1) [Chrome] › share-dialog/share-dialog.spec.ts:49:9 › Share dialog › should share an event @screenshot
Retry #2 ───────────────────────────────────────────────────────────────────────────────────────
Error: expect(locator).toHaveScreenshot(expected)
265 pixels (ratio 0.01 of all image pixels) are different.
Expected: /home/runner/work/element-web/element-web/playwright/snapshots/share-dialog/share-dialog.spec.ts/share-dialog-event-linux.png
Received: /home/runner/work/element-web/element-web/playwright/test-results/share-dialog-share-dialog-Share-dialog-should-share-an-event-Chrome-retry2/share-dialog-event-actual.png
Diff: /home/runner/work/element-web/element-web/playwright/test-results/share-dialog-share-dialog-Share-dialog-should-share-an-event-Chrome-retry2/share-dialog-event-diff.png
Call log:
- expect.toHaveScreenshot(share-dialog-event.png) with timeout 5000ms
- verifying given screenshot expectation
- waiting for getByRole('dialog', { name: 'Share Room Message' })
- locator resolved to <div role="dialog" class="mx_ShareDialog" data-focus-lock-disabled="false" aria-describedby="mx_Dialog_content" aria-labelledby="mx_BaseDialog_title">…</div>
- taking element screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- attempting scroll into view action
- waiting for element to be stable
- 265 pixels (ratio 0.01 of all image pixels) are different.
- waiting 100ms before taking screenshot
- waiting for getByRole('dialog', { name: 'Share Room Message' })
- locator resolved to <div role="dialog" class="mx_ShareDialog" data-focus-lock-disabled="false" aria-describedby="mx_Dialog_content" aria-labelledby="mx_BaseDialog_title">…</div>
- taking element screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- attempting scroll into view action
- waiting for element to be stable
- captured a stable screenshot
- 265 pixels (ratio 0.01 of all image pixels) are different.
58 | const dialog = page.getByRole("dialog", { name: "Share Room Message" });
59 | await expect(dialog.getByRole("checkbox", { name: "Link to selected message" })).toBeChecked();
> 60 | await expect(dialog).toMatchScreenshot("share-dialog-event.png", {
| ^
61 | // QRCode and url changes at every run
62 | mask: [page.locator(".mx_QRCode"), page.locator(".mx_ShareDialog_top > span")],
63 | });
at /home/runner/work/element-web/element-web/playwright/e2e/share-dialog/share-dialog.spec.ts:60:30
|
Run Playwright tests:
playwright/e2e/room_options/marked_unread.spec.ts#L51
2) [Chrome] › room_options/marked_unread.spec.ts:22:9 › Mark as Unread › should mark a room as unread
Error: expect(locator).toBeVisible()
Locator: getByLabel('The mark unread test room Unread messages.')
Expected: visible
Received: <element(s) not found>
Call log:
- expect.toBeVisible with timeout 5000ms
- waiting for getByLabel('The mark unread test room Unread messages.')
49 | await page.getByRole("menuitem", { name: "Mark as unread" }).click();
50 |
> 51 | expect(page.getByLabel(TEST_ROOM_NAME + " Unread messages.")).toBeVisible();
| ^
52 | });
53 | });
54 |
at /home/runner/work/element-web/element-web/playwright/e2e/room_options/marked_unread.spec.ts:51:71
|
Run Playwright tests:
playwright/e2e/room-directory/room-directory.spec.ts#L34
3) [Chrome] › room-directory/room-directory.spec.ts:18:9 › Room Directory › should allow admin to add alias & publish room to directory @no-webkit
Error: Timed out 5000ms waiting for expect(locator).toHaveClass(expected)
Locator: locator('.mx_SettingsFieldset').filter({ hasText: 'Local Addresses' }).getByText('#gaming:localhost')
Expected string: "mx_EditableItem_item"
Received: <element(s) not found>
Call log:
- expect.toHaveClass with timeout 5000ms
- waiting for locator('.mx_SettingsFieldset').filter({ hasText: 'Local Addresses' }).getByText('#gaming:localhost')
32 | await localAddresses.getByRole("textbox").fill("gaming");
33 | await localAddresses.getByRole("button", { name: "Add" }).click();
> 34 | await expect(localAddresses.getByText("#gaming:localhost")).toHaveClass("mx_EditableItem_item");
| ^
35 |
36 | // Publish into the public rooms directory
37 | const publishedAddresses = page.locator(".mx_SettingsFieldset", { hasText: "Published Addresses" });
at /home/runner/work/element-web/element-web/playwright/e2e/room-directory/room-directory.spec.ts:34:73
|
Run Playwright tests
Process completed with exit code 1.
|
Run Playwright tests:
[Chrome] › sliding-sync/sliding-sync.spec.ts#L1
[Chrome] › sliding-sync/sliding-sync.spec.ts took 57.7s
|
Run Playwright tests:
[Chrome] › room/room-header.spec.ts#L1
[Chrome] › room/room-header.spec.ts took 24.7s
|
Run Playwright tests:
[Chrome] › settings/account-user-settings-tab.spec.ts#L1
[Chrome] › settings/account-user-settings-tab.spec.ts took 17.5s
|
Run Playwright tests
1 failed
[Chrome] › share-dialog/share-dialog.spec.ts:49:9 › Share dialog › should share an event @screenshot
2 flaky
[Chrome] › room_options/marked_unread.spec.ts:22:9 › Mark as Unread › should mark a room as unread
[Chrome] › room-directory/room-directory.spec.ts:18:9 › Room Directory › should allow admin to add alias & publish room to directory @no-webkit
1 skipped
46 passed (4.1m)
|
Loading