Skip to content

Fix flaky playwright tests #4003

Fix flaky playwright tests

Fix flaky playwright tests #4003

Triggered via pull request January 10, 2025 18:20
Status Failure
Total duration 11m 40s
Artifacts 8

end-to-end-tests.yaml

on: pull_request
Build Element-Web
1m 59s
Build Element-Web
Matrix: playwright
end-to-end-tests
48s
end-to-end-tests
Fit to window
Zoom out
Zoom in

Annotations

31 errors, 25 warnings, and 6 notices
[Chrome] › right-panel/right-panel.spec.ts:41:13 › RightPanel › in rooms › should handle long room address and long room name @screenshot: playwright/e2e/right-panel/right-panel.spec.ts#L51
1) [Chrome] › right-panel/right-panel.spec.ts:41:13 › RightPanel › in rooms › should handle long room address and long room name @screenshot Error: Timed out 5000ms waiting for expect(locator).toHaveClass(expected) Locator: locator('.mx_SettingsFieldset').filter({ hasText: 'Local Addresses' }).getByText('#loremIpsumDolorSitAmetConsecteturAdipisicingElitSedDoEiusmodTemporIncididuntUtLaboreEtDoloreMagnaAliqua:localhost') Expected string: "mx_EditableItem_item" Received: <element(s) not found> Call log: - expect.toHaveClass with timeout 5000ms - waiting for locator('.mx_SettingsFieldset').filter({ hasText: 'Local Addresses' }).getByText('#loremIpsumDolorSitAmetConsecteturAdipisicingElitSedDoEiusmodTemporIncididuntUtLaboreEtDoloreMagnaAliqua:localhost') 49 | await localAddresses.getByRole("textbox").fill(ROOM_ADDRESS_LONG); 50 | await localAddresses.getByRole("button", { name: "Add" }).click(); > 51 | await expect(localAddresses.getByText(`#${ROOM_ADDRESS_LONG}:localhost`)).toHaveClass( | ^ 52 | "mx_EditableItem_item", 53 | ); 54 | at /home/runner/work/element-web/element-web/playwright/e2e/right-panel/right-panel.spec.ts:51:87
[Chrome] › right-panel/right-panel.spec.ts:41:13 › RightPanel › in rooms › should handle long room address and long room name @screenshot: playwright/e2e/right-panel/right-panel.spec.ts#L51
1) [Chrome] › right-panel/right-panel.spec.ts:41:13 › RightPanel › in rooms › should handle long room address and long room name @screenshot Retry #1 ─────────────────────────────────────────────────────────────────────────────────────── Error: Timed out 5000ms waiting for expect(locator).toHaveClass(expected) Locator: locator('.mx_SettingsFieldset').filter({ hasText: 'Local Addresses' }).getByText('#loremIpsumDolorSitAmetConsecteturAdipisicingElitSedDoEiusmodTemporIncididuntUtLaboreEtDoloreMagnaAliqua:localhost') Expected string: "mx_EditableItem_item" Received: <element(s) not found> Call log: - expect.toHaveClass with timeout 5000ms - waiting for locator('.mx_SettingsFieldset').filter({ hasText: 'Local Addresses' }).getByText('#loremIpsumDolorSitAmetConsecteturAdipisicingElitSedDoEiusmodTemporIncididuntUtLaboreEtDoloreMagnaAliqua:localhost') 49 | await localAddresses.getByRole("textbox").fill(ROOM_ADDRESS_LONG); 50 | await localAddresses.getByRole("button", { name: "Add" }).click(); > 51 | await expect(localAddresses.getByText(`#${ROOM_ADDRESS_LONG}:localhost`)).toHaveClass( | ^ 52 | "mx_EditableItem_item", 53 | ); 54 | at /home/runner/work/element-web/element-web/playwright/e2e/right-panel/right-panel.spec.ts:51:87
[Chrome] › share-dialog/share-dialog.spec.ts:49:9 › Share dialog › should share an event @screenshot: playwright/e2e/share-dialog/share-dialog.spec.ts#L60
1) [Chrome] › share-dialog/share-dialog.spec.ts:49:9 › Share dialog › should share an event @screenshot Error: expect(locator).toHaveScreenshot(expected) 265 pixels (ratio 0.01 of all image pixels) are different. Expected: /home/runner/work/element-web/element-web/playwright/snapshots/share-dialog/share-dialog.spec.ts/share-dialog-event-linux.png Received: /home/runner/work/element-web/element-web/playwright/test-results/share-dialog-share-dialog-Share-dialog-should-share-an-event-Chrome/share-dialog-event-actual.png Diff: /home/runner/work/element-web/element-web/playwright/test-results/share-dialog-share-dialog-Share-dialog-should-share-an-event-Chrome/share-dialog-event-diff.png Call log: - expect.toHaveScreenshot(share-dialog-event.png) with timeout 5000ms - verifying given screenshot expectation - waiting for getByRole('dialog', { name: 'Share Room Message' }) - locator resolved to <div role="dialog" class="mx_ShareDialog" data-focus-lock-disabled="false" aria-describedby="mx_Dialog_content" aria-labelledby="mx_BaseDialog_title">…</div> - taking element screenshot - disabled all CSS animations - waiting for fonts to load... - fonts loaded - attempting scroll into view action - waiting for element to be stable - 265 pixels (ratio 0.01 of all image pixels) are different. - waiting 100ms before taking screenshot - waiting for getByRole('dialog', { name: 'Share Room Message' }) - locator resolved to <div role="dialog" class="mx_ShareDialog" data-focus-lock-disabled="false" aria-describedby="mx_Dialog_content" aria-labelledby="mx_BaseDialog_title">…</div> - taking element screenshot - disabled all CSS animations - waiting for fonts to load... - fonts loaded - attempting scroll into view action - waiting for element to be stable - captured a stable screenshot - 265 pixels (ratio 0.01 of all image pixels) are different. 58 | const dialog = page.getByRole("dialog", { name: "Share Room Message" }); 59 | await expect(dialog.getByRole("checkbox", { name: "Link to selected message" })).toBeChecked(); > 60 | await expect(dialog).toMatchScreenshot("share-dialog-event.png", { | ^ 61 | // QRCode and url changes at every run 62 | mask: [page.locator(".mx_QRCode"), page.locator(".mx_ShareDialog_top > span")], 63 | }); at /home/runner/work/element-web/element-web/playwright/e2e/share-dialog/share-dialog.spec.ts:60:30
[Chrome] › share-dialog/share-dialog.spec.ts:49:9 › Share dialog › should share an event @screenshot: playwright/e2e/share-dialog/share-dialog.spec.ts#L60
1) [Chrome] › share-dialog/share-dialog.spec.ts:49:9 › Share dialog › should share an event @screenshot Retry #1 ─────────────────────────────────────────────────────────────────────────────────────── Error: expect(locator).toHaveScreenshot(expected) 265 pixels (ratio 0.01 of all image pixels) are different. Expected: /home/runner/work/element-web/element-web/playwright/snapshots/share-dialog/share-dialog.spec.ts/share-dialog-event-linux.png Received: /home/runner/work/element-web/element-web/playwright/test-results/share-dialog-share-dialog-Share-dialog-should-share-an-event-Chrome-retry1/share-dialog-event-actual.png Diff: /home/runner/work/element-web/element-web/playwright/test-results/share-dialog-share-dialog-Share-dialog-should-share-an-event-Chrome-retry1/share-dialog-event-diff.png Call log: - expect.toHaveScreenshot(share-dialog-event.png) with timeout 5000ms - verifying given screenshot expectation - waiting for getByRole('dialog', { name: 'Share Room Message' }) - locator resolved to <div role="dialog" class="mx_ShareDialog" data-focus-lock-disabled="false" aria-describedby="mx_Dialog_content" aria-labelledby="mx_BaseDialog_title">…</div> - taking element screenshot - disabled all CSS animations - waiting for fonts to load... - fonts loaded - attempting scroll into view action - waiting for element to be stable - 265 pixels (ratio 0.01 of all image pixels) are different. - waiting 100ms before taking screenshot - waiting for getByRole('dialog', { name: 'Share Room Message' }) - locator resolved to <div role="dialog" class="mx_ShareDialog" data-focus-lock-disabled="false" aria-describedby="mx_Dialog_content" aria-labelledby="mx_BaseDialog_title">…</div> - taking element screenshot - disabled all CSS animations - waiting for fonts to load... - fonts loaded - attempting scroll into view action - waiting for element to be stable - captured a stable screenshot - 265 pixels (ratio 0.01 of all image pixels) are different. 58 | const dialog = page.getByRole("dialog", { name: "Share Room Message" }); 59 | await expect(dialog.getByRole("checkbox", { name: "Link to selected message" })).toBeChecked(); > 60 | await expect(dialog).toMatchScreenshot("share-dialog-event.png", { | ^ 61 | // QRCode and url changes at every run 62 | mask: [page.locator(".mx_QRCode"), page.locator(".mx_ShareDialog_top > span")], 63 | }); at /home/runner/work/element-web/element-web/playwright/e2e/share-dialog/share-dialog.spec.ts:60:30
[Chrome] › share-dialog/share-dialog.spec.ts:49:9 › Share dialog › should share an event @screenshot: playwright/e2e/share-dialog/share-dialog.spec.ts#L60
1) [Chrome] › share-dialog/share-dialog.spec.ts:49:9 › Share dialog › should share an event @screenshot Retry #2 ─────────────────────────────────────────────────────────────────────────────────────── Error: expect(locator).toHaveScreenshot(expected) 265 pixels (ratio 0.01 of all image pixels) are different. Expected: /home/runner/work/element-web/element-web/playwright/snapshots/share-dialog/share-dialog.spec.ts/share-dialog-event-linux.png Received: /home/runner/work/element-web/element-web/playwright/test-results/share-dialog-share-dialog-Share-dialog-should-share-an-event-Chrome-retry2/share-dialog-event-actual.png Diff: /home/runner/work/element-web/element-web/playwright/test-results/share-dialog-share-dialog-Share-dialog-should-share-an-event-Chrome-retry2/share-dialog-event-diff.png Call log: - expect.toHaveScreenshot(share-dialog-event.png) with timeout 5000ms - verifying given screenshot expectation - waiting for getByRole('dialog', { name: 'Share Room Message' }) - locator resolved to <div role="dialog" class="mx_ShareDialog" data-focus-lock-disabled="false" aria-describedby="mx_Dialog_content" aria-labelledby="mx_BaseDialog_title">…</div> - taking element screenshot - disabled all CSS animations - waiting for fonts to load... - fonts loaded - attempting scroll into view action - waiting for element to be stable - 265 pixels (ratio 0.01 of all image pixels) are different. - waiting 100ms before taking screenshot - waiting for getByRole('dialog', { name: 'Share Room Message' }) - locator resolved to <div role="dialog" class="mx_ShareDialog" data-focus-lock-disabled="false" aria-describedby="mx_Dialog_content" aria-labelledby="mx_BaseDialog_title">…</div> - taking element screenshot - disabled all CSS animations - waiting for fonts to load... - fonts loaded - attempting scroll into view action - waiting for element to be stable - captured a stable screenshot - 265 pixels (ratio 0.01 of all image pixels) are different. 58 | const dialog = page.getByRole("dialog", { name: "Share Room Message" }); 59 | await expect(dialog.getByRole("checkbox", { name: "Link to selected message" })).toBeChecked(); > 60 | await expect(dialog).toMatchScreenshot("share-dialog-event.png", { | ^ 61 | // QRCode and url changes at every run 62 | mask: [page.locator(".mx_QRCode"), page.locator(".mx_ShareDialog_top > span")], 63 | }); at /home/runner/work/element-web/element-web/playwright/e2e/share-dialog/share-dialog.spec.ts:60:30
[Chrome] › room_options/marked_unread.spec.ts:22:9 › Mark as Unread › should mark a room as unread: playwright/e2e/room_options/marked_unread.spec.ts#L51
2) [Chrome] › room_options/marked_unread.spec.ts:22:9 › Mark as Unread › should mark a room as unread Error: expect(locator).toBeVisible() Locator: getByLabel('The mark unread test room Unread messages.') Expected: visible Received: <element(s) not found> Call log: - expect.toBeVisible with timeout 5000ms - waiting for getByLabel('The mark unread test room Unread messages.') 49 | await page.getByRole("menuitem", { name: "Mark as unread" }).click(); 50 | > 51 | expect(page.getByLabel(TEST_ROOM_NAME + " Unread messages.")).toBeVisible(); | ^ 52 | }); 53 | }); 54 | at /home/runner/work/element-web/element-web/playwright/e2e/room_options/marked_unread.spec.ts:51:71
[Chrome] › room-directory/room-directory.spec.ts:18:9 › Room Directory › should allow admin to add alias & publish room to directory @no-webkit: playwright/e2e/room-directory/room-directory.spec.ts#L34
3) [Chrome] › room-directory/room-directory.spec.ts:18:9 › Room Directory › should allow admin to add alias & publish room to directory @no-webkit Error: Timed out 5000ms waiting for expect(locator).toHaveClass(expected) Locator: locator('.mx_SettingsFieldset').filter({ hasText: 'Local Addresses' }).getByText('#gaming:localhost') Expected string: "mx_EditableItem_item" Received: <element(s) not found> Call log: - expect.toHaveClass with timeout 5000ms - waiting for locator('.mx_SettingsFieldset').filter({ hasText: 'Local Addresses' }).getByText('#gaming:localhost') 32 | await localAddresses.getByRole("textbox").fill("gaming"); 33 | await localAddresses.getByRole("button", { name: "Add" }).click(); > 34 | await expect(localAddresses.getByText("#gaming:localhost")).toHaveClass("mx_EditableItem_item"); | ^ 35 | 36 | // Publish into the public rooms directory 37 | const publishedAddresses = page.locator(".mx_SettingsFieldset", { hasText: "Published Addresses" }); at /home/runner/work/element-web/element-web/playwright/e2e/room-directory/room-directory.spec.ts:34:73
Run Tests [Chrome] 4/6
Process completed with exit code 1.
[Chrome] › knock/create-knock-room.spec.ts:61:9 › Create Knock Room › should create a public knock room: playwright/e2e/knock/create-knock-room.spec.ts#L82
1) [Chrome] › knock/create-knock-room.spec.ts:61:9 › Create Knock Room › should create a public knock room Error: Timed out 5000ms waiting for expect(locator).toContainText(expected) Locator: locator('[role=dialog][aria-label="Search Dialog"]').locator('.mx_SpotlightDialog_section.mx_SpotlightDialog_results .mx_SpotlightDialog_option').first() Expected string: "Cybersecurity" Received string: "TTestRoom!jPQUevSsVsmXeEyMET:localhost2 Members↵Join" Call log: - expect.toContainText with timeout 5000ms - waiting for locator('[role=dialog][aria-label="Search Dialog"]').locator('.mx_SpotlightDialog_section.mx_SpotlightDialog_results .mx_SpotlightDialog_option').first() 7 × locator resolved to <li tabindex="-1" role="option" aria-selected="true" id="mx_SpotlightDialog_button_result_!jPQUevSsVsmXeEyMET:localhost" aria-labelledby="mx_SpotlightDialog_button_result_!jPQUevSsVsmXeEyMET:localhost_name" aria-details="mx_SpotlightDialog_button_result_!jPQUevSsVsmXeEyMET:localhost_details" aria-describedby="mx_SpotlightDialog_button_result_!jPQUevSsVsmXeEyMET:localhost_alias" class="mx_AccessibleButton mx_SpotlightDialog_result_multiline mx_SpotlightDialog_option">…</li> - unexpected value "TTestRoom!jPQUevSsVsmXeEyMET:localhost2 Members↵Join" 80 | const spotlightDialog = await app.openSpotlight(); 81 | await spotlightDialog.filter(Filter.PublicRooms); > 82 | await expect(spotlightDialog.results.nth(0)).toContainText("Cybersecurity"); | ^ 83 | }); 84 | }); 85 | at /home/runner/work/element-web/element-web/playwright/e2e/knock/create-knock-room.spec.ts:82:54
[Chrome] › messages/messages.spec.ts:95:17 › Message rendering › with rtl display name › should render an LTR rich text emote @screenshot: playwright/e2e/messages/messages.spec.ts#L99
2) [Chrome] › messages/messages.spec.ts:95:17 › Message rendering › with rtl display name › should render an LTR rich text emote @screenshot Error: expect(locator).toHaveScreenshot(expected) 18 pixels (ratio 0.01 of all image pixels) are different. Expected: /home/runner/work/element-web/element-web/playwright/snapshots/messages/messages.spec.ts/emote-rich-ltr-rtldisplayname-linux.png Received: /home/runner/work/element-web/element-web/playwright/test-results/messages-messages-Message--2831a-nder-an-LTR-rich-text-emote-Chrome/emote-rich-ltr-rtldisplayname-actual.png Diff: /home/runner/work/element-web/element-web/playwright/test-results/messages-messages-Message--2831a-nder-an-LTR-rich-text-emote-Chrome/emote-rich-ltr-rtldisplayname-diff.png Call log: - expect.toHaveScreenshot(emote-rich-ltr-rtldisplayname.png) with timeout 5000ms - verifying given screenshot expectation - waiting for locator('.mx_EventTile_last') - locator resolved to <li tabindex="-1" aria-live="off" data-self="true" aria-atomic="true" data-layout="group" data-has-reply="false" data-event-id="$EWCMQaDgcgSH2ObQZdjiB43LWAva6-UqSTUayP6imn8" data-scroll-tokens="$EWCMQaDgcgSH2ObQZdjiB43LWAva6-UqSTUayP6imn8" class="mx_EventTile mx_EventTile_last mx_EventTile_lastInSection mx_EventTile_emote mx_EventTile_noBubble">…</li> - taking element screenshot - disabled all CSS animations - waiting for fonts to load... - fonts loaded - attempting scroll into view action - waiting for element to be stable - 18 pixels (ratio 0.01 of all image pixels) are different. - waiting 100ms before taking screenshot - waiting for locator('.mx_EventTile_last') - locator resolved to <li tabindex="-1" aria-live="off" data-self="true" aria-atomic="true" data-layout="group" data-has-reply="false" data-event-id="$EWCMQaDgcgSH2ObQZdjiB43LWAva6-UqSTUayP6imn8" data-scroll-tokens="$EWCMQaDgcgSH2ObQZdjiB43LWAva6-UqSTUayP6imn8" class="mx_EventTile mx_EventTile_last mx_EventTile_lastInSection mx_EventTile_emote mx_EventTile_noBubble">…</li> - taking element screenshot - disabled all CSS animations - waiting for fonts to load... - fonts loaded - attempting scroll into view action - waiting for element to be stable - captured a stable screenshot - 18 pixels (ratio 0.01 of all image pixels) are different. 97 | 98 | const msgTile = await sendMessage(page, "/me lays a *free range* egg"); > 99 | await expect(msgTile).toMatchScreenshot(`emote-rich-ltr-${direction}displayname.png`); | ^ 100 | }); 101 | 102 | test("should render an edited LTR message", async ({ page, user, app, room }) => { at /home/runner/work/element-web/element-web/playwright/e2e/messages/messages.spec.ts:99:39
[Chrome] › oidc/oidc-native.spec.ts:20:9 › OIDC Native › can register the oauth2 client and an account @no-firefox @no-webkit: playwright/e2e/oidc/index.ts#L35
1) [Chrome] › oidc/oidc-native.spec.ts:20:9 › OIDC Native › can register the oauth2 client and an account @no-firefox @no-webkit Error: expect(received).toEqual(expected) // deep equality Expected: "alice_f13781756eb57ed65475-62abe8a81a983f58e943 <alice@email.com>" Received: "alice <alice@email.com>" at oidc/index.ts:35 33 | expect(messages.items).toHaveLength(1); 34 | }).toPass(); > 35 | expect(messages.items[0].to).toEqual(`${username} <${email}>`); | ^ 36 | const [code] = messages.items[0].text.match(/(\d{6})/); 37 | 38 | await page.getByRole("textbox", { name: "6-digit code" }).fill(code); at registerAccountMas (/home/runner/work/element-web/element-web/playwright/e2e/oidc/index.ts:35:34) at /home/runner/work/element-web/element-web/playwright/e2e/oidc/oidc-native.spec.ts:38:9
[Chrome] › oidc/oidc-native.spec.ts:20:9 › OIDC Native › can register the oauth2 client and an account @no-firefox @no-webkit: playwright/e2e/oidc/oidc-native.spec.ts#L30
1) [Chrome] › oidc/oidc-native.spec.ts:20:9 › OIDC Native › can register the oauth2 client and an account @no-firefox @no-webkit Error: page.waitForRequest: Test ended. 28 | 29 | const tokenUri = `${mas.baseUrl}/oauth2/token`; > 30 | const tokenApiPromise = page.waitForRequest( | ^ 31 | (request) => request.url() === tokenUri && request.postDataJSON()["grant_type"] === "authorization_code", 32 | ); 33 | at /home/runner/work/element-web/element-web/playwright/e2e/oidc/oidc-native.spec.ts:30:38
[Chrome] › oidc/oidc-native.spec.ts:20:9 › OIDC Native › can register the oauth2 client and an account @no-firefox @no-webkit: playwright/e2e/oidc/index.ts#L40
1) [Chrome] › oidc/oidc-native.spec.ts:20:9 › OIDC Native › can register the oauth2 client and an account @no-firefox @no-webkit Retry #1 ─────────────────────────────────────────────────────────────────────────────────────── Error: Timed out 5000ms waiting for expect(locator).toBeVisible() Locator: getByText('Allow access to your account?') Expected: visible Received: <element(s) not found> Call log: - expect.toBeVisible with timeout 5000ms - waiting for getByText('Allow access to your account?') at oidc/index.ts:40 38 | await page.getByRole("textbox", { name: "6-digit code" }).fill(code); 39 | await page.getByRole("button", { name: "Continue" }).click(); > 40 | await expect(page.getByText("Allow access to your account?")).toBeVisible(); | ^ 41 | await page.getByRole("button", { name: "Continue" }).click(); 42 | } 43 | at registerAccountMas (/home/runner/work/element-web/element-web/playwright/e2e/oidc/index.ts:40:67) at /home/runner/work/element-web/element-web/playwright/e2e/oidc/oidc-native.spec.ts:38:9
[Chrome] › oidc/oidc-native.spec.ts:20:9 › OIDC Native › can register the oauth2 client and an account @no-firefox @no-webkit: playwright/e2e/oidc/oidc-native.spec.ts#L30
1) [Chrome] › oidc/oidc-native.spec.ts:20:9 › OIDC Native › can register the oauth2 client and an account @no-firefox @no-webkit Retry #1 ─────────────────────────────────────────────────────────────────────────────────────── Error: page.waitForRequest: Test ended. 28 | 29 | const tokenUri = `${mas.baseUrl}/oauth2/token`; > 30 | const tokenApiPromise = page.waitForRequest( | ^ 31 | (request) => request.url() === tokenUri && request.postDataJSON()["grant_type"] === "authorization_code", 32 | ); 33 | at /home/runner/work/element-web/element-web/playwright/e2e/oidc/oidc-native.spec.ts:30:38
[Chrome] › oidc/oidc-native.spec.ts:20:9 › OIDC Native › can register the oauth2 client and an account @no-firefox @no-webkit: playwright/e2e/oidc/index.ts#L40
1) [Chrome] › oidc/oidc-native.spec.ts:20:9 › OIDC Native › can register the oauth2 client and an account @no-firefox @no-webkit Retry #2 ─────────────────────────────────────────────────────────────────────────────────────── Error: Timed out 5000ms waiting for expect(locator).toBeVisible() Locator: getByText('Allow access to your account?') Expected: visible Received: <element(s) not found> Call log: - expect.toBeVisible with timeout 5000ms - waiting for getByText('Allow access to your account?') at oidc/index.ts:40 38 | await page.getByRole("textbox", { name: "6-digit code" }).fill(code); 39 | await page.getByRole("button", { name: "Continue" }).click(); > 40 | await expect(page.getByText("Allow access to your account?")).toBeVisible(); | ^ 41 | await page.getByRole("button", { name: "Continue" }).click(); 42 | } 43 | at registerAccountMas (/home/runner/work/element-web/element-web/playwright/e2e/oidc/index.ts:40:67) at /home/runner/work/element-web/element-web/playwright/e2e/oidc/oidc-native.spec.ts:38:9
[Chrome] › oidc/oidc-native.spec.ts:20:9 › OIDC Native › can register the oauth2 client and an account @no-firefox @no-webkit: playwright/e2e/oidc/oidc-native.spec.ts#L30
1) [Chrome] › oidc/oidc-native.spec.ts:20:9 › OIDC Native › can register the oauth2 client and an account @no-firefox @no-webkit Retry #2 ─────────────────────────────────────────────────────────────────────────────────────── Error: page.waitForRequest: Test ended. 28 | 29 | const tokenUri = `${mas.baseUrl}/oauth2/token`; > 30 | const tokenApiPromise = page.waitForRequest( | ^ 31 | (request) => request.url() === tokenUri && request.postDataJSON()["grant_type"] === "authorization_code", 32 | ); 33 | at /home/runner/work/element-web/element-web/playwright/e2e/oidc/oidc-native.spec.ts:30:38
[Chrome] › crypto/backups-mas.spec.ts:31:9 › Encryption state after registration › user is prompted to set up recovery: playwright/e2e/crypto/backups-mas.spec.ts#L1
2) [Chrome] › crypto/backups-mas.spec.ts:31:9 › Encryption state after registration › user is prompted to set up recovery Test timeout of 30000ms exceeded.
[Chrome] › crypto/backups-mas.spec.ts:31:9 › Encryption state after registration › user is prompted to set up recovery: playwright/e2e/oidc/index.ts#L38
2) [Chrome] › crypto/backups-mas.spec.ts:31:9 › Encryption state after registration › user is prompted to set up recovery Error: locator.fill: Test timeout of 30000ms exceeded. Call log: - waiting for getByRole('textbox', { name: '6-digit code' }) at oidc/index.ts:38 36 | const [code] = messages.items[0].text.match(/(\d{6})/); 37 | > 38 | await page.getByRole("textbox", { name: "6-digit code" }).fill(code); | ^ 39 | await page.getByRole("button", { name: "Continue" }).click(); 40 | await expect(page.getByText("Allow access to your account?")).toBeVisible(); 41 | await page.getByRole("button", { name: "Continue" }).click(); at registerAccountMas (/home/runner/work/element-web/element-web/playwright/e2e/oidc/index.ts:38:63) at /home/runner/work/element-web/element-web/playwright/e2e/crypto/backups-mas.spec.ts:34:9
[Chrome] › crypto/backups-mas.spec.ts:48:9 › Key backup reset from elsewhere › Key backup is disabled when reset from elsewhere: playwright/e2e/crypto/backups-mas.spec.ts#L1
3) [Chrome] › crypto/backups-mas.spec.ts:48:9 › Key backup reset from elsewhere › Key backup is disabled when reset from elsewhere Test timeout of 30000ms exceeded.
[Chrome] › crypto/backups-mas.spec.ts:48:9 › Key backup reset from elsewhere › Key backup is disabled when reset from elsewhere: playwright/e2e/oidc/index.ts#L38
3) [Chrome] › crypto/backups-mas.spec.ts:48:9 › Key backup reset from elsewhere › Key backup is disabled when reset from elsewhere Error: locator.fill: Test timeout of 30000ms exceeded. Call log: - waiting for getByRole('textbox', { name: '6-digit code' }) at oidc/index.ts:38 36 | const [code] = messages.items[0].text.match(/(\d{6})/); 37 | > 38 | await page.getByRole("textbox", { name: "6-digit code" }).fill(code); | ^ 39 | await page.getByRole("button", { name: "Continue" }).click(); 40 | await expect(page.getByText("Allow access to your account?")).toBeVisible(); 41 | await page.getByRole("button", { name: "Continue" }).click(); at registerAccountMas (/home/runner/work/element-web/element-web/playwright/e2e/oidc/index.ts:38:63) at /home/runner/work/element-web/element-web/playwright/e2e/crypto/backups-mas.spec.ts:58:9
[Chrome] › spotlight/spotlight.spec.ts:132:9 › Spotlight › should find known public rooms: playwright/e2e/spotlight/spotlight.spec.ts#L138
1) [Chrome] › spotlight/spotlight.spec.ts:132:9 › Spotlight › should find known public rooms ───── Error: Timed out 5000ms waiting for expect(locator).toHaveCount(expected) Locator: locator('[role=dialog][aria-label="Search Dialog"]').locator('.mx_SpotlightDialog_section.mx_SpotlightDialog_results .mx_SpotlightDialog_option') Expected: 1 Received: 3 Call log: - expect.toHaveCount with timeout 5000ms - waiting for locator('[role=dialog][aria-label="Search Dialog"]').locator('.mx_SpotlightDialog_section.mx_SpotlightDialog_results .mx_SpotlightDialog_option') 2 × locator resolved to 0 elements - unexpected value "0" 7 × locator resolved to 3 elements - unexpected value "3" 136 | await spotlight.search(room1Name); 137 | const resultLocator = spotlight.results; > 138 | await expect(resultLocator).toHaveCount(1); | ^ 139 | await expect(resultLocator.first()).toContainText(room1Name); 140 | await expect(resultLocator.first()).toContainText("View"); 141 | await resultLocator.first().click(); at /home/runner/work/element-web/element-web/playwright/e2e/spotlight/spotlight.spec.ts:138:37
[Chrome] › spotlight/spotlight.spec.ts:146:9 › Spotlight › should find unknown public rooms: playwright/e2e/spotlight/spotlight.spec.ts#L152
2) [Chrome] › spotlight/spotlight.spec.ts:146:9 › Spotlight › should find unknown public rooms ─── Error: Timed out 5000ms waiting for expect(locator).toHaveCount(expected) Locator: locator('[role=dialog][aria-label="Search Dialog"]').locator('.mx_SpotlightDialog_section.mx_SpotlightDialog_results .mx_SpotlightDialog_option') Expected: 1 Received: 2 Call log: - expect.toHaveCount with timeout 5000ms - waiting for locator('[role=dialog][aria-label="Search Dialog"]').locator('.mx_SpotlightDialog_section.mx_SpotlightDialog_results .mx_SpotlightDialog_option') 2 × locator resolved to 0 elements - unexpected value "0" 7 × locator resolved to 2 elements - unexpected value "2" 150 | await spotlight.search(room2Name); 151 | const resultLocator = spotlight.results; > 152 | await expect(resultLocator).toHaveCount(1); | ^ 153 | await expect(resultLocator.first()).toContainText(room2Name); 154 | await expect(resultLocator.first()).toContainText("Join"); 155 | await resultLocator.first().click(); at /home/runner/work/element-web/element-web/playwright/e2e/spotlight/spotlight.spec.ts:152:37
[Chrome] › spotlight/spotlight.spec.ts:146:9 › Spotlight › should find unknown public rooms: playwright/e2e/spotlight/spotlight.spec.ts#L89
2) [Chrome] › spotlight/spotlight.spec.ts:146:9 › Spotlight › should find unknown public rooms ─── Retry #1 ─────────────────────────────────────────────────────────────────────────────────────── Error: expect.not.toBeAttached: Error: strict mode violation: locator('.mx_RoomSublist_skeletonUI') resolved to 8 elements: 1) <div class="mx_RoomSublist_skeletonUI"></div> aka locator('.mx_RoomSublist_skeletonUI').first() 2) <div class="mx_RoomSublist_skeletonUI"></div> aka locator('div:nth-child(2) > .mx_RoomSublist_skeletonUI') 3) <div class="mx_RoomSublist_skeletonUI"></div> aka locator('div:nth-child(3) > .mx_RoomSublist_skeletonUI') 4) <div class="mx_RoomSublist_skeletonUI"></div> aka locator('div:nth-child(4) > .mx_RoomSublist_skeletonUI') 5) <div class="mx_RoomSublist_skeletonUI"></div> aka locator('div:nth-child(5) > .mx_RoomSublist_skeletonUI') 6) <div class="mx_RoomSublist_skeletonUI"></div> aka locator('div:nth-child(6) > .mx_RoomSublist_skeletonUI') 7) <div class="mx_RoomSublist_skeletonUI"></div> aka locator('div:nth-child(7) > .mx_RoomSublist_skeletonUI') 8) <div class="mx_RoomSublist_skeletonUI"></div> aka locator('div:nth-child(8) > .mx_RoomSublist_skeletonUI') Call log: - expect.not.toBeAttached with timeout 5000ms - waiting for locator('.mx_RoomSublist_skeletonUI') 87 | 88 | await page.goto("/#/room/" + room1Id); > 89 | await expect(page.locator(".mx_RoomSublist_skeletonUI")).not.toBeAttached(); | ^ 90 | }); 91 | 92 | test("should be able to add and remove filters via keyboard", async ({ page, app }) => { at /home/runner/work/element-web/element-web/playwright/e2e/spotlight/spotlight.spec.ts:89:70
[Chrome] › spotlight/spotlight.spec.ts:161:9 › Spotlight › should find unknown public world readable rooms: playwright/e2e/spotlight/spotlight.spec.ts#L167
3) [Chrome] › spotlight/spotlight.spec.ts:161:9 › Spotlight › should find unknown public world readable rooms Error: Timed out 5000ms waiting for expect(locator).toHaveCount(expected) Locator: locator('[role=dialog][aria-label="Search Dialog"]').locator('.mx_SpotlightDialog_section.mx_SpotlightDialog_results .mx_SpotlightDialog_option') Expected: 1 Received: 2 Call log: - expect.toHaveCount with timeout 5000ms - waiting for locator('[role=dialog][aria-label="Search Dialog"]').locator('.mx_SpotlightDialog_section.mx_SpotlightDialog_results .mx_SpotlightDialog_option') 2 × locator resolved to 0 elements - unexpected value "0" 7 × locator resolved to 2 elements - unexpected value "2" 165 | await spotlight.search(room3Name); 166 | const resultLocator = spotlight.results; > 167 | await expect(resultLocator).toHaveCount(1); | ^ 168 | await expect(resultLocator.first()).toContainText(room3Name); 169 | await expect(resultLocator.first()).toContainText("View"); 170 | await resultLocator.first().click(); at /home/runner/work/element-web/element-web/playwright/e2e/spotlight/spotlight.spec.ts:167:37
[Chrome] › spotlight/spotlight.spec.ts:220:9 › Spotlight › should find unknown people: playwright/e2e/spotlight/spotlight.spec.ts#L226
4) [Chrome] › spotlight/spotlight.spec.ts:220:9 › Spotlight › should find unknown people ───────── Error: Timed out 5000ms waiting for expect(locator).toHaveCount(expected) Locator: locator('[role=dialog][aria-label="Search Dialog"]').locator('.mx_SpotlightDialog_section.mx_SpotlightDialog_results .mx_SpotlightDialog_option') Expected: 1 Received: 3 Call log: - expect.toHaveCount with timeout 5000ms - waiting for locator('[role=dialog][aria-label="Search Dialog"]').locator('.mx_SpotlightDialog_section.mx_SpotlightDialog_results .mx_SpotlightDialog_option') 2 × locator resolved to 0 elements - unexpected value "0" 7 × locator resolved to 3 elements - unexpected value "3" 224 | await spotlight.search(bot2Name); 225 | const resultLocator = spotlight.results; > 226 | await expect(resultLocator).toHaveCount(1); | ^ 227 | await expect(resultLocator.first()).toContainText(bot2Name); 228 | await resultLocator.first().click(); 229 | await expect(roomHeaderName(page)).toHaveText(bot2Name); at /home/runner/work/element-web/element-web/playwright/e2e/spotlight/spotlight.spec.ts:226:37
[Chrome] › spotlight/spotlight.spec.ts:321:9 › Spotlight › should allow opening group chat dialog: playwright/e2e/spotlight/spotlight.spec.ts#L329
5) [Chrome] › spotlight/spotlight.spec.ts:321:9 › Spotlight › should allow opening group chat dialog Error: Timed out 5000ms waiting for expect(locator).toHaveCount(expected) Locator: locator('[role=dialog][aria-label="Search Dialog"]').locator('.mx_SpotlightDialog_section.mx_SpotlightDialog_results .mx_SpotlightDialog_option') Expected: 1 Received: 4 Call log: - expect.toHaveCount with timeout 5000ms - waiting for locator('[role=dialog][aria-label="Search Dialog"]').locator('.mx_SpotlightDialog_section.mx_SpotlightDialog_results .mx_SpotlightDialog_option') 9 × locator resolved to 4 elements - unexpected value "4" 327 | 328 | const resultLocator = spotlight.results; > 329 | await expect(resultLocator).toHaveCount(1); | ^ 330 | await expect(resultLocator.first()).toContainText(bot2Name); 331 | 332 | await expect(spotlight.dialog.locator(".mx_SpotlightDialog_startGroupChat")).toContainText( at /home/runner/work/element-web/element-web/playwright/e2e/spotlight/spotlight.spec.ts:329:37
[Chrome] › spotlight/spotlight.spec.ts:339:9 › Spotlight › should close spotlight after starting a DM: playwright/e2e/spotlight/spotlight.spec.ts#L27
6) [Chrome] › spotlight/spotlight.spec.ts:339:9 › Spotlight › should close spotlight after starting a DM Error: Timed out 5000ms waiting for expect(locator).toHaveCount(expected) Locator: locator('[role=dialog][aria-label="Search Dialog"]').locator('.mx_SpotlightDialog_section.mx_SpotlightDialog_results .mx_SpotlightDialog_option') Expected: 1 Received: 2 Call log: - expect.toHaveCount with timeout 5000ms - waiting for locator('[role=dialog][aria-label="Search Dialog"]').locator('.mx_SpotlightDialog_section.mx_SpotlightDialog_results .mx_SpotlightDialog_option') 9 × locator resolved to 2 elements - unexpected value "2" 25 | await expect(spotlight.dialog.locator(".mx_Spinner")).not.toBeAttached(); 26 | const result = spotlight.results; > 27 | await expect(result).toHaveCount(1); | ^ 28 | await expect(result.first()).toContainText(name); 29 | await result.first().click(); 30 | at startDM (/home/runner/work/element-web/element-web/playwright/e2e/spotlight/spotlight.spec.ts:27:26) at /home/runner/work/element-web/element-web/playwright/e2e/spotlight/spotlight.spec.ts:340:9
[Chrome] › spotlight/spotlight.spec.ts:344:9 › Spotlight › should show the same user only once: playwright/e2e/spotlight/spotlight.spec.ts#L27
7) [Chrome] › spotlight/spotlight.spec.ts:344:9 › Spotlight › should show the same user only once Error: Timed out 5000ms waiting for expect(locator).toHaveCount(expected) Locator: locator('[role=dialog][aria-label="Search Dialog"]').locator('.mx_SpotlightDialog_section.mx_SpotlightDialog_results .mx_SpotlightDialog_option') Expected: 1 Received: 2 Call log: - expect.toHaveCount with timeout 5000ms - waiting for locator('[role=dialog][aria-label="Search Dialog"]').locator('.mx_SpotlightDialog_section.mx_SpotlightDialog_results .mx_SpotlightDialog_option') 9 × locator resolved to 2 elements - unexpected value "2" 25 | await expect(spotlight.dialog.locator(".mx_Spinner")).not.toBeAttached(); 26 | const result = spotlight.results; > 27 | await expect(result).toHaveCount(1); | ^ 28 | await expect(result.first()).toContainText(name); 29 | await result.first().click(); 30 | at startDM (/home/runner/work/element-web/element-web/playwright/e2e/spotlight/spotlight.spec.ts:27:26) at /home/runner/work/element-web/element-web/playwright/e2e/spotlight/spotlight.spec.ts:345:9
[Chrome] › spotlight/spotlight.spec.ts:357:9 › Spotlight › should be able to navigate results via keyboard: playwright/e2e/spotlight/spotlight.spec.ts#L364
8) [Chrome] › spotlight/spotlight.spec.ts:357:9 › Spotlight › should be able to navigate results via keyboard Error: Timed out 5000ms waiting for expect(locator).toHaveCount(expected) Locator: locator('[role=dialog][aria-label="Search Dialog"]').locator('.mx_SpotlightDialog_section.mx_SpotlightDialog_results .mx_SpotlightDialog_option') Expected: 2 Received: 4 Call log: - expect.toHaveCount with timeout 5000ms - waiting for locator('[role=dialog][aria-label="Search Dialog"]').locator('.mx_SpotlightDialog_section.mx_SpotlightDialog_results .mx_SpotlightDialog_option') 2 × locator resolved to 1 element - unexpected value "1" 7 × locator resolved to 4 elements - unexpected value "4" 362 | 363 | let resultLocator = spotlight.results; > 364 | await expect(resultLocator).toHaveCount(2); | ^ 365 | await expect(resultLocator.first()).toHaveAttribute("aria-selected", "true"); 366 | await expect(resultLocator.last()).toHaveAttribute("aria-selected", "false"); 367 | at /home/runner/work/element-web/element-web/playwright/e2e/spotlight/spotlight.spec.ts:364:37
end-to-end-tests
Process completed with exit code 1.
Slow Test: [Chrome] › pinned-messages/pinned-messages.spec.ts#L1
[Chrome] › pinned-messages/pinned-messages.spec.ts took 57.2s
Slow Test: [Chrome] › right-panel/file-panel.spec.ts#L1
[Chrome] › right-panel/file-panel.spec.ts took 19.1s
Slow Test: [Chrome] › polls/polls.spec.ts#L1
[Chrome] › polls/polls.spec.ts took 18.7s
Slow Test: [Chrome] › one-to-one-chat/one-to-one-chat.spec.ts#L1
[Chrome] › one-to-one-chat/one-to-one-chat.spec.ts took 17.6s
Slow Test: [Chrome] › sliding-sync/sliding-sync.spec.ts#L1
[Chrome] › sliding-sync/sliding-sync.spec.ts took 57.7s
Slow Test: [Chrome] › room/room-header.spec.ts#L1
[Chrome] › room/room-header.spec.ts took 24.7s
Slow Test: [Chrome] › settings/account-user-settings-tab.spec.ts#L1
[Chrome] › settings/account-user-settings-tab.spec.ts took 17.5s
Slow Test: [Chrome] › crypto/invisible-crypto.spec.ts#L1
[Chrome] › crypto/invisible-crypto.spec.ts took 23.6s
Slow Test: [Chrome] › knock/knock-into-room.spec.ts#L1
[Chrome] › knock/knock-into-room.spec.ts took 18.4s
Slow Test: [Chrome] › integration-manager/kick.spec.ts#L1
[Chrome] › integration-manager/kick.spec.ts took 18.2s
Slow Test: [Chrome] › editing/editing.spec.ts#L1
[Chrome] › editing/editing.spec.ts took 15.6s
Slow Test: [Chrome] › lazy-loading/lazy-loading.spec.ts#L1
[Chrome] › lazy-loading/lazy-loading.spec.ts took 15.6s
Slow Test: [Chrome] › crypto/event-shields.spec.ts#L1
[Chrome] › crypto/event-shields.spec.ts took 50.7s
Slow Test: [Chrome] › crypto/decryption-failure-messages.spec.ts#L1
[Chrome] › crypto/decryption-failure-messages.spec.ts took 42.4s
Slow Test: [Chrome] › audio-player/audio-player.spec.ts#L1
[Chrome] › audio-player/audio-player.spec.ts took 37.8s
Slow Test: [Chrome] › composer/RTE.spec.ts#L1
[Chrome] › composer/RTE.spec.ts took 35.6s
Slow Test: [Chrome] › crypto/crypto.spec.ts#L1
[Chrome] › crypto/crypto.spec.ts took 34.0s
Slow Test: [Chrome] › login/login-consent.spec.ts#L1
[Chrome] › login/login-consent.spec.ts took 45.9s
Slow Test: [Chrome] › user-onboarding/user-onboarding-new.spec.ts#L1
[Chrome] › user-onboarding/user-onboarding-new.spec.ts took 24.0s
Slow Test: [Chrome] › app-loading/guest-registration.spec.ts#L1
[Chrome] › app-loading/guest-registration.spec.ts took 17.4s
Slow Test: [Chrome] › register/register.spec.ts#L1
[Chrome] › register/register.spec.ts took 16.3s
Slow Test: [Chrome] › timeline/timeline.spec.ts#L1
[Chrome] › timeline/timeline.spec.ts took 1.3m
Slow Test: [Chrome] › spaces/threads-activity-centre/threadsActivityCentre.spec.ts#L1
[Chrome] › spaces/threads-activity-centre/threadsActivityCentre.spec.ts took 49.2s
Slow Test: [Chrome] › spaces/spaces.spec.ts#L1
[Chrome] › spaces/spaces.spec.ts took 46.4s
Slow Test: [Chrome] › threads/threads.spec.ts#L1
[Chrome] › threads/threads.spec.ts took 19.4s
🎭 Playwright Run Summary
1 flaky [Chrome] › right-panel/right-panel.spec.ts:41:13 › RightPanel › in rooms › should handle long room address and long room name @screenshot 1 skipped 34 passed (3.3m)
🎭 Playwright Run Summary
1 failed [Chrome] › share-dialog/share-dialog.spec.ts:49:9 › Share dialog › should share an event @screenshot 2 flaky [Chrome] › room_options/marked_unread.spec.ts:22:9 › Mark as Unread › should mark a room as unread [Chrome] › room-directory/room-directory.spec.ts:18:9 › Room Directory › should allow admin to add alias & publish room to directory @no-webkit 1 skipped 46 passed (4.1m)
🎭 Playwright Run Summary
2 flaky [Chrome] › knock/create-knock-room.spec.ts:61:9 › Create Knock Room › should create a public knock room [Chrome] › messages/messages.spec.ts:95:17 › Message rendering › with rtl display name › should render an LTR rich text emote @screenshot 1 skipped 63 passed (4.8m)
🎭 Playwright Run Summary
56 passed (5.0m)
🎭 Playwright Run Summary
1 failed [Chrome] › oidc/oidc-native.spec.ts:20:9 › OIDC Native › can register the oauth2 client and an account @no-firefox @no-webkit 4 flaky [Chrome] › crypto/backups-mas.spec.ts:31:9 › Encryption state after registration › user is prompted to set up recovery [Chrome] › crypto/backups-mas.spec.ts:48:9 › Key backup reset from elsewhere › Key backup is disabled when reset from elsewhere [Chrome] › login/soft_logout_oauth.spec.ts:40:9 › Soft logout with SSO user › shows the soft-logout page when a request fails, and allows a re-login [Chrome] › register/email.spec.ts:34:9 › Email Registration › registers an account and lands on the use case selection screen @screenshot 39 passed (7.2m)
🎭 Playwright Run Summary
8 flaky [Chrome] › spotlight/spotlight.spec.ts:132:9 › Spotlight › should find known public rooms ────── [Chrome] › spotlight/spotlight.spec.ts:146:9 › Spotlight › should find unknown public rooms ──── [Chrome] › spotlight/spotlight.spec.ts:161:9 › Spotlight › should find unknown public world readable rooms [Chrome] › spotlight/spotlight.spec.ts:220:9 › Spotlight › should find unknown people ────────── [Chrome] › spotlight/spotlight.spec.ts:321:9 › Spotlight › should allow opening group chat dialog [Chrome] › spotlight/spotlight.spec.ts:339:9 › Spotlight › should close spotlight after starting a DM [Chrome] › spotlight/spotlight.spec.ts:344:9 › Spotlight › should show the same user only once ─ [Chrome] › spotlight/spotlight.spec.ts:357:9 › Spotlight › should be able to navigate results via keyboard 3 skipped 49 passed (7.3m)

Artifacts

Produced during runtime
Name Size
all-blob-reports-Chrome-1
188 KB
all-blob-reports-Chrome-2
7.82 MB
all-blob-reports-Chrome-3
5.97 MB
all-blob-reports-Chrome-4
11.3 MB
all-blob-reports-Chrome-5
36.4 MB
all-blob-reports-Chrome-6
23.3 MB
html-report
77.1 MB
webapp
28.7 MB