chore(deps): Update playwright to 1.50.0 #15164

mydea · 2025-01-27T08:22:02Z

Also align to use the same version everywhere, tilde-restricted.

codecov · 2025-01-27T08:31:07Z

❌ 3 Tests Failed:

Tests completed	Failed	Passed	Skipped
451	3	448	294

View the top 3 failed tests by shortest run time

tracing/browserTracingIntegration/http-timings/test.ts should create fetch spans with http timing @firefox

Stack Traces | 10.3s run time

test.ts:7:11 should create fetch spans with http timing @firefox

tracing/browserTracingIntegration/interactions/test.ts should capture interaction transaction. @firefox

Stack Traces | 30.1s run time

test.ts:11:11 should capture interaction transaction. @firefox

tracing/browserTracingIntegration/interactions/test.ts should create only one transaction per interaction @firefox

Stack Traces | 30.1s run time

test.ts:50:11 should create only one transaction per interaction @firefox

To view more test analytics, go to the Test Analytics Dashboard
📢 Thoughts on this report? Let us know!

mydea · 2025-01-27T12:07:26Z

Hmm, looking some more into this 😬 it seems that Firefox has finally implemented keepalive in 133 (Playwright 1.50.0 uses 134). And apparently, it handles this a bit funky. It seems to not be inspectable/blocked in playwright :/ Not quite sure what is going on there. I looked into a regular Sentry app in firefox, and what I noticed is that sentyr requests do not show up in the network tab. However, they do seem to get send. This is "fixed" if I remove the keepalive: true flag. So my suspicion is that Firefox somehow handles keepalive requests separately (as they are kind of background requests) which maybe also conflicts with Playwright 🤔

mydea · 2025-01-27T12:17:05Z

See microsoft/playwright#34497

mydea · 2025-01-27T12:19:37Z

dev-packages/browser-integration-tests/suites/stacktraces/init.js

+  transportOptions: {
+    fetchOptions: {
+      // See: https://github.com/microsoft/playwright/issues/34497
+      keepalive: false,


Important note: This is apparently a bug in playwright/firefox 🤔 I opened an issue here: microsoft/playwright#34497 but this seems to fix it. I tried to reproduce this in a regular FF instance, and it seems that Sentry data is correctly sent, but cannot be inspected. Possibly this interferes with PW abilities to intercept stuff too.

github-actions · 2025-01-27T12:22:05Z

size-limit report 📦

Path	Size	% Change	Change
@sentry/browser	23.06 KB	+0.02%	+4 B 🔺
@sentry/browser - with treeshaking flags	22.95 KB	+0.02%	+4 B 🔺
@sentry/browser (incl. Tracing)	35.79 KB	+0.01%	+3 B 🔺
@sentry/browser (incl. Tracing, Replay)	72.58 KB	+0.01%	+4 B 🔺
@sentry/browser (incl. Tracing, Replay) - with treeshaking flags	66.13 KB	+0.01%	+4 B 🔺
@sentry/browser (incl. Tracing, Replay with Canvas)	76.83 KB	+0.01%	+4 B 🔺
@sentry/browser (incl. Tracing, Replay, Feedback)	89.32 KB	+0.01%	+4 B 🔺
@sentry/browser (incl. Feedback)	39.74 KB	+0.01%	+4 B 🔺
@sentry/browser (incl. sendFeedback)	27.69 KB	+0.02%	+4 B 🔺
@sentry/browser (incl. FeedbackAsync)	32.47 KB	+0.02%	+4 B 🔺
@sentry/react	25.73 KB	+0.02%	+4 B 🔺
@sentry/react (incl. Tracing)	38.56 KB	+0.01%	+3 B 🔺
@sentry/vue	27.16 KB	+0.02%	+4 B 🔺
@sentry/vue (incl. Tracing)	37.51 KB	+0.02%	+5 B 🔺
@sentry/svelte	23.18 KB	+0.02%	+4 B 🔺
CDN Bundle	24.27 KB	+0.02%	+3 B 🔺
CDN Bundle (incl. Tracing)	35.92 KB	+0.02%	+4 B 🔺
CDN Bundle (incl. Tracing, Replay)	70.54 KB	+0.01%	+6 B 🔺
CDN Bundle (incl. Tracing, Replay, Feedback)	75.68 KB	+0.01%	+5 B 🔺
CDN Bundle - uncompressed	70.95 KB	+0.02%	+14 B 🔺
CDN Bundle (incl. Tracing) - uncompressed	106.58 KB	+0.02%	+14 B 🔺
CDN Bundle (incl. Tracing, Replay) - uncompressed	217.43 KB	+0.01%	+14 B 🔺
CDN Bundle (incl. Tracing, Replay, Feedback) - uncompressed	230 KB	+0.01%	+14 B 🔺
@sentry/nextjs (client)	38.66 KB	+0.01%	+3 B 🔺
@sentry/sveltekit (client)	36.3 KB	+0.01%	+3 B 🔺
@sentry/node	156.39 KB	-0.01%	-1 B 🔽
@sentry/node - without tracing	97.48 KB	-	-
@sentry/aws-serverless	106.94 KB	-	-

View base workflow run

mydea · 2025-01-27T12:45:16Z

...egration-tests/suites/tracing/browserTracingIntegration/long-animation-frame-enabled/test.ts

@@ -100,6 +98,7 @@ sentryTest(
        data: {
          'browser.script.invoker': 'BUTTON#clickme.onclick',
          'browser.script.invoker_type': 'event-listener',
+          'code.filepath': 'https://example.com/path/to/script.js',


this is new, but seems correct to me!

mydea · 2025-01-27T12:45:22Z

...ests/suites/tracing/browserTracingIntegration/long-tasks-and-animation-frame-enabled/test.ts

@@ -102,6 +100,7 @@ sentryTest(
        data: {
          'browser.script.invoker': 'BUTTON#clickme.onclick',
          'browser.script.invoker_type': 'event-listener',
+          'code.filepath': 'https://example.com/path/to/script.js',


this is new, but seems correct to me!

Probably came from the browser version bump 🤔

mydea · 2025-01-27T12:45:36Z

dev-packages/browser-integration-tests/suites/tracing/metrics/web-vitals-lcp/test.ts

-  // renderTime is 0 because we don't return the `Timing-Allow-Origin` header
-  // and the image is loaded from a 3rd party origin
-  expect(eventData.contexts?.trace?.data?.['lcp.renderTime']).toBe(0);
+  expect(eventData.contexts?.trace?.data?.['lcp.renderTime']).toBeGreaterThan(0);


@Lms24 are these changes correct? 😅

I think this change can be attributed to browsers now exposing renderTime even without the timing-allow-origin header: https://developer.mozilla.org/en-US/docs/Web/API/PerformanceElementTiming/renderTime#cross-origin_image_render_time

Which makes me think that we can probably merge both tests in this file. Feel free to do in this PR, otherwise I can also do it in a follow-up PR.

ah, I see. I'll merge these then, apparently that changed in chromium versions here!

Lms24 · 2025-01-27T13:45:18Z

dev-packages/browser-integration-tests/suites/tracing/metrics/web-vitals-lcp/test.ts

-  // renderTime is 0 because we don't return the `Timing-Allow-Origin` header
-  // and the image is loaded from a 3rd party origin
-  expect(eventData.contexts?.trace?.data?.['lcp.renderTime']).toBe(0);
+  expect(eventData.contexts?.trace?.data?.['lcp.renderTime']).toBeGreaterThan(0);


I think this change can be attributed to browsers now exposing renderTime even without the timing-allow-origin header: https://developer.mozilla.org/en-US/docs/Web/API/PerformanceElementTiming/renderTime#cross-origin_image_render_time

Which makes me think that we can probably merge both tests in this file. Feel free to do in this PR, otherwise I can also do it in a follow-up PR.

mydea · 2025-01-28T08:57:30Z

dev-packages/browser-integration-tests/scripts/detectFlakyTests.ts

-  if (process.env.CI) {
-    console.log('::endgroup::');
-  }
+  return paths.map(p => `${path.dirname(p)}/`);


this was a bit buggy - e.g. if one dirname was suites/tracing/metrics/web-vitals/test.ts, it would have made it to suites/tracing/metrics/web-vitals, which then in turn also matches e.g. web-vitals-cls etc. dirs. Now, it should be stricter in that regard.

mydea · 2025-01-28T08:57:36Z

dev-packages/browser-integration-tests/scripts/detectFlakyTests.ts

-  return paths.map(p => path.dirname(p));
-}
-
-function logError(error: unknown) {


was not used

mydea · 2025-01-28T08:57:49Z

dev-packages/browser-integration-tests/scripts/detectFlakyTests.ts

@@ -156,7 +127,7 @@ function logError(error: unknown) {
 function getApproximateNumberOfTests(testPath: string): number {
  try {
    const content = fs.readFileSync(path.join(process.cwd(), testPath, 'test.ts'), 'utf-8');
-    const matches = content.match(/it\(|test\(|sentryTest\(/g);
+    const matches = content.match(/sentryTest\(/g);


we always use sentryTest, so no need to check others here IMHO

Also align to use the same version everywhere, tilde-restricted.

mydea self-assigned this Jan 27, 2025

mydea mentioned this pull request Jan 27, 2025

[Bug]: Firefox not handling keepalive: true fetch requests microsoft/playwright#34497

Closed

mydea commented Jan 27, 2025

View reviewed changes

mydea marked this pull request as ready for review January 27, 2025 12:44

mydea requested a review from a team as a code owner January 27, 2025 12:44

mydea requested review from Lms24 and andreiborza January 27, 2025 12:44

mydea commented Jan 27, 2025

View reviewed changes

Lms24 approved these changes Jan 27, 2025

View reviewed changes

Lms24 mentioned this pull request Jan 27, 2025

ref(browser): Add protocol attributes to resource spans #15161

Merged

2 tasks

mydea force-pushed the fn/update-playwright branch from 3943ca4 to f5550b7 Compare January 27, 2025 16:20

mydea commented Jan 28, 2025

View reviewed changes

mydea force-pushed the fn/update-playwright branch from b0b2f85 to 6865dc2 Compare January 28, 2025 13:14

mydea added 11 commits January 28, 2025 17:00

chore(deps): Update playwright to 1.50.0

5d910a8

Also align to use the same version everywhere, tilde-restricted.

fix test

f29ffed

fix tests

ca2cdfb

update snapshots ??

96ca4fd

remove package stuff

f4b07f1

small ref

a7d87d6

fix ci

8001258

FIXES

a74f633

fix it

3889554

fix keepalive

7aa9685

fix remaining firefox tests

6cc0402

mydea added 2 commits January 28, 2025 17:00

merge tests

01bf194

fix flaky test detector

aaac579

mydea force-pushed the fn/update-playwright branch from 6865dc2 to aaac579 Compare January 28, 2025 16:00

mydea merged commit a27652f into develop Jan 28, 2025
167 checks passed

mydea deleted the fn/update-playwright branch January 28, 2025 16:59

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

chore(deps): Update playwright to 1.50.0 #15164

chore(deps): Update playwright to 1.50.0 #15164

mydea commented Jan 27, 2025

codecov bot commented Jan 27, 2025 •

edited

Loading

mydea commented Jan 27, 2025

mydea commented Jan 27, 2025

mydea Jan 27, 2025 •

edited

Loading

github-actions bot commented Jan 27, 2025 •

edited

Loading

mydea Jan 27, 2025

mydea Jan 27, 2025

Lms24 Jan 27, 2025

mydea Jan 27, 2025

Lms24 Jan 27, 2025 •

edited

Loading

mydea Jan 27, 2025

Lms24 Jan 27, 2025 •

edited

Loading

mydea Jan 28, 2025

mydea Jan 28, 2025

mydea Jan 28, 2025

chore(deps): Update playwright to 1.50.0 #15164

chore(deps): Update playwright to 1.50.0 #15164

Conversation

mydea commented Jan 27, 2025

codecov bot commented Jan 27, 2025 • edited Loading

❌ 3 Tests Failed:

mydea commented Jan 27, 2025

mydea commented Jan 27, 2025

mydea Jan 27, 2025 • edited Loading

Choose a reason for hiding this comment

github-actions bot commented Jan 27, 2025 • edited Loading

size-limit report 📦

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Lms24 Jan 27, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Lms24 Jan 27, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov bot commented Jan 27, 2025 •

edited

Loading

mydea Jan 27, 2025 •

edited

Loading

github-actions bot commented Jan 27, 2025 •

edited

Loading

Lms24 Jan 27, 2025 •

edited

Loading

Lms24 Jan 27, 2025 •

edited

Loading