Very high CLS scores on web.dev despite stable layout, inconsistent with Chrome/Node CLI

[widmanski]

Provide the steps to reproduce

1. Run LH via https://web.dev on https://chylak.com/eu/en
2. Observe very high CLS scores (> 3), despite seemingly stable layout
3. Run LH via Chrome Dev Tools or node to observe CLS scores of < 0.05

What is the current behavior?

CLS scores returned by LH of https://web.dev are very high - > 3
Example test JSON: https://gist.github.com/widmanski/07c0eeb3b0630364b3641f9ff782a0a5

CLS scores returned by LH run in Chrome (LH 6.2.0) is 0 or via Node CLI (LH 6.4.1) are < 0.05

The elements that are listed as affecting CLS are as follow:

• The hero element, which changes the css position from fixed to absolute as part of the animation [does not really shift any content though]
• Lazy loaded images, which are absolutely positioned inside div-wrappers, that enforce the correct aspect ratio using the padding-top technique

What is the expected behavior?

The CLS score returned by LH run via web.dev should be consistent with the actual user experience, and with scores returned by other versions of LH.

The site is very fast and stable overall - getting really good LCP, TTI, and other metrics.
The site uses animations - but apart from the hero element, it's just css transforms or opacity.

Would love to get to the bottom of this - please let me know if I can provide any more info.

Separately, the site seem to also trigger the Accessibility bug reported here.

Environment Information

• Affected Channels: web.dev
• Lighthouse version: 6.3.0
• Chrome version: N/A
• Node.js version: N/A
• Operating System: N/A

[patrickhulce]

Thanks for splitting this out @widmanski :)

For future Lighthouse investigators, even at a very slow throttling preset locally I could not reproduce the layout shifts in DevTools or in Lighthouse that we see in PSI. Not sure what's going on.

[adamraine]

I cannot reproduce the super high CLS (>3) in PSI or web.dev measure. @widmanski are you still seeing a CLS this high on your end?

The CLS is still not 0 though, its ~0.032. Looking at the performance panel, these layout shifts are caused by the hero image moving around, but Chrome does not add them to CLS because the had_recent_input flag is set. Lighthouse ignores this flag at the start of the trace and adds the events to CLS anyway:

lighthouse/lighthouse-core/computed/layout-shift-variants.js

Lines 107 to 111 in d19b646

	// Chromium will set `had_recent_input` if there was recent user input, which
	// skips shift events from contributing to CLS. This flag is also set when
	// Lighthouse changes the emulation size. This results in the first few shift
	// events having `had_recent_input` set, so ignore it for those events.
	// See https://bugs.chromium.org/p/chromium/issues/detail?id=1094974.

Since the had_recent_input flag is set in the performance panel without changing the emulation size, we may not need to add these layout shifts to CLS. That being said, I am interested to know what user input Chrome is recording to mark these events with had_recent_input.

[widmanski]

@adamraine thanks for looking into this.

Since posting the issue, the website was reengineered somewhat.
Previously, the CLS was apparently caused by the hero element changing position from fixed to absolute, on scroll.
The issue was resolved by leaving the hero element in fixed position all the time (it simply hides behind the content as you scroll down).

I don't think the hero image should cause CLS - it's only animated using css transforms.

My understanding is that the low CLS may be caused by layout shifts caused by fonts loading causing the sizes to change, on elements above the hero?

[adamraine]

The latest version of Lighthouse will list possible root causes for layout shifts. In your case the large layout shift may be caused by a style update from https://chylak.com/dist/assets/style-3c439e926f7af83ed637.css

In any case, there does appear to be content shifting towards the bottom of the page so don't think there is a bug here.