Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Canvas: Show base color / blurhash while resource is loading #11849

Merged
merged 15 commits into from
Jul 11, 2022

Conversation

spacedmonkey
Copy link
Contributor

@spacedmonkey spacedmonkey commented Jun 30, 2022

Context

Summary

Show base color / blurhash based placeholder for element on canvas.

Relevant Technical Choices

To-do

User-facing changes

Screen.Recording.2022-06-30.at.16.57.02.mov

Testing Instructions

  • This is a non-user-facing change and requires no QA

This PR can be tested by following these steps:

  1. Create story.
  2. Insert a number of video / image elements.
  3. Save story as draft.
  4. Go to media library.
  5. Delete video / image used in elements.
  6. Edit draft story.
  7. See placeholder.
  8. Resized elements, see placeholder change size as well.

Reviews

Does this PR have a security-related impact?

Does this PR change what data or activity we track or use?

Does this PR have a legal-related impact?

Checklist

  • This PR addresses an existing issue and I have linked this PR to it in ZenHub
  • I have tested this code to the best of my abilities
  • I have verified accessibility to the best of my abilities (docs)
  • I have verified i18n and l10n (translation, right-to-left layout) to the best of my abilities
  • This code is covered by automated tests (unit, integration, and/or e2e) to verify it works as intended (docs)
  • I have added documentation where necessary
  • I have added a matching Type: XYZ label to the PR

Fixes #10258

@spacedmonkey spacedmonkey added Type: Enhancement New feature or improvement of an existing feature Group: Media P2 Should do soon Elements: Video Elements: Image Group: Canvas Pod: WP Elements: GIF GIF element as used by Tenor 3P Media integration labels Jun 30, 2022
@spacedmonkey spacedmonkey self-assigned this Jun 30, 2022
@swissspidy
Copy link
Collaborator

Seems to work well already!

@spacedmonkey spacedmonkey marked this pull request as ready for review July 1, 2022 13:00
@spacedmonkey
Copy link
Contributor Author

Added a fix for #10167, as it was really effecting my testing in safari.

@spacedmonkey
Copy link
Contributor Author

Screen.Recording.2022-07-01.at.16.07.09.mov

As noted above, here is the issue in safari. It is none issue, but it make testing this PR a little harder.

@github-actions
Copy link
Contributor

github-actions bot commented Jul 1, 2022

Size Change: +223 B (0%)

Total Size: 2.65 MB

Filename Size Change
assets/js/wp-story-editor.js 324 kB +188 B (0%)
ℹ️ View Unchanged
Filename Size Change
assets/css/carousel-view-rtl.css 702 B 0 B
assets/css/carousel-view.css 701 B 0 B
assets/css/web-stories-block-rtl.css 4.52 kB 0 B
assets/css/web-stories-block.css 4.56 kB 0 B
assets/css/web-stories-embed-rtl.css 318 B 0 B
assets/css/web-stories-embed.css 317 B 0 B
assets/css/web-stories-list-styles-rtl.css 2.36 kB 0 B
assets/css/web-stories-list-styles.css 2.39 kB 0 B
assets/css/web-stories-theme-style-twentyeleven-rtl.css 102 B 0 B
assets/css/web-stories-theme-style-twentyeleven.css 102 B 0 B
assets/css/web-stories-theme-style-twentyfifteen-rtl.css 251 B 0 B
assets/css/web-stories-theme-style-twentyfifteen.css 251 B 0 B
assets/css/web-stories-theme-style-twentyfourteen-rtl.css 287 B 0 B
assets/css/web-stories-theme-style-twentyfourteen.css 287 B 0 B
assets/css/web-stories-theme-style-twentyseventeen-rtl.css 288 B 0 B
assets/css/web-stories-theme-style-twentyseventeen.css 288 B 0 B
assets/css/web-stories-theme-style-twentysixteen-rtl.css 224 B 0 B
assets/css/web-stories-theme-style-twentysixteen.css 224 B 0 B
assets/css/web-stories-theme-style-twentyten-rtl.css 143 B 0 B
assets/css/web-stories-theme-style-twentyten.css 143 B 0 B
assets/css/web-stories-theme-style-twentytwelve-rtl.css 256 B 0 B
assets/css/web-stories-theme-style-twentytwelve.css 256 B 0 B
assets/css/web-stories-theme-style-twentytwenty-rtl.css 86 B 0 B
assets/css/web-stories-theme-style-twentytwenty.css 86 B 0 B
assets/css/web-stories-theme-style-twentytwentyone-rtl.css 326 B 0 B
assets/css/web-stories-theme-style-twentytwentyone.css 326 B 0 B
assets/css/web-stories-widget-rtl.css 482 B 0 B
assets/css/web-stories-widget.css 482 B 0 B
assets/css/wp-dashboard-rtl.css 657 B 0 B
assets/css/wp-dashboard.css 659 B 0 B
assets/css/wp-story-editor-rtl.css 737 B 0 B
assets/css/wp-story-editor.css 738 B 0 B
assets/js/1814.js 7.45 kB 0 B
assets/js/2505.js 34.8 kB 0 B
assets/js/3617.js 224 kB 0 B
assets/js/4422.js 49.3 kB 0 B
assets/js/5980.js 5.48 kB 0 B
assets/js/7.js 1.14 MB +35 B (0%)
assets/js/carousel-view.js 3.41 kB 0 B
assets/js/chunk-colorthief.js 2.64 kB 0 B
assets/js/chunk-ffmpeg.js 5.64 kB 0 B
assets/js/chunk-focus-visible.js 1.01 kB 0 B
assets/js/chunk-getStoryMarkup.js 5.82 kB 0 B
assets/js/chunk-html-to-image.js 4.6 kB 0 B
assets/js/chunk-opentype.js 96 B 0 B
assets/js/chunk-react-calendar.js 12.4 kB 0 B
assets/js/chunk-react-color.js 44.3 kB 0 B
assets/js/chunk-resize-observer-polyfill.js 2.57 kB 0 B
assets/js/chunk-web-animations-js.js 14.6 kB 0 B
assets/js/chunk-web-stories-template-0-metaData.js 546 B 0 B
assets/js/chunk-web-stories-template-0.js 10.6 kB 0 B
assets/js/chunk-web-stories-template-1-metaData.js 540 B 0 B
assets/js/chunk-web-stories-template-1.js 9.01 kB 0 B
assets/js/chunk-web-stories-template-10-metaData.js 533 B 0 B
assets/js/chunk-web-stories-template-10.js 6.91 kB 0 B
assets/js/chunk-web-stories-template-11-metaData.js 540 B 0 B
assets/js/chunk-web-stories-template-11.js 8.51 kB 0 B
assets/js/chunk-web-stories-template-12-metaData.js 496 B 0 B
assets/js/chunk-web-stories-template-12.js 9.48 kB 0 B
assets/js/chunk-web-stories-template-13-metaData.js 525 B 0 B
assets/js/chunk-web-stories-template-13.js 7.3 kB 0 B
assets/js/chunk-web-stories-template-14-metaData.js 582 B 0 B
assets/js/chunk-web-stories-template-14.js 7.58 kB 0 B
assets/js/chunk-web-stories-template-15-metaData.js 544 B 0 B
assets/js/chunk-web-stories-template-15.js 8.21 kB 0 B
assets/js/chunk-web-stories-template-16-metaData.js 588 B 0 B
assets/js/chunk-web-stories-template-16.js 10.3 kB 0 B
assets/js/chunk-web-stories-template-17-metaData.js 539 B 0 B
assets/js/chunk-web-stories-template-17.js 8.52 kB 0 B
assets/js/chunk-web-stories-template-18-metaData.js 585 B 0 B
assets/js/chunk-web-stories-template-18.js 9.05 kB 0 B
assets/js/chunk-web-stories-template-19-metaData.js 501 B 0 B
assets/js/chunk-web-stories-template-19.js 9.99 kB 0 B
assets/js/chunk-web-stories-template-2-metaData.js 586 B 0 B
assets/js/chunk-web-stories-template-2.js 9.16 kB 0 B
assets/js/chunk-web-stories-template-20-metaData.js 548 B 0 B
assets/js/chunk-web-stories-template-20.js 8.59 kB 0 B
assets/js/chunk-web-stories-template-21-metaData.js 534 B 0 B
assets/js/chunk-web-stories-template-21.js 9.16 kB 0 B
assets/js/chunk-web-stories-template-22-metaData.js 525 B 0 B
assets/js/chunk-web-stories-template-22.js 7.37 kB 0 B
assets/js/chunk-web-stories-template-23-metaData.js 605 B 0 B
assets/js/chunk-web-stories-template-23.js 6.99 kB 0 B
assets/js/chunk-web-stories-template-24-metaData.js 518 B 0 B
assets/js/chunk-web-stories-template-24.js 10.8 kB 0 B
assets/js/chunk-web-stories-template-25-metaData.js 544 B 0 B
assets/js/chunk-web-stories-template-25.js 7.07 kB 0 B
assets/js/chunk-web-stories-template-26-metaData.js 601 B 0 B
assets/js/chunk-web-stories-template-26.js 6.85 kB 0 B
assets/js/chunk-web-stories-template-27-metaData.js 543 B 0 B
assets/js/chunk-web-stories-template-27.js 7.36 kB 0 B
assets/js/chunk-web-stories-template-28-metaData.js 532 B 0 B
assets/js/chunk-web-stories-template-28.js 8.49 kB 0 B
assets/js/chunk-web-stories-template-29-metaData.js 561 B 0 B
assets/js/chunk-web-stories-template-29.js 8.49 kB 0 B
assets/js/chunk-web-stories-template-3-metaData.js 540 B 0 B
assets/js/chunk-web-stories-template-3.js 8.22 kB 0 B
assets/js/chunk-web-stories-template-30-metaData.js 576 B 0 B
assets/js/chunk-web-stories-template-30.js 7.67 kB 0 B
assets/js/chunk-web-stories-template-31-metaData.js 503 B 0 B
assets/js/chunk-web-stories-template-31.js 9.61 kB 0 B
assets/js/chunk-web-stories-template-32-metaData.js 551 B 0 B
assets/js/chunk-web-stories-template-32.js 12.2 kB 0 B
assets/js/chunk-web-stories-template-33-metaData.js 492 B 0 B
assets/js/chunk-web-stories-template-33.js 8.86 kB 0 B
assets/js/chunk-web-stories-template-34-metaData.js 571 B 0 B
assets/js/chunk-web-stories-template-34.js 7.57 kB 0 B
assets/js/chunk-web-stories-template-35-metaData.js 565 B 0 B
assets/js/chunk-web-stories-template-35.js 8.81 kB 0 B
assets/js/chunk-web-stories-template-36-metaData.js 576 B 0 B
assets/js/chunk-web-stories-template-36.js 11.6 kB 0 B
assets/js/chunk-web-stories-template-37-metaData.js 528 B 0 B
assets/js/chunk-web-stories-template-37.js 6.47 kB 0 B
assets/js/chunk-web-stories-template-38-metaData.js 572 B 0 B
assets/js/chunk-web-stories-template-38.js 7.96 kB 0 B
assets/js/chunk-web-stories-template-39-metaData.js 589 B 0 B
assets/js/chunk-web-stories-template-39.js 7.67 kB 0 B
assets/js/chunk-web-stories-template-4-metaData.js 565 B 0 B
assets/js/chunk-web-stories-template-4.js 11.5 kB 0 B
assets/js/chunk-web-stories-template-40-metaData.js 556 B 0 B
assets/js/chunk-web-stories-template-40.js 9.13 kB 0 B
assets/js/chunk-web-stories-template-41-metaData.js 572 B 0 B
assets/js/chunk-web-stories-template-41.js 7.75 kB 0 B
assets/js/chunk-web-stories-template-42-metaData.js 522 B 0 B
assets/js/chunk-web-stories-template-42.js 7 kB 0 B
assets/js/chunk-web-stories-template-43-metaData.js 558 B 0 B
assets/js/chunk-web-stories-template-43.js 8.37 kB 0 B
assets/js/chunk-web-stories-template-44-metaData.js 582 B 0 B
assets/js/chunk-web-stories-template-44.js 10.1 kB 0 B
assets/js/chunk-web-stories-template-45-metaData.js 564 B 0 B
assets/js/chunk-web-stories-template-45.js 7.12 kB 0 B
assets/js/chunk-web-stories-template-46-metaData.js 531 B 0 B
assets/js/chunk-web-stories-template-46.js 5.01 kB 0 B
assets/js/chunk-web-stories-template-47-metaData.js 592 B 0 B
assets/js/chunk-web-stories-template-47.js 8.46 kB 0 B
assets/js/chunk-web-stories-template-48-metaData.js 556 B 0 B
assets/js/chunk-web-stories-template-48.js 8.31 kB 0 B
assets/js/chunk-web-stories-template-49-metaData.js 518 B 0 B
assets/js/chunk-web-stories-template-49.js 9.7 kB 0 B
assets/js/chunk-web-stories-template-5-metaData.js 555 B 0 B
assets/js/chunk-web-stories-template-5.js 9.38 kB 0 B
assets/js/chunk-web-stories-template-50-metaData.js 504 B 0 B
assets/js/chunk-web-stories-template-50.js 8.26 kB 0 B
assets/js/chunk-web-stories-template-51-metaData.js 527 B 0 B
assets/js/chunk-web-stories-template-51.js 9.89 kB 0 B
assets/js/chunk-web-stories-template-52-metaData.js 602 B 0 B
assets/js/chunk-web-stories-template-52.js 10.1 kB 0 B
assets/js/chunk-web-stories-template-53-metaData.js 553 B 0 B
assets/js/chunk-web-stories-template-53.js 5.79 kB 0 B
assets/js/chunk-web-stories-template-54-metaData.js 547 B 0 B
assets/js/chunk-web-stories-template-54.js 7.52 kB 0 B
assets/js/chunk-web-stories-template-55-metaData.js 574 B 0 B
assets/js/chunk-web-stories-template-55.js 6.56 kB 0 B
assets/js/chunk-web-stories-template-56-metaData.js 543 B 0 B
assets/js/chunk-web-stories-template-56.js 9.5 kB 0 B
assets/js/chunk-web-stories-template-57-metaData.js 528 B 0 B
assets/js/chunk-web-stories-template-57.js 14.1 kB 0 B
assets/js/chunk-web-stories-template-58-metaData.js 556 B 0 B
assets/js/chunk-web-stories-template-58.js 5.61 kB 0 B
assets/js/chunk-web-stories-template-59-metaData.js 588 B 0 B
assets/js/chunk-web-stories-template-59.js 8.52 kB 0 B
assets/js/chunk-web-stories-template-6-metaData.js 569 B 0 B
assets/js/chunk-web-stories-template-6.js 7.04 kB 0 B
assets/js/chunk-web-stories-template-60-metaData.js 509 B 0 B
assets/js/chunk-web-stories-template-60.js 8.89 kB 0 B
assets/js/chunk-web-stories-template-7-metaData.js 569 B 0 B
assets/js/chunk-web-stories-template-7.js 7.21 kB 0 B
assets/js/chunk-web-stories-template-8-metaData.js 569 B 0 B
assets/js/chunk-web-stories-template-8.js 8.4 kB 0 B
assets/js/chunk-web-stories-template-9-metaData.js 581 B 0 B
assets/js/chunk-web-stories-template-9.js 8.49 kB 0 B
assets/js/chunk-web-stories-templates.js 443 B 0 B
assets/js/chunk-web-stories-textset-0.js 5.08 kB 0 B
assets/js/chunk-web-stories-textset-1.js 6.64 kB 0 B
assets/js/chunk-web-stories-textset-2.js 7.67 kB 0 B
assets/js/chunk-web-stories-textset-3.js 15.1 kB 0 B
assets/js/chunk-web-stories-textset-4.js 4.16 kB 0 B
assets/js/chunk-web-stories-textset-5.js 5.49 kB 0 B
assets/js/chunk-web-stories-textset-6.js 5.3 kB 0 B
assets/js/chunk-web-stories-textset-7.js 10.2 kB 0 B
assets/js/generateBlurhash.worker.worker.js 1.1 kB 0 B
assets/js/imgareaselect.js 3.77 kB 0 B
assets/js/lightbox.js 550 B 0 B
assets/js/tinymce-button.js 2.84 kB 0 B
assets/js/web-stories-activation-notice.js 26.9 kB 0 B
assets/js/web-stories-block.js 22.7 kB 0 B
assets/js/web-stories-embed.js 20 B 0 B
assets/js/web-stories-widget.js 587 B 0 B
assets/js/wp-dashboard.js 72 kB 0 B

compressed-size-action

@googleforcreators-bot
Copy link
Collaborator

googleforcreators-bot commented Jul 1, 2022

Plugin builds for 7fd25aa are ready 🛎️!

@spacedmonkey spacedmonkey requested a review from miina July 4, 2022 13:54
Copy link
Contributor

@miina miina left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

(Looks like e2e tests didn't run, not sure if that's expected or should be restarted)

Copy link
Contributor

@merapi merapi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, just one thing to fix:

width and height passed to BlurhashContainer is taken from the original image (big one) so the canvas blurhash looks different from the thumbnail blurhash.

I think we can remove the width and height props from the BlurhashContainer as they are optional and the parent will take care of the rest, or use width=100%, height=auto.

Example (1st load - current, 2nd - fixed):

blur.mp4

@spacedmonkey
Copy link
Contributor Author

width and height passed to BlurhashContainer is taken from the original image (big one) so the canvas blurhash looks different from the thumbnail blurhash.

@merapi The height and width are required here. Depending on these, the blurhash looks different. But you are correct that the height and width of the resource are the wrong. I have tweaked the logic to use the size of the element, which improves performance and looks a lot better.

@merapi
Copy link
Contributor

merapi commented Jul 6, 2022

@merapi The height and width are required here. Depending on these, the blurhash looks different. But you are correct that the height and width of the resource are the wrong. I have tweaked the logic to use the size of the element, which improves performance and looks a lot better.

Looks like the docs are not updated? it defaults to auto.
https://github.com/woltapp/react-blurhash/blob/master/src/Blurhash.tsx#L8

It is a bit misleading but width/height is only used in div style and what actually impacts performance is the resolution props:

<div
  {...rest}
  style={{ display: 'inline-block', height, width, ...style, position: 'relative' }}
>
  <BlurhashCanvas
    hash={hash}
    height={resolutionY}
    width={resolutionX}
    punch={punch}
    style={canvasStyle}
  />
</div>

Either solution will make it look correct 👍🏻

@spacedmonkey
Copy link
Contributor Author

The height and width are used to generate the canvas. See https://github.com/woltapp/react-blurhash/blob/master/src/BlurhashCanvas.tsx

@merapi
Copy link
Contributor

merapi commented Jul 6, 2022

The height and width are used to generate the canvas. See https://github.com/woltapp/react-blurhash/blob/master/src/BlurhashCanvas.tsx

Nope, we are not using BlurhashCanvas, we use Blurhash.

@spacedmonkey
Copy link
Contributor Author

Nope, we are not using BlurhashCanvas, we use Blurhash.

But blurhash does blurhashcanvas - https://github.com/woltapp/react-blurhash/blob/458b44880845085066e5640bdc5a24a51f077002/src/Blurhash.tsx#L53-L59

@merapi
Copy link
Contributor

merapi commented Jul 7, 2022

Nope, we are not using BlurhashCanvas, we use Blurhash.

But blurhash does blurhashcanvas - https://github.com/woltapp/react-blurhash/blob/458b44880845085066e5640bdc5a24a51f077002/src/Blurhash.tsx#L53-L59

Correct, but look how it passes the props.

image

@spacedmonkey
Copy link
Contributor Author

Correct, but look how it passes the props.

Right, I have passed x and y props.

@merapi
Copy link
Contributor

merapi commented Jul 7, 2022

Correct, but look how it passes the props.

Right, I have passed x and y props.

That is unnecessary (actually degrading the performance).

The X-axis resolution in which the decoded image will be rendered at. Recommended min. 32px. Large sizes (>128px) will greatly decrease rendering performance. (Default: 32)

Right now my test image is rendered with resolution=336, it takes ~62ms to render, while with recommended/default resolution=32 it takes ~1ms.

I have a little tool for performance testing and there is no point to go above >32 in resolution because there is no visual difference, it just takes longer to render (we encode the image with components=4 so there is not much data to work with).

image

So the best settings are: resolution=32 (default), width/height can be undefined/auto or actual box dimensions - no difference.

@spacedmonkey
Copy link
Contributor Author

So the best settings are: resolution=32 (default), width/height can be undefined/auto or actual box dimensions - no difference.

Height and width have been removed.

@swissspidy swissspidy merged commit 7029aeb into main Jul 11, 2022
@swissspidy swissspidy deleted the try/base-color-canvas branch July 11, 2022 10:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Elements: GIF GIF element as used by Tenor 3P Media integration Elements: Image Elements: Video Group: Canvas Group: Media P2 Should do soon Type: Enhancement New feature or improvement of an existing feature
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Canvas: Show base color / blurhash while resource is loading
6 participants