Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add SSAA with SSIM-based perceptual downscaling #13959

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

HybridDog
Copy link
Contributor

@HybridDog HybridDog commented Nov 5, 2023

Super-sampling anti-aliasing with a conventional downscaling algorithm smoothes the image, which is a disadvantage compared to disabled anti-aliasing. The SSIM-based perceptual downscaling algorithm tackles this problem and it parallelises well on the GPU.

The new SSAA method can be enabled with the anti aliasing setting and it is the slowest anti-aliasing technique now.

aa_comparison
additional_images.7z.zip

For more pictures and information about the algorithm, see #6976 and the Perceptually Based Downscaling of Images paper.

Closes #6976
Related roadmap goal: 2.1 Rendering/Graphics improvements

How to test

On GNU/Linux with X11, it is possible to render everything shown on the screen with a smaller resolution and upscale it with nearest neighbour scaling by an integer factor: xrandr --output CHANGE_THIS --transform 0.5,0,0,0,0.5,0,0,0,1 --filter nearest (here it's 2x upscaling).
This may help testing the anti-aliasing on a high resolution screen.

In Minetest the new SSAA method can be tested by setting the anti-aliasing setting to antialiasing = ssaa_ssim_based and playing around with different scales (fsaa setting).
By setting gl_FragColor = vec4(ycbcr_to_linear_srgb(col_l), 1.0); in client/shaders/ssaa_ssim_based_3/opengl_fragment.glsl, the algorithm behaves like the current smooth SSAA in Minetest except that it uses gamma-correct scaling.
Note that the new SSAA method also preserves details which may be undesired, for example Moire artifacts if mip mapping is disabled.

Strange behaviour with tonemapping

If tonemapping is enabled, strange artifacts appear with testnodes:generated_png_ck:
screenshot_20231214_183454
With other nodes or without tonemapping, it looks fine.
Perhaps the problem wouldn't appear if the downscaling was applied after tonemapping instead of before.
We can ignore this behaviour because tonemapping in Minetest is a questionable feature anyway as the tonemapping input is already in SDR.

@Zughy Zughy added Roadmap The change matches an item on the current roadmap Feature ✨ PRs that add or enhance a feature @ Client rendering labels Nov 5, 2023
@HybridDog HybridDog force-pushed the m_oversampling_new branch 3 times, most recently from 3466f1d to 143c1ff Compare November 6, 2023 18:43
@HybridDog HybridDog marked this pull request as ready for review November 6, 2023 18:43
@HybridDog HybridDog force-pushed the m_oversampling_new branch from 9330846 to 7f5aba3 Compare March 30, 2024 10:32
@SmallJoker SmallJoker added the Rebase needed The PR needs to be rebased by its author label Dec 29, 2024
@Zughy
Copy link
Contributor

Zughy commented Dec 29, 2024

@HybridDog rebase needed

Super-sampling anti-aliasing with a conventional downscaling algorithm
smoothes the image, which is a disadvantage compared to disabled
anti-aliasing.
The SSIM-based perceptual downscaling algorithm tackles this problem and
it parallelises well on the GPU.

The new SSAA method can be enabled with the anti aliasing setting and it
is the slowest anti-aliasing technique now.
@SmallJoker SmallJoker removed the Rebase needed The PR needs to be rebased by its author label Jan 3, 2025
@HybridDog
Copy link
Contributor Author

I have rebased it; thanks for the notification.

Recently I found out that there's a patent associated with the paper which belongs to the Percim company I think. I still don't understand what exactly is patented and what this means for this Pull Request. The downscaling method explained in the paper is not a single algorithm but an idea which encompasses many ways to implement a downscaling algorithm.
The patent: https://patentimages.storage.googleapis.com/ad/3d/41/438150ce88bfba/WO2017017584A1.pdf
Percim: https://percim.com/#intro
Comments about the patent: https://gist.github.com/HybridDog/dd95a99d411972f030fb18543280ad60

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
@ Client rendering Feature ✨ PRs that add or enhance a feature Roadmap The change matches an item on the current roadmap
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Oversampling without smoothing
3 participants