Add antialiasing filters (FXAA, SSAA)

[x2048]

Adds FXAA and SSAA post-processing filter controlled by antialiasing setting.

FXAA is based on WebGL code circulating on the internet, MIT-licensed.

SSAA is a simple render to 2x the texture sixe + 3x3 tent filter using bilinear filtering.

To do

This PR is Ready for Review.

How to test

1. antialiasing = fxaa

2. Look around

3. There should be less pixelation

4. antialiasing = ssaa

5. Look around

6. There should be even less pixelation and visual noise

[kilbith]

FXAA disabled

FXAA enabled

---

FXAA disabled

FXAA enabled

[AbduSharif]

How is the performance difference between FXAA and SSAA?

[rubenwardy]

How is the performance difference between FXAA and SSAA?

SSAA is simpler and accurate but is slow, as it renders multiple times to get multiple samples.
FXAA is faster

[AbduSharif]

Yeah, SSAA was rather expensive on my Android device, it doubled the drawtime and lowered FPS from 60 (without) to ~25 in the same area, other settings were the same when testing with and without.

[x2048]

Impact varies by hardware, here is an example for Core i5 3317U (a 10yo laptop)

Setting	FPS
none	50
fxaa	44
ssaa	20

[kilbith]

On my GPU (RTX 3050), the impact on FPS with SSAA is marginal (-4 FPS at worst), but the quality is second to none.

[lhofhansl]

On my "NVIDIA GeForce GTX 1650 Max-Q":

• the impact of FXAA is not measurable. I get exactly the same FPS.
• SSAA loses about 25% FPS.

Nice!

[lhofhansl]

Tested in various scenarios. Works fine.

Code looks reasonable (but I am not an expert in this)

[lhofhansl]

QQ: How does this all relate to the "Antialiasing" setting on the main settings page?

[x2048]

That setting controls built-in / "hardware" which only works partially and in direct pipeline without postprocessing (which is coupled to shaders)

[AbduSharif]

Is the SSAA shader set to 2X or 4X?

[rubenwardy]

That setting controls built-in / "hardware" which only works partially and in direct pipeline without postprocessing (which is coupled to shaders)

I suggest removing that feature or using the same setting for it

[x2048]

SSAA is at 2x

[AbduSharif]

Can it be increased to 4X through the shader or not? If not, can we have it available to tweak, at least from the shader code if you don't want to expose it?

[lhofhansl]

Hmm... The conflicting fixed-pipeline (right?) antialiasing on the settingspage is indeed confusing. It would indeed be nice if we somehow could have one setting. At least remove it from the main settingspage, as there is nothing special about it anymore.

@AbduSharif 2x is already comparatively slow. Do you expect a great visual improvement from this?

[AbduSharif]

Yes I do expect a visual improvement from it, the slowness I had was during a worst case scenario only and it works well other times for me.

Having the option wouldn't hurt as the shader is already labled as "(expensive)".

[lhofhansl]

(Removed the "One Approval" as there is still work needed)

[rubenwardy]

If enable_fsaa requires post-processing to be disabled, should we disable the dropdown for 5.7.0 when post-processing enabled?

[numberZero]

SSAA is a simple render to 2x the texture sixe
SSAA is at 2x

To my understanding that’s 2× in each direction, so 4× samples.

3x3 tent filter using bilinear filtering.

Won’t that make the image a bit blurry? I’d expect simple averaging (which is equivalent to bilinear sampling for 2× actually)

Anyway, did you try using multisampling with per-sample shading? That should work better than equivalent SSAA, and is a bit more flexible.

[Calinou]

Anyway, did you try using multisampling with per-sample shading? That should work better than equivalent SSAA, and is a bit more flexible.

You can try allowing the usage of 16× MSAA, which is implemented as 4× MSAA + 4× SSAA in most graphics drivers. (Despite the name, there is no such thing as actual 16× MSAA; it's always a mix of supersampling and multisampling.)
Right now, the maximum value you can set in the settings menu is 8×.

However, I've noticed that forcing 16× MSAA in the graphics driver (like in #13397) makes 2D elements in Minetest blurry (including text). This occurs with some other games too, but not all of them.

[Iniquitatis]

I'm probably missing something, but for simple 2x SSAA implementation I'd expect something like:

	vec4 color = 0.25 * (
			texture2D(rendered, uv + vec2(0.0, 0.0) * texelSize0).rgba +
			texture2D(rendered, uv + vec2(0.5, 0.0) * texelSize0).rgba +
			texture2D(rendered, uv + vec2(0.5, 0.5) * texelSize0).rgba +
			texture2D(rendered, uv + vec2(0.0, 0.5) * texelSize0).rgba);

(Maybe with half-pixel offset.)
Is there a reason you've decided to go with this "plus" shape?

[numberZero]

There is no “plus” shape. The samples are in square corners, spaced by 0.5*texelSize0, but centered at uv unlike your sample. Assuming texelSize0 is the size of destination pixel, the code is correct.

OTOH it could leverage built-in linear filtering and take only 2 samples, at ±{0, 0.25} (or ±{0.25, 0}) of texelSize0 1 sample at uv exactly, that’d be the same. Not sure does Irrlicht support that currently, I think I saw some code that disables filtration for RTT textures. Or whether it makes any sense at all with actual hardware.

Or it could do texelFetches instead of sampling. Using maybe 2 * gl_FragCoord for coordinates.

[numberZero]

Wait, I miss something. It seems the code in secondstage.cpp enables linear filter but this shader still does 4 samples at locations that are either exact texel centers (so the filter doesn’t add anything) or weird. @x2048 ?

[Iniquitatis]

There is no “plus” shape.

Brain fart. Somehow I thought that these offsets are like (0.25, 0.0), (0.0, 0.25), etc.

Note to self: Don't ask questions before your morning coffee.

[x2048]

@numberZero I've changed the shader to make full use of bilinear filtering. For 2x2 it will make a single sample.

[AbduSharif]

Thanks for adding in the configuration possibility.

[x2048]

The settings have been changed as follows:
antialiasing setting now selects the method (none/disabled, FSAA/fixed pipeline, FXAA, SSAA)
fsaa now selects the scale/number of samples

@Calinou I don't have nVidia so cannot test driver-forced antialiasing. I guess blurring of the UI occurs because AA filter is applied to entire screen including UI. Here we apply filter to 3D scene only (not even the wield item). The config now allows SSAA 2x2, 4x4, 8x8 and 16x16 (insane).

[x2048]

Also, there is a configuration option already for MSAA at 16x16, but it has no effect on my hardware (maybe, nVidia works).

[lhofhansl]

This looks good to me now.

[lhofhansl]

fsaa does not do anything, it seems like that is expected. ssaa reduces my FPS to almost 1/2.

[x2048]

fsaa works when shaders are off (and the other options don't)

[lhofhansl]

I maintain my "One approval"

[lhofhansl]

What's holding this one?

[sfan5]

@x2048 if this is considered ready feel free to merge

[HybridDog]

3x3 tent filter using bilinear filtering.

Won’t that make the image a bit blurry? I’d expect simple averaging (which is equivalent to bilinear sampling for 2× actually)

I'm probably missing something, but for simple 2x SSAA implementation I'd expect something like: […]

As far as I know, that would be box filtering or pixel mixing and is not the same as bilinear filtering.
https://entropymine.com/imageworsener/pixelmixing/

A related issue about SSAA with SSIM-based perceptual downscaling: #6976