First-party skybox support

[james7132]

What problem does this solve or what need does it fill?

A single color background is boring for 3D environments. In other engines like Unity and Unreal, there's an effect that shows either a set of textures or something procedurally generated on the infinite horizon. It'd be nice to have this in Bevy.

This can also be combined with environment light maps for a nicer combined effect.

What solution would you like?

A first-party solution that adds this at the very beginning of the render graph.

In Bevy 0.5, @jomala wrote a bevy_skybox crate, though it seems like it wasn't published to crates.io.

What alternative(s) have you considered?

Revive the 0.5 ecosystem crate.

[mockersf]

Would https://github.com/bevyengine/bevy/blob/main/examples/3d/skybox.rs work if that feature was brought in engine rather than in an example?

[IceSentry]

If it's a first party integration, there should also be some way to sync the environment map light texture with the skybox. It's already mentioned, I somehow skipped that part when reading the issue.

As for third_party there's also bevy_atmosphere that is really nice

[JMS55]

To start, we should provide a way to have a user-provided cubemap background image (skybox).

bevy_atmosphere would be great to have integrated as well, but it's not necessary for a first step.

Generating EnvironmentMapLight textures from a given texture (such as, automatically once you set a skybox) would also be nice to have, but not necessary as a first step.

[james7132]

Pinging @JonahPlusPlus about this. Not sure how much of bevy_atmosphere we can upstream, but having it be first-party, well integrated with rest of the first-party renderer, and supporting the base case for user-provided cubemaps seems like a plan to me.

[JMS55]

I did some research on how to best render a skybox. Here's what I learned:

• Draw a fullscreen triangle at the end of the opaque pass (or maybe alpha pass?)
  • The reason to draw at the end is to reduce overdraw - we only have to shade pixels that haven't been shaded and written to the depth buffer
• Before drawing, set the viewport to min_depth = max_depth = 1 (or maybe 0, whichever is the furthest depth)
  • This forces the triangle to be written behind everything else
• We can put the skybox + sampler under a separate bind group, and use a separate non-material pipeline, and switch to that before drawing the skybox rather then stick it into the view bind group and use the material API.
• If we split up the main pass node, we could add the skybox as a separate node between the opaque node and the next, in a separate plugin, which would be more modular.

[JonahPlusPlus]

I'm open to the possibility of bevy_atmosphere being upstreamed. I could see it being part of a 3dPlugins group, alongside plugins for 3D terrain, physics, etc.

For those unfamiliar with bevy_atmosphere, it's a complex system for procedural atmospheres (so currently just the sky, but hopefully other phenomena as well). 0.6 will bring a new layer system, which will allow for combining shaders (for example, overlaying a cloud or starmap texture over a procedural sky). The layer system will just be another model, meaning it's completely optional and you could even have layers within layers (performance permitting). It's capable of all this by rendering the entire sky at once when settings change. This does mean it's not aimed at rapid changes, but I have set the groundwork for an update to change this.

Support for cubemaps is something I'll investigating. With the current asset system, processing images to be in the proper format is cumbersome, but I might have a solution.

[JonahPlusPlus]

One issue would be WebGL compatibility. WebGL doesn't have compute shaders, which bevy_atmosphere depends on. It might be necessary to add a fallback when compiling for browsers.

Also, it currently uses a cube that fits into the camera far plane. If there are any tricks with the render graph to make it "clear" with the sky texture instead, I would love to move to that instead.

[JMS55]

One issue would be WebGL compatibility. WebGL doesn't have compute shaders, which bevy_atmosphere depends on. It might be necessary to add a fallback when compiling for browsers.

We can make procedural atmospheres a non-webgl only feature.

Also, it currently uses a cube that fits into the camera far plane. If there are any tricks with the render graph to make it "clear" with the sky texture instead, I would love to move to that instead.

Read my comment above for a good way to do this. Tldr full screen triangle, set the viewport, run at the end of the opaque pass for maximum speed.

[jomala]

It's embedding that agreed best approach to when and how to fill in the background that I'd love to see as part of Bevy. If it can remain flexible enough about what to fill it in with that it supports skyboxes and compute shaders, that'd be perfect.

Once you have the concept of "the camera is always at 0,0,0, and ignore the depths, this is background", I'm not sure how much more it makes sense to upstream. bevy_skybox is very rough-and-ready, but that's because most of it is heuristics for parsing images from the internet. The basic object of an inward-facing cube with texture on it is pretty simple to implement.

[james7132]

• If we split up the main pass node, we could add the skybox as a separate node between the opaque node and the next, in a separate plugin, which would be more modular.

We already had a few attempts/investigations into this already. I like this approach.

[JMS55]

I've started working on this, but run into the issue that I need the main pass 3d node split up, so I can insert a skybox node in between the opaque/alpha and transparent phases.