Repository navigation
Cleanup core pipeline core_3d render nodes/order/options/etc #7981
Description
After working on bevy for a while, I've noticed some papercuts with the core_3d passes/components. Let's discuss how to improve it :)
What problem does this solve or what need does it fill?
- It's easy to accidentally get the order of post process nodes wrong. You must order it after the main pass, and before tonemapping, but also order the pass before/after other specific post processing passes. Otherwise, you end up with TAA running before bloom by accident. It's too easy to underconstrain the node orders
- The "tonemap in shader" (main pass) default is kinda a bad idea. Tonemapping should always go last, after/during post processing. Currently this only happens if you enable
hdr. - The "upscale" pipeline (blit main texture to swapchain) is pointless. I'm not sure what the original intention behind naming it upscaling is, but for TSR (temporal upscaling) techniques like FSR/DLSS, they should run where TAA goes - right after the main pass, before any other post processing.
- Most people will probably want to enable hdr, but it's off by default.
- After bevy 0.10, we have better tonemapping options. Lets change the default tonemapper.
What solution would you like?
- Define a set of 3 pass labels (PREPASS, MAIN_PASS, POST_PROCESS), that run in that order.
- Within POST_PROCESS, define a strict ordering (setup as part of Core3dPlugin, rather than each individual plugin needing to set their order properly) as follows: (TAA, BLOOM, TONEMAPPING).
- Tonemapping will write directly to the swapchain, and combines/replaces the previous tonemap/upscale nodes
- I'm unsure where FXAA should go. FXAA wants to go after tonemapping, but we want to setup tonemapping to write to the swapchain. Perhaps we have an abstraction in ViewTarget where the last pass always writes directly to the swapchain. That, or special case tonemapping to write to either the swapchain or ViewTarget depending on if FXAA exists.
- Remove the tonemap in shader option, or at least make it off by default unless you enable a setting. The default should be a separate tonemap node for better compatibility with post processing.
- A hypothetical future
LowEndMobileCamera3dBundlecould add this component to enable the setting and cut down on render passes
- A hypothetical future
Camera { hdr }should be on by default, and renamed to something better. It has nothing to do with hdr monitors / output - it's just what texture format bevy uses internally during render passes.- On wasm/webgl2, hdr has issues, I think in combination with MSAA. We can either turn it off by default on the web until webgpu releases, replace MSAA with FXAA as our default AA method, or something else.
Reacted by Alice Cecile
Activity
DGriffin91 commentedon Mar 8, 2023
Ideally I think FXAA is supposed to be after tonemapping. I think it's also probably ok to have FXAA before tonemapping if that ends up being easier. Mostly the user settings just have to be slightly tweaked. iirc we did some testing previously and it work ok in either position. Though we could test more. We could also do reversible tonemapping, which we probably should be doing to make MSAA work correctly too.
Could be most efficient to put FXAA in the "upscale" (as it's currently called) pass with reversible tonemapping, then output into color grading/actual tonemapping, that way the extra pass is avoided.
IceSentry commentedon Mar 8, 2023
It was recently asked if it would be possible to insert a node between the opaque and transparent phase of the main pass. I'm not sure if this fits here because it might be a bit more involved, but I felt it was worth mentioning.
JMS55 commentedon Mar 14, 2023
So, here's an update on how discussion has progressed, in order from easiest/most fleshed out to hardest/needs more discussion. The first ~5 are good first issues if anyone is looking to help out!:
START_MAIN_PASS_POST_PROCESSINGlabel in addition to the end label. Maybe we should also rename it to not include "main_pass".bevy_core_pipeline/src/core_3d/mod.rsor a separate postprocessing submodule or something. Currently, FXAA, bloom, and upcoming features do the ordering themselves in their respective plugins, which is very easy to break from my experience.cart commentedon Mar 20, 2023
Not necessarily. Notably the HypeHype mobile renderer does one "separate" post processing pass that reads from the intermediate "main texture" and does a single unified post processing pass that writes to the swapchain.
Imo we should build a high level "post processing effect" abstraction that generates shader snippets that can get combined into a single post processing shader. We would limit mobile to those effects (at least by default), but it could still be composed with more advanced / unconstrained multi-pass effect chains on desktop.
JMS55 commentedon Jun 11, 2023
Closing this for now. Some of these ideas have gotten implemented, others we've since come up with better or different ideas.