Using a custom CameraProjection and directional light shadows crash #9077

Bevy version

0.10.1 but I'm pretty sure it's the case for 0.11/main too

What you did

To opt-out of automatic Frustum definition, in bevy_basic_portals, I defined a custom component implementing the trait CameraProjection.
I added the plugin CameraProjectionPlugin::<PortalProjection>::default(), I voluntarily didn't add the system update_frusta::<PortalProjection>.
But I couldn't do anything about update_directional_light_cascades and extract_taa_settings because they're not generic over the projection.

How to trigger the bug

For directional light, predictively, if you have a DirectionalLight with shadows_enabled = true and a camera with a custom CameraProjection component, Bevy crashes.

More specifically, it happens for an entity with a Camera3d component but no Projection, could happen by using PerspectiveProjection or OrthographicProjection directly too, not using Camera3dBundle.

What happens exactly

What happens is that update_directional_light_cascades doesn't add our camera to Cascades's cascades (of type HashMap<Entity, Vec<Cascade>>, the entities are cameras), this cascades is given to ExtractedDirectionalLight, then in prepare_lights (line 1106 of bevy_pbr/src/render/light.rs on main), it tries to get our camera from this hashset, unwraps, and panics.

A similar thing might happen for TAA, since extract_taa_settings is not generic either, I didn't test it.

Why I think this should be fixed

Even if defining a custom projection is not something usual, it has its uses.
For my previous usage of defining a custom Frustum I could still use Bevy's Projection and just fight against update_frusta, but I also wanted to use a custom projection in the future of bevy_basic_portals to define a projection to the portal destination directly instead of doing the "render to image + masking" technique, I will need a custom projection for that.
Bevy is supposed to fit all usages and be modular.

How to fix it

Here are some possibilities/thoughts to fix this:

1. add a generic method to calculate the corners of the Frustum in the trait CameraProjection to do it generically in update_directional_light_cascades and replace frustum_corners/frustum_corners_ortho
2. Make update_directional_light_cascades generic like update_frusta and camera_system, this is a bit annoying since this system clears the Cascades, but it may be possible to get around this using a resource to only clear once per tick or a different system for clearing which wouldn't be generic and run before update_directional_light_cascades.
3. Make PbrPlugin (which adds update_directional_light_cascades) generic like CameraProjectionPlugin? Probably a bad idea, but maybe it can be possible to make a sub-plugin that would be generic to add update_directional_light_cascades, but it's probably easier for people who want a custom projection to add update_directional_light_cascades directly anyway (?)
4. Instead of adding a component that implements CameraProjection directly, we could add a component which is a box of a CameraProjection, then do everything on that component, this would remove the need to make all the systems generic over CameraProjection and then having to add the systems for every kind of projection one by one, at the cost of doing dynamic typing (which is probably less optimized)
5. Same as 4., but only for custom projections, like either you use Bevy's default projection, or you use a custom projection in a box
6. Don't do an unwrap in prepare_lights. If no entity is found, just ignore it and continue.

I might edit later if I have other ideas.
I didn't check how TAA works.
I can probably do it myself if nobody else wants to but I would appreciate insights before jumping into the code.