Ambient occlusion and dynlight shaders with point light math

Moderator: Graf Zahl

dpJudas
Developer
Developer
Posts: 798
Joined: Sat Jul 23, 2016 7:53

Re: Ambient occlusion and dynlight shaders with point light math

Post by dpJudas » Fri Oct 14, 2016 18:53

The more I think about the portal issues, the more I'm reaching the conclusion that my current solution is never going to work well. I probably have to change the portal rendering so that the full opaque scene is rendered first, then ssao, then translucent on top. If this can be achieved it also opens up other possibilities such as doing deferred lighting for the dynamic lights and so on.

While looking at the code related to stencils I'm noticing there are two classes doing stencil things. There's GLPortal and then there is FDrawInfo. What is the second one using it for?

I'd like to change the code so that each portal is filled with its own stencil value. From what I can tell it is already sort of doing this. What I'd like to change is that it does this the same way as the software renderer: draw opaque stuff first while marking wall and flat portal windows. By using depth tests it should be possible to mark the exact shape of the portals, giving each portal its own unique stencil reference value.

Then, in the second step it would grab the first portal, enable stencil test for the portal's reference value and draw its opaque scene. Then mark child portal walls and flats in it with its own stencil values. Continue recursively like this until the entire scene has been drawn with depth data intact and all portals visible in the stencil buffer at the same time.

At this point it will do the SSAO and put that on top of the scene.

Drawing the translucent stuff is done after this by drawing the portals back to front. This should be possible to do by filling each child portal window with the stencil value of its parent while filling in depth data to seal a child (same algorithm as it uses today for the entire scene - we just postponed the fill basically). That will allow all translucent stuff to clip correctly.

Do you see any problems with this approach?

User avatar
Graf Zahl
GZDoom Developer
GZDoom Developer
Posts: 7148
Joined: Wed Jul 20, 2005 9:48
Location: Germany
Contact:

Re: Ambient occlusion and dynlight shaders with point light math

Post by Graf Zahl » Fri Oct 14, 2016 20:26

The stencils in FDrawInfo are used to flood wall parts with flats where upper or lower textures are missing. This emulates some exploit in the software renderer.

Regarding your approach, I have my doubts that this will work. The portal stencil values are not sequential, when doing two portals in the same scene they are drawn sequentially, and most likely use the same stencil value beause the first one will be completely gone before the second one starts and they are starting from the same position.

dpJudas
Developer
Developer
Posts: 798
Joined: Sat Jul 23, 2016 7:53

Re: Ambient occlusion and dynlight shaders with point light math

Post by dpJudas » Sat Oct 15, 2016 9:50

Yes, right now it does not use a sequential value (as it just uses the recursion count as its reference value), but the idea is to change it so that it does. That puts an upper limit of 255 rendered portals per scene, but I'm assuming no map ever uses that many at the same time.

User avatar
Nash
Developer
Developer
Posts: 1226
Joined: Sun Sep 25, 2005 1:49
Location: Kuala Lumpur, Malaysia
Contact:

Re: Ambient occlusion and dynlight shaders with point light math

Post by Nash » Sat Oct 15, 2016 20:39

Noticed another thing: with gl_ssao_portals 0, models only casts shadows on to the world, but not on itself. The model itself is fully white when viewed in debug mode.

User avatar
Graf Zahl
GZDoom Developer
GZDoom Developer
Posts: 7148
Joined: Wed Jul 20, 2005 9:48
Location: Germany
Contact:

Re: Ambient occlusion and dynlight shaders with point light math

Post by Graf Zahl » Sat Oct 15, 2016 20:48

Models do not have normals yet, so this is not surprising.

User avatar
Nash
Developer
Developer
Posts: 1226
Joined: Sun Sep 25, 2005 1:49
Location: Kuala Lumpur, Malaysia
Contact:

Re: Ambient occlusion and dynlight shaders with point light math

Post by Nash » Sun Oct 16, 2016 8:11

Understood, although I find it strange that if gl_ssao_portals > 0 the models will be shaded with SSAO. The data has to come from somewhere, otherwise they wouldn't have been shaded... :O

dpJudas
Developer
Developer
Posts: 798
Joined: Sat Jul 23, 2016 7:53

Re: Ambient occlusion and dynlight shaders with point light math

Post by dpJudas » Sun Oct 16, 2016 9:01

gl_ssao_portals can't conjure up normals for the models(!)

User avatar
Rachael
Developer
Developer
Posts: 3616
Joined: Sat May 13, 2006 10:30

Re: Ambient occlusion and dynlight shaders with point light math

Post by Rachael » Sun Oct 16, 2016 9:08

That awkward moment when your code starts having a mind of its own...
Spoiler: Zen Sarcasm

User avatar
Nash
Developer
Developer
Posts: 1226
Joined: Sun Sep 25, 2005 1:49
Location: Kuala Lumpur, Malaysia
Contact:

Re: Ambient occlusion and dynlight shaders with point light math

Post by Nash » Sun Oct 16, 2016 9:12

dpJudas I couldn't tell if that was a joke or you really weren't able to reproduce it... XD But just in case it's the latter - you can trigger it by being inside a sector with a sector skybox, turning on gl_ssao_portals to something non-zero and spawning a model in the open. Tormentor's Blade of Agony's first map would be the best place to quickly test this since I assume you already have it on hand for testing purposes.

dpJudas
Developer
Developer
Posts: 798
Joined: Sat Jul 23, 2016 7:53

Re: Ambient occlusion and dynlight shaders with point light math

Post by dpJudas » Sun Oct 16, 2016 9:25

Haha eruanna - we've all been there haven't we? :D

@Nash: I didn't test it because what you are describing sounds really really weird. If it really does begin to shade the model, it must be because it is now getting some bonkers normals from somewhere else (maybe the portal sealing off), but at the same time with still depth data from the other side of the portal. Whatever you are seeing, I can't be the correct normals. :)

User avatar
Nash
Developer
Developer
Posts: 1226
Joined: Sun Sep 25, 2005 1:49
Location: Kuala Lumpur, Malaysia
Contact:

Re: Ambient occlusion and dynlight shaders with point light math

Post by Nash » Sun Oct 16, 2016 10:39

Image
Image

0 portals and 1 portal. With portals, the skybox's floor seems to be bleeding through the play area's floor. The general SSAO of the entire scene seems to be negative. :O

Image

When I fly up a bit so that the bleeding skybox floor is drawn on top of the car, it appears that the car's shading starts to look more "correct" (nevermind the fact that there shouldn't even be any shading on the model in the first place ;))

dpJudas
Developer
Developer
Posts: 798
Joined: Sat Jul 23, 2016 7:53

Re: Ambient occlusion and dynlight shaders with point light math

Post by dpJudas » Fri Oct 21, 2016 23:18

Okay, I've tracked down the source of those portal rendering issues and fixed it. There are still some minor visible seams at the portal entrances, but I don't see a realistic way to deal with that unless the general portal algorithm is somehow changed to allow a full opaque render before translucent things.

Nash: if you could give this another test spin that would be great. Hopefully this will be the last issue.

User avatar
Rachael
Developer
Developer
Posts: 3616
Joined: Sat May 13, 2006 10:30

Re: Ambient occlusion and dynlight shaders with point light math

Post by Rachael » Fri Oct 21, 2016 23:43

Build 0.2pre-36 up just for this test.
Spoiler: Zen Sarcasm

User avatar
Nash
Developer
Developer
Posts: 1226
Joined: Sun Sep 25, 2005 1:49
Location: Kuala Lumpur, Malaysia
Contact:

Re: Ambient occlusion and dynlight shaders with point light math

Post by Nash » Fri Oct 21, 2016 23:55

I got hit by a nasty sudden fever so I'm afraid there won't be any gaming for me this weekend... but I'll be sure to get to testing ASAp, as soon as my bones don't feel like glass...

User avatar
Rachael
Developer
Developer
Posts: 3616
Joined: Sat May 13, 2006 10:30

Re: Ambient occlusion and dynlight shaders with point light math

Post by Rachael » Fri Oct 21, 2016 23:58

Ouch! I hope you feel better soon.
Spoiler: Zen Sarcasm

Locked

Return to “Closed Feature Suggestions”