SSAO discussion

Advanced OpenGL source port fork from ZDoom, picking up where ZDoomGL left off.
[Home] [Download] [Git builds (Win)] [Git builds (Mac)] [Wiki] [Repo] [Bugs&Suggestions]

Moderator: Graf Zahl

User avatar
Rachael
Developer
Developer
Posts: 3645
Joined: Sat May 13, 2006 10:30

SSAO discussion

Post by Rachael »

Starting this thread as a split from this one where Nash asked about SSAO.

Before we get too much into it, one problem we'll have with SSAO is that GZDoom is rendering sprites to the screen - which are meant to resemble solid objects. With standard SSAO filters that may not be too much of a problem but with the HBAO+ filter will actually use the geometry presented to the GPU.
User avatar
Nash
Developer
Developer
Posts: 1226
Joined: Sun Sep 25, 2005 1:49
Location: Kuala Lumpur, Malaysia
Contact:

Re: SSAO discussion

Post by Nash »

So does that mean sprites will end up having rectangular shadows instead of whatever pixel content is in the sprite? :S
User avatar
Rachael
Developer
Developer
Posts: 3645
Joined: Sat May 13, 2006 10:30

Re: SSAO discussion

Post by Rachael »

Honestly, I am really not sure what sprites will end up having. I've never seen anyone do SSAO with sprites before.
User avatar
Nash
Developer
Developer
Posts: 1226
Joined: Sun Sep 25, 2005 1:49
Location: Kuala Lumpur, Malaysia
Contact:

Re: SSAO discussion

Post by Nash »

I haven't either, come to think of it. I've never seen billboarded sprites in games cast shadows and IMO that would look weird - imagine if things like explosions or particle FX casting AO Shadows..
User avatar
Graf Zahl
GZDoom Developer
GZDoom Developer
Posts: 7148
Joined: Wed Jul 20, 2005 9:48
Location: Germany
Contact:

Re: SSAO discussion

Post by Graf Zahl »

Sprites are always rendered in the translucent pass, so I have no idea what this may imply. Also don't forget that the engine makes extensive use of stencils for portal rendering, which also leaves certain parts in a state that may produce incorrect results.
User avatar
Rachael
Developer
Developer
Posts: 3645
Joined: Sat May 13, 2006 10:30

Re: SSAO discussion

Post by Rachael »

With SSAO itself (the super-sampling part of it, that is, that darkens corners), that may not be too huge of an issue, though there might be inappropriate shadow lines where there shouldn't be at the portal borders. As I said though, I don't think anyone is going to fret TOO much over it, especially considering SSAO will be an optional feature if it ever gets implemented.

With *actual* shadows though - like I think Nash is suggesting - that can present much bigger issues.
User avatar
Nash
Developer
Developer
Posts: 1226
Joined: Sun Sep 25, 2005 1:49
Location: Kuala Lumpur, Malaysia
Contact:

Re: SSAO discussion

Post by Nash »

I wasn't actually talking about casted shadows if that's what you thought I meant... I might have used the word "shadow" incorrectly in this thread but what I've been talking about all this while is indeed the darkening that happens when 2 surfaces are close to each other. :D

On another topic, I don't know what kind of ambient occlusion Doomsday Engine is using with their "FakeRadio" feature but I noticed that sprites aren't included with the FakeRadio... it's literally just the level that's being "shadowed".

Another interesting observation I noticed a while back when qeffects.dll was popular and trendy around here - enabling SSAO produces weird shadows in the sky/skybox. :O I had a skybox sector that is textured with a flat blue texture, and with Line_Horizon on the walls but I ended up seeing contact shadows in the sky...
User avatar
Graf Zahl
GZDoom Developer
GZDoom Developer
Posts: 7148
Joined: Wed Jul 20, 2005 9:48
Location: Germany
Contact:

Re: SSAO discussion

Post by Graf Zahl »

What you saw with the skybox is precisely the stencil effects at play. To ensure that the actual level gets rendered properly sky drawing is doing the following:

Step 1: Draw the stencil
Step 2: Draw the sky into the stencil buffer (or as a separate pass into the main buffer if there's only one sky)
Step 3: Draw the stencil again, this time only affecting the depth buffer.

So you actually get 'edges' in the depth buffer where you see the sky.
dpJudas
Developer
Developer
Posts: 798
Joined: Sat Jul 23, 2016 7:53

Re: SSAO discussion

Post by dpJudas »

GZDoom doesn't render the sprites into the depth buffer, so they will (luckily) not affect the darkening generated by a SSAO pass. The trickiest thing is handling the sky issue along with running the pass exactly between where it switches from solid to translucent objects. If the pass is run after the sprites have been rendered the darkening of things behind the sprites will overlay them.

If you're interested I can upload a build of the SSAO pass I have working in gzdoom right now. It probably looks pretty close to what you saw with qeffects at its current stage though. I'm not sure what is the best way to deal with the stencil/depth thing. Essentially there are two choices: either the main shader outputs linear depth values to a second color buffer (that would introduce the first real gbuffer in gzdoom). Or the SSAO shader looks at the stencil values and assumes there was no ray hit if the stencil changes between the two pixels examined. I'm not sure yet which of the two I prefer.

I don't know the details of HBAO+ right now (in 3D there's so many trending algorithms - can't keep up :D), but most of the algorithms purely use depth data. The SSAO algorithm I've implemented uses purely the depth buffer, but it does attempt to reconstruct a normal for the pixel by looking at 4 neighboring pixels. From there it does 32 ray tests around the pixel and dots that with the reconstructed normal. If that makes it HBAO or HBAO+ I'm not sure, but I'm pretty sure the original SSAO algorithm did not use any normal.
User avatar
Graf Zahl
GZDoom Developer
GZDoom Developer
Posts: 7148
Joined: Wed Jul 20, 2005 9:48
Location: Germany
Contact:

Re: SSAO discussion

Post by Graf Zahl »

it's a bit more tricky than that because of portals in general. Unless something is done, everything behind a portal will show the same problems as the sky. Horizons will do so as well. Every time a portal is finalized the depth buffer will be rewritten in its place, and on top of that, sprites inside portals will be rendered before sprites in the main scene because a portal will be fully processed before going on to the next.
dpJudas
Developer
Developer
Posts: 798
Joined: Sat Jul 23, 2016 7:53

Re: SSAO discussion

Post by dpJudas »

Okay that rules out the possibility to look at the stencil. It will have to output the real depth in a second color buffer for the SSAO pass to work. The sprites thing is certainly more tricky - but maybe it could write some info into the second color buffer so the SSAO pass will know this pixel is covered by a sprite and skip it. On the plus side that means the pass doesn't have to run in the middle of scene rendering.
dpJudas
Developer
Developer
Posts: 798
Joined: Sat Jul 23, 2016 7:53

Re: SSAO discussion

Post by dpJudas »

Thinking about this a little further, I'm not sure flagging sprites in a color buffer will do the trick. It will probably work for monster sprites, but much more difficult with stuff like fireballs.

Maybe it could run a partial SSAO pass covering the portal entrance and then only work on the pixels marked by the stencil. A bit expensive maybe if there's a lot of overlapping portals, but would it work?
User avatar
Nash
Developer
Developer
Posts: 1226
Joined: Sun Sep 25, 2005 1:49
Location: Kuala Lumpur, Malaysia
Contact:

Re: SSAO discussion

Post by Nash »

dpJudas wrote: If you're interested I can upload a build of the SSAO pass I have working in gzdoom right now. It probably looks pretty close to what you saw with qeffects at its current stage though.
I'm interested, if only for curiosity purposes (and to see how much performance difference it is on my computer). :D
dpJudas
Developer
Developer
Posts: 798
Joined: Sat Jul 23, 2016 7:53

Re: SSAO discussion

Post by dpJudas »

Okay, here it is: https://github.com/dpjudas/zdoom/releases/tag/ssao1

Note that the pass has some outstanding issues that causes it to create some annoying shadows due to some bug in the shader itself. You can see how the SSAO part looks like using ssao_debug 1. The gaussian blur also needs to be depth aware.

Overall though, without a solution to the portal situation, the entire thing is more or less useless.
User avatar
Nash
Developer
Developer
Posts: 1226
Joined: Sun Sep 25, 2005 1:49
Location: Kuala Lumpur, Malaysia
Contact:

Re: SSAO discussion

Post by Nash »

I know it's just a test build but while we're here, might as well...

http://imgur.com/a/DtDWw

The shadow position seems to not match the 3D scene. All in all, this was pretty cool to look at, even in the debug mode. I'm amazed at how it handles the transparent wall textures so well, for some reason. :mrgreen:
Locked

Return to “GZDoom”