Page 2 of 2

Re: [GZDOOM renderer q1.1pre-260] Palette tonemap black screen

Posted: Sun Nov 27, 2016 10:15
by Rachael
Does LLVM have a debugger? Maybe you can compile them with that, and maybe it's possible at the very least see what the differences are in the shaders between GZDoom's initialization and QZDoom's - even if the LLVM shaders do not produce any errors.

At this point, though, I am just shooting out ideas. I'm just as clueless as you are what's causing this. NVidia has a pretty solid driver history compared to the other two, but it's not infallible and I've had quite a few botched NVidia drivers in the past.

Also - do try the rename exe trick - it's possible NVidia already accounted for this in GZDoom but failed to take QZDoom into consideration due to it being too new. That kind of fuckery certainly exists with ATI drivers and is quite common there.

Re: [GZDOOM renderer q1.1pre-260] Palette tonemap black screen

Posted: Sun Nov 27, 2016 10:22
by dpJudas
Here's what I got so far:

1. It works in my local build of QZDoom, GZDoom, and the GZDoom ssao branch.
2. It does not work in the latest nightly build that I use when playing Doom (your builds got all the sound compiled in - mine is missing some of that).
3. It also does not work in a much older October build of the GZDoom ssao branch that I used to use for playing in the past.

Right now I'm betting on it being an issue with the nightly build somehow using files out of date. Is there a way to force a rebuild?

Re: [GZDOOM renderer q1.1pre-260] Palette tonemap black screen

Posted: Sun Nov 27, 2016 10:27
by Rachael
I'll clean my build tree. Were you going to do any work on the code base today? It would probably help a lot if the git hash number gets bumped, instead of me deleting and replacing the latest one. :P Actually - it seems like you made a commit after the latest build, so I'll just build that commit. New builds should be up somewhere between 20-40 minutes after this post.

Re: [GZDOOM renderer q1.1pre-260] Palette tonemap black screen

Posted: Sun Nov 27, 2016 10:38
by dpJudas
Excellent. Thanks Eruanna. Hopefully this is the cause, because otherwise it has to be some oddity with differences between my ini files that triggers it one place but not the other.

Re: [GZDOOM renderer q1.1pre-260] Palette tonemap black screen

Posted: Sun Nov 27, 2016 11:26
by dpJudas
I've tracked down what decides if it works or not: the texture filtering mode. If it is set to None then it works, if its set to None (Trilinear) it doesn't.

What is probably happening is that the sampler is in trilinear mode, which requires mipmap. It shouldn't matter as the tonemap GLSL uses texelFetch, which doesn't do any sampling. Either there's some rule deep down in OpenGL that says the texture must always be texture complete, even for texelFetch, and they just fixed that in their driver. Or they introduced a bug that makes texelFetch not work in a situation where it should.

Anyhow, I'll do some test a bit later whether GZDoom is affected by it, or just the SSAO branch. Then fix it by making sure that the sampler is set to nearest always for the tonemap texture, regardless of the setting in the menu.

Re: [GZDOOM renderer q1.1pre-260] Palette tonemap black screen

Posted: Sun Nov 27, 2016 12:45
by Graf Zahl
dpJudas wrote:Anyhow, I'll do some test a bit later whether GZDoom is affected by it, or just the SSAO branch. Then fix it by making sure that the sampler is set to nearest always for the tonemap texture, regardless of the setting in the menu.

That should be done for any postprocessing anyway, if you do not have a mipmap complete texture, always choose a mode that doesn't use mipmaps, i.e. GL_NEAREST or GL_LINEAR, depending on what is needed. It really wouldn't surprise me if this causes other black screen issues as well. AFAIK the GL spec is very strict here: Use of a mipmap incomplete texture with such a mode is an error.

Re: [GZDOOM renderer q1.1pre-260] Palette tonemap black screen

Posted: Sun Nov 27, 2016 16:40
by William
I have the texture filtering set to Trilinear and 16x anisotropic. The display output seems to come back when toggling the filter between Trilinear and None, and stays working until the next level load, at which point the screen turns black again.

Re: [GZDOOM renderer q1.1pre-260] Palette tonemap black screen

Posted: Sun Nov 27, 2016 17:35
by dpJudas
Graf Zahl wrote:That should be done for any postprocessing anyway, if you do not have a mipmap complete texture, always choose a mode that doesn't use mipmaps, i.e. GL_NEAREST or GL_LINEAR, depending on what is needed. It really wouldn't surprise me if this causes other black screen issues as well. AFAIK the GL spec is very strict here: Use of a mipmap incomplete texture with such a mode is an error.
I agree, but somehow I must have forgotten to do that with the tonemap LUT texture. Since apparently all the vendors didn't check for this properly I never noticed.

Re: [GZDOOM renderer q1.1pre-260] Palette tonemap black screen

Posted: Mon Nov 28, 2016 2:19
by William
So does this mean it is a GZDoom bug? If so, and if the forum allows, you can move it to the GZDoom forum.

Re: [GZDOOM renderer q1.1pre-260] Palette tonemap black screen

Posted: Mon Nov 28, 2016 2:39
by dpJudas
Just fixed it. Should be working in the next build of QZDoom.

Turns out it was due to a regression in the SSAO branch made for GZDoom. The good news is that this hasn't been merged into GZDoom yet, so GZDoom is in the clear.