[2.1.1] Bug : Broken HQNx algorithms (in comp. to 2.0.05)

[theleo_ua]

1) Run GZdoom 2.1.1 and 2.0.05 with -iwad heretic.wad
2) type "summon crossbow"
3) Choose next options: http://storage7.static.itmages.ru/i/16/ ... da66ff.png
4) Observe, that crossbow is displayed incorrectly in 2.1.1 (in comparison to 2.0.05), see screenshot above

NOTE: please dont pay attention to "mmx" option in screenshot, because bug reproduced on both "hq4x" and "hq4x mmx" options in 2.1.1

NOTE2: same bug was in 1.8.7 (and wasn't in 2.0.03)

[_mental_]

I made a pull request with the fix. I could commit it by myself, but looks like that damn alpha check is cursed I will be glad if someone else will verify the fix.

[Graf Zahl]

Why did you shuffle around the meanings of the values in the first place? That was the source of all the problems, it seems.

[_mental_]

Why did you shuffle around the meanings of the values in the first place? That was the source of all the problems, it seems.

Where did I do so? In commit message?

[Graf Zahl]

Not this time. In earlier versions 4-6 and 7-9 were swapped and at some point - I think when the asm versions were replaced - they were changed. And this was causing the problems because all the checks were not changed along.

[_mental_]

Not this time. In earlier versions 4-6 and 7-9 were swapped and at some point - I think when the asm versions were replaced - they were changed. And this was causing the problems because all the checks were not changed along.

No, they were not swapped.
Initially, indices from 4 to 6 were used by MMX version. Later it was replaced with generic implementation. Then MMX one was re-added but indices of generic one were not changes.
So, generic implementation occupies indices 4..6 and MMX occupies 7..9. You can check log/blame in your Git client or on GitHub website.

Here is a complete unified diff of changes done to src/gl/textures/gl_hqresize.cpp. It's not so easy to check it without syntax highlight though.

---

Code: Select all

commit cbcde3a950abf62d2989063034506ab020b9e00b
Author: alexey.lysiuk
Date:   Mon Apr 4 12:10:26 2016 +0300

    Fixed check for alpha channel in texture to select hqNx upscaling mode
    
    Now it's the initial check with the adjustment in mode indices only, as old hqNx MMX indices (4..6) are now occupied by generic hqNx implementation
    See http://forum.drdteam.org/viewtopic.php?t=6872

diff --git a/src/gl/textures/gl_hqresize.cpp b/src/gl/textures/gl_hqresize.cpp
index 1563895..2118352 100644
--- a/src/gl/textures/gl_hqresize.cpp
+++ b/src/gl/textures/gl_hqresize.cpp
@@ -289,8 +289,8 @@ unsigned char *gl_CreateUpsampledTextureBuffer ( const FTexture *inputTexture, u
 		outWidth = inWidth;
 		outHeight = inHeight;
 #ifdef HAVE_MMX
-		// ASM-hqNx does not preserve the alpha channel so fall back to C-version for such textures
-		if (!hasAlpha && type > 6 && type <= 9)
+		// hqNx MMX does not preserve the alpha channel so fall back to C-version for such textures
+		if (hasAlpha && type > 6 && type <= 9)
 		{
 			type -= 3;
 		}

commit 37ac6ef9a04078cd0491acab82970e3777fd481f
Author: Christoph Oelckers
Date:   Sat Jan 30 02:13:47 2016 +0100

    - fixed: Translucency detection for GL textures was broken.
    - fixed: Textures which are already scaled should not be upsampled.
    - fixed: The transparency check in the upscaling code checked the wrong modes for exclusion when handling translucent textures.

diff --git a/src/gl/textures/gl_hqresize.cpp b/src/gl/textures/gl_hqresize.cpp
index 261f66e..1563895 100644
--- a/src/gl/textures/gl_hqresize.cpp
+++ b/src/gl/textures/gl_hqresize.cpp
@@ -263,6 +263,10 @@ unsigned char *gl_CreateUpsampledTextureBuffer ( const FTexture *inputTexture, u
 	if ( inputTexture->bHasCanvas )
 		return inputBuffer;
 
+	// already scaled?
+	if (inputTexture->xScale >= FRACUNIT*2 && inputTexture->yScale >= FRACUNIT*2)
+		return inputBuffer;
+
 	switch (inputTexture->UseType)
 	{
 	case FTexture::TEX_Sprite:
@@ -281,14 +285,14 @@ unsigned char *gl_CreateUpsampledTextureBuffer ( const FTexture *inputTexture, u
 
 	if (inputBuffer)
 	{
+		int type = gl_texture_hqresize;
 		outWidth = inWidth;
 		outHeight = inHeight;
-		int type = gl_texture_hqresize;
 #ifdef HAVE_MMX
 		// ASM-hqNx does not preserve the alpha channel so fall back to C-version for such textures
-		if (!hasAlpha && type > 3 && type <= 6)
+		if (!hasAlpha && type > 6 && type <= 9)
 		{
-			type += 3;
+			type -= 3;
 		}
 #endif
 

commit 1c5d0ccd65e8a8ae631fd3160abf808fb26635d6
Author: alexey.lysiuk
Date:   Fri Dec 25 15:41:06 2015 +0200

    - enabled hqNx MMX on all platforms with Intel intrinsics support

diff --git a/src/gl/textures/gl_hqresize.cpp b/src/gl/textures/gl_hqresize.cpp
index 621f444..261f66e 100644
--- a/src/gl/textures/gl_hqresize.cpp
+++ b/src/gl/textures/gl_hqresize.cpp
@@ -40,13 +40,13 @@
 #include "gl/textures/gl_texture.h"
 #include "c_cvars.h"
 #include "gl/hqnx/hqx.h"
-#ifdef _MSC_VER
+#ifdef HAVE_MMX
 #include "gl/hqnx_asm/hqnx_asm.h"
 #endif
 
 CUSTOM_CVAR(Int, gl_texture_hqresize, 0, CVAR_ARCHIVE | CVAR_GLOBALCONFIG | CVAR_NOINITCALL)
 {
-#ifdef _MSC_VER
+#ifdef HAVE_MMX
 	if (self < 0 || self > 9)
 #else
 	if (self < 0 || self > 6)
@@ -186,8 +186,7 @@ static unsigned char *scaleNxHelper( void (*scaleNxFunction) ( uint32* , uint32*
 	return newBuffer;
 }
 
-// [BB] hqnx scaling is only supported with the MS compiler.
-#ifdef _MSC_VER
+#ifdef HAVE_MMX
 static unsigned char *hqNxAsmHelper( void (*hqNxFunction) ( int*, unsigned char*, int, int, int ),
 							  const int N,
 							  unsigned char *inputBuffer,
@@ -285,7 +284,7 @@ unsigned char *gl_CreateUpsampledTextureBuffer ( const FTexture *inputTexture, u
 		outWidth = inWidth;
 		outHeight = inHeight;
 		int type = gl_texture_hqresize;
-#ifdef _MSC_VER
+#ifdef HAVE_MMX
 		// ASM-hqNx does not preserve the alpha channel so fall back to C-version for such textures
 		if (!hasAlpha && type > 3 && type <= 6)
 		{
@@ -307,7 +306,7 @@ unsigned char *gl_CreateUpsampledTextureBuffer ( const FTexture *inputTexture, u
 			return hqNxHelper( &hq3x_32, 3, inputBuffer, inWidth, inHeight, outWidth, outHeight );
 		case 6:
 			return hqNxHelper( &hq4x_32, 4, inputBuffer, inWidth, inHeight, outWidth, outHeight );
-#ifdef _MSC_VER
+#ifdef HAVE_MMX
 		case 7:
 			return hqNxAsmHelper( &HQnX_asm::hq2x_32, 2, inputBuffer, inWidth, inHeight, outWidth, outHeight );
 		case 8:

commit 19ae244f66fdda01de08b0db458aa0a5a486d024
Author: alexey.lysiuk
Date:   Thu Dec 24 10:33:30 2015 +0200

    - fixed: allow to use all hqNx texture upscale modes

diff --git a/src/gl/textures/gl_hqresize.cpp b/src/gl/textures/gl_hqresize.cpp
index a4f192c..621f444 100644
--- a/src/gl/textures/gl_hqresize.cpp
+++ b/src/gl/textures/gl_hqresize.cpp
@@ -46,7 +46,11 @@
 
 CUSTOM_CVAR(Int, gl_texture_hqresize, 0, CVAR_ARCHIVE | CVAR_GLOBALCONFIG | CVAR_NOINITCALL)
 {
+#ifdef _MSC_VER
+	if (self < 0 || self > 9)
+#else
 	if (self < 0 || self > 6)
+#endif
 		self = 0;
 	GLRenderer->FlushTextures();
 }

commit 6f65bccf1c805ef001294b86f604cd49d7a291ec
Author: Christoph Oelckers
Date:   Wed Aug 20 12:45:33 2014 +0200

    - reinstated the far superior assembly HQnX version for Visual C++.

diff --git a/src/gl/textures/gl_hqresize.cpp b/src/gl/textures/gl_hqresize.cpp
index 59e8405..a4f192c 100644
--- a/src/gl/textures/gl_hqresize.cpp
+++ b/src/gl/textures/gl_hqresize.cpp
@@ -40,6 +40,9 @@
 #include "gl/textures/gl_texture.h"
 #include "c_cvars.h"
 #include "gl/hqnx/hqx.h"
+#ifdef _MSC_VER
+#include "gl/hqnx_asm/hqnx_asm.h"
+#endif
 
 CUSTOM_CVAR(Int, gl_texture_hqresize, 0, CVAR_ARCHIVE | CVAR_GLOBALCONFIG | CVAR_NOINITCALL)
 {
@@ -179,6 +182,38 @@ static unsigned char *scaleNxHelper( void (*scaleNxFunction) ( uint32* , uint32*
 	return newBuffer;
 }
 
+// [BB] hqnx scaling is only supported with the MS compiler.
+#ifdef _MSC_VER
+static unsigned char *hqNxAsmHelper( void (*hqNxFunction) ( int*, unsigned char*, int, int, int ),
+							  const int N,
+							  unsigned char *inputBuffer,
+							  const int inWidth,
+							  const int inHeight,
+							  int &outWidth,
+							  int &outHeight )
+{
+	outWidth = N * inWidth;
+	outHeight = N *inHeight;
+
+	static int initdone = false;
+
+	if (!initdone)
+	{
+		HQnX_asm::InitLUTs();
+		initdone = true;
+	}
+
+	HQnX_asm::CImage cImageIn;
+	cImageIn.SetImage(inputBuffer, inWidth, inHeight, 32);
+	cImageIn.Convert32To17();
+
+	unsigned char * newBuffer = new unsigned char[outWidth*outHeight*4];
+	hqNxFunction( reinterpret_cast<int*>(cImageIn.m_pBitmap), newBuffer, cImageIn.m_Xres, cImageIn.m_Yres, outWidth*4 );
+	delete[] inputBuffer;
+	return newBuffer;
+}
+#endif
+
 static unsigned char *hqNxHelper( void (*hqNxFunction) ( unsigned*, unsigned*, int, int ),
 							  const int N,
 							  unsigned char *inputBuffer,
@@ -203,6 +238,7 @@ static unsigned char *hqNxHelper( void (*hqNxFunction) ( unsigned*, unsigned*, i
 	return newBuffer;
 }
 
+
 //===========================================================================
 // 
 // [BB] Upsamples the texture in inputBuffer, frees inputBuffer and returns
@@ -245,11 +281,11 @@ unsigned char *gl_CreateUpsampledTextureBuffer ( const FTexture *inputTexture, u
 		outWidth = inWidth;
 		outHeight = inHeight;
 		int type = gl_texture_hqresize;
-#if 0
-		// hqNx does not preserve the alpha channel so fall back to ScaleNx for such textures
-		if (hasAlpha && type > 3)
+#ifdef _MSC_VER
+		// ASM-hqNx does not preserve the alpha channel so fall back to C-version for such textures
+		if (!hasAlpha && type > 3 && type <= 6)
 		{
-			type -= 3;
+			type += 3;
 		}
 #endif
 
@@ -267,6 +303,14 @@ unsigned char *gl_CreateUpsampledTextureBuffer ( const FTexture *inputTexture, u
 			return hqNxHelper( &hq3x_32, 3, inputBuffer, inWidth, inHeight, outWidth, outHeight );
 		case 6:
 			return hqNxHelper( &hq4x_32, 4, inputBuffer, inWidth, inHeight, outWidth, outHeight );
+#ifdef _MSC_VER
+		case 7:
+			return hqNxAsmHelper( &HQnX_asm::hq2x_32, 2, inputBuffer, inWidth, inHeight, outWidth, outHeight );
+		case 8:
+			return hqNxAsmHelper( &HQnX_asm::hq3x_32, 3, inputBuffer, inWidth, inHeight, outWidth, outHeight );
+		case 9:
+			return hqNxAsmHelper( &HQnX_asm::hq4x_32, 4, inputBuffer, inWidth, inHeight, outWidth, outHeight );
+#endif
 		}
 	}
 	return inputBuffer;

commit 2925c96b596ce71a4e01463a611318465e8c4661
Author: Christoph Oelckers
Date:   Sat Jun 21 15:50:32 2014 +0200

    removed all GL 2.x code.
    
    After thinking about it for a day or so I believe it's the best option to remove all compatibility code because it's a major obstacle for a transition to a core profile.

diff --git a/src/gl/textures/gl_hqresize.cpp b/src/gl/textures/gl_hqresize.cpp
index 8379771..59e8405 100644
--- a/src/gl/textures/gl_hqresize.cpp
+++ b/src/gl/textures/gl_hqresize.cpp
@@ -224,10 +224,6 @@ unsigned char *gl_CreateUpsampledTextureBuffer ( const FTexture *inputTexture, u
 	if ( inputTexture->bHasCanvas )
 		return inputBuffer;
 
-	// [BB] Don't upsample non-shader handled warped textures. Needs too much memory and time
-	if (inputTexture->bWarped && !gl.hasGLSL())
-		return inputBuffer;
-
 	switch (inputTexture->UseType)
 	{
 	case FTexture::TEX_Sprite:

commit 09f407143649616bba8c35755dced5711481504e
Author: Christoph Oelckers
Date:   Sun May 11 13:27:51 2014 +0200

    Ok, it had to be done: Removed shader support for pre GLSL 1.3/GL 3.0 hardware. The compromises needed to accomodate these are just too bad and would block any attempt at streamlining the code.

diff --git a/src/gl/textures/gl_hqresize.cpp b/src/gl/textures/gl_hqresize.cpp
index 2c5f831..8379771 100644
--- a/src/gl/textures/gl_hqresize.cpp
+++ b/src/gl/textures/gl_hqresize.cpp
@@ -225,7 +225,7 @@ unsigned char *gl_CreateUpsampledTextureBuffer ( const FTexture *inputTexture, u
 		return inputBuffer;
 
 	// [BB] Don't upsample non-shader handled warped textures. Needs too much memory and time
-	if (gl.shadermodel == 2 || (gl.shadermodel == 3 && inputTexture->bWarped))
+	if (inputTexture->bWarped && !gl.hasGLSL())
 		return inputBuffer;
 
 	switch (inputTexture->UseType)

commit 4909aa750fa44853e88647509dac2049522182a1
Author: galtgendo
Date:   Sun Mar 2 11:32:45 2014 +0100

    Use more portable hqx code from googlecode hqx
    
    Conflicts:
    	src/CMakeLists.txt

diff --git a/src/gl/textures/gl_hqresize.cpp b/src/gl/textures/gl_hqresize.cpp
index 9145ac3..2c5f831 100644
--- a/src/gl/textures/gl_hqresize.cpp
+++ b/src/gl/textures/gl_hqresize.cpp
@@ -39,18 +39,11 @@
 #include "gl/renderer/gl_renderer.h"
 #include "gl/textures/gl_texture.h"
 #include "c_cvars.h"
-// [BB] hqnx scaling is only supported with the MS compiler.
-#if (defined _MSC_VER) && (!defined _WIN64)
-#include "gl/hqnx/hqnx.h"
-#endif
+#include "gl/hqnx/hqx.h"
 
 CUSTOM_CVAR(Int, gl_texture_hqresize, 0, CVAR_ARCHIVE | CVAR_GLOBALCONFIG | CVAR_NOINITCALL)
 {
-#ifdef _MSC_VER
 	if (self < 0 || self > 6)
-#else
-	if (self < 0 || self > 3)
-#endif
 		self = 0;
 	GLRenderer->FlushTextures();
 }
@@ -186,9 +179,7 @@ static unsigned char *scaleNxHelper( void (*scaleNxFunction) ( uint32* , uint32*
 	return newBuffer;
 }
 
-// [BB] hqnx scaling is only supported with the MS compiler.
-#if (defined _MSC_VER) && (!defined _WIN64)
-static unsigned char *hqNxHelper( void (*hqNxFunction) ( int*, unsigned char*, int, int, int ),
+static unsigned char *hqNxHelper( void (*hqNxFunction) ( unsigned*, unsigned*, int, int ),
 							  const int N,
 							  unsigned char *inputBuffer,
 							  const int inWidth,
@@ -200,22 +191,17 @@ static unsigned char *hqNxHelper( void (*hqNxFunction) ( int*, unsigned char*, i
 
 	if (!initdone)
 	{
-		InitLUTs();
+		hqxInit();
 		initdone = true;
 	}
 	outWidth = N * inWidth;
 	outHeight = N *inHeight;
 
-	CImage cImageIn;
-	cImageIn.SetImage(inputBuffer, inWidth, inHeight, 32);
-	cImageIn.Convert32To17();
-
 	unsigned char * newBuffer = new unsigned char[outWidth*outHeight*4];
-	hqNxFunction( reinterpret_cast<int*>(cImageIn.m_pBitmap), newBuffer, cImageIn.m_Xres, cImageIn.m_Yres, outWidth*4 );
+	hqNxFunction( reinterpret_cast<unsigned*>(inputBuffer), reinterpret_cast<unsigned*>(newBuffer), inWidth, inHeight );
 	delete[] inputBuffer;
 	return newBuffer;
 }
-#endif
 
 //===========================================================================
 // 
@@ -263,11 +249,13 @@ unsigned char *gl_CreateUpsampledTextureBuffer ( const FTexture *inputTexture, u
 		outWidth = inWidth;
 		outHeight = inHeight;
 		int type = gl_texture_hqresize;
+#if 0
 		// hqNx does not preserve the alpha channel so fall back to ScaleNx for such textures
 		if (hasAlpha && type > 3)
 		{
 			type -= 3;
 		}
+#endif
 
 		switch (type)
 		{
@@ -277,15 +265,12 @@ unsigned char *gl_CreateUpsampledTextureBuffer ( const FTexture *inputTexture, u
 			return scaleNxHelper( &scale3x, 3, inputBuffer, inWidth, inHeight, outWidth, outHeight );
 		case 3:
 			return scaleNxHelper( &scale4x, 4, inputBuffer, inWidth, inHeight, outWidth, outHeight );
-// [BB] hqnx scaling is only supported with the MS compiler.
-#if (defined _MSC_VER) && (!defined _WIN64)
 		case 4:
 			return hqNxHelper( &hq2x_32, 2, inputBuffer, inWidth, inHeight, outWidth, outHeight );
 		case 5:
 			return hqNxHelper( &hq3x_32, 3, inputBuffer, inWidth, inHeight, outWidth, outHeight );
 		case 6:
 			return hqNxHelper( &hq4x_32, 4, inputBuffer, inWidth, inHeight, outWidth, outHeight );
-#endif
 		}
 	}
 	return inputBuffer;

commit 2885056f46eefd70fd3f67455249dff37bee8f50
Author: Christoph Oelckers <coelckers@zdoom.fake>
Date:   Tue Sep 3 18:29:39 2013 +0200

    - moved GLEXT API out of RenderContext struct.

diff --git a/src/gl/textures/gl_hqresize.cpp b/src/gl/textures/gl_hqresize.cpp
index 36861f7..9145ac3 100644
--- a/src/gl/textures/gl_hqresize.cpp
+++ b/src/gl/textures/gl_hqresize.cpp
@@ -35,6 +35,7 @@
 */
 
 #include "gl/system/gl_system.h"
+#include "gl/system/gl_interface.h"
 #include "gl/renderer/gl_renderer.h"
 #include "gl/textures/gl_texture.h"
 #include "c_cvars.h"

commit 399d0974abb54cb53fa7b643bd1b27cc39523d28
Author: Christoph Oelckers
Date:   Sun Jun 23 09:49:34 2013 +0200

    - added GL render as of SVN revision 1600.