[PATCH] wined3d: fix color in range comparison

Wed May 27 02:39:02 CDT 2015

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Am 2015-05-26 um 14:41 schrieb Sergey Isakov:
> +#define RED_KEY(x)   (x & 0xff0000)
> +#define GREEN_KEY(x) (x & 0x00ff00)  
> +#define BLUE_KEY(x)  (x & 0x0000ff)
> +    return ((RED_KEY(color) >= RED_KEY(color_key->color_space_low_value)) &&
> +            (RED_KEY(color) <= RED_KEY(color_key->color_space_high_value)) &&
> +            (GREEN_KEY(color) >= GREEN_KEY(color_key->color_space_low_value)) &&
> +            (GREEN_KEY(color) <= GREEN_KEY(color_key->color_space_high_value)) &&
> +            (BLUE_KEY(color) >= BLUE_KEY(color_key->color_space_low_value)) &&
> +            (BLUE_KEY(color) <= BLUE_KEY(color_key->color_space_high_value)));
> +#undef RED_KEY
> +#undef GREEN_KEY
> +#undef BLUE_KEY
The basic idea is most likely right, but it needs tests. Do you have a game that needs this behavior?

The way you're extracting individual colors from the key is wrong. The bitmasks depend on the texture format.

The code you're modifying is unused on any modern GPU. On GPUs that support ARB_fragment_program or GLSL we do the color key comparison in the fragment shader nowadays.