[PATCH 3/9] wined3d: Implement color keying in arbfp_blit.

[Matteo Bruni]

2015-04-09 22:48 GMT+02:00 Stefan Dösinger <stefan at codeweavers.com>:
> This intentionally discards on the alpha value. To do a proper color
> compare we have to migrate fixed function pipeline color keying and
> blits at the same time because the color keying conversion changes the
> texture contents and can cause the key not to match.
> ---
>  dlls/wined3d/arb_program_shader.c | 68 +++++++++++++++++++++++++++++++++------
>  dlls/wined3d/surface.c            | 51 +++++++++++++++++++----------
>  dlls/wined3d/swapchain.c          |  2 +-
>  dlls/wined3d/wined3d_private.h    |  7 ++--
>  4 files changed, 99 insertions(+), 29 deletions(-)
>
> diff --git a/dlls/wined3d/arb_program_shader.c b/dlls/wined3d/arb_program_shader.c
> index 79439f5..f740123 100644
> --- a/dlls/wined3d/arb_program_shader.c
> +++ b/dlls/wined3d/arb_program_shader.c
> @@ -7539,7 +7549,24 @@ static GLuint arbfp_gen_plain_shader(struct arbfp_blit_priv *priv,
>      GL_EXTCALL(glBindProgramARB(GL_FRAGMENT_PROGRAM_ARB, shader));
>
>      shader_addline(&buffer, "!!ARBfp1.0\n");
> -    shader_addline(&buffer, "TEX result.color, fragment.texcoord[0], texture[0], %s;\n", tex_target);
> +
> +    if (type->use_color_key)
> +    {
> +        shader_addline(&buffer, "TEMP color;\n");
> +        shader_addline(&buffer, "TEMP compare;\n");
> +        shader_addline(&buffer, "PARAM color_key = program.local[%u];\n", ARBFP_BLIT_PARAM_COLOR_KEY);
> +        shader_addline(&buffer, "TEX color, fragment.texcoord[0], texture[0], %s;\n", tex_target);
> +        shader_addline(&buffer, "SGE compare.r, color.a, color_key.a;\n");
> +        shader_addline(&buffer, "SGE compare.g, -color.a, color_key.a;\n");
> +        shader_addline(&buffer, "MUL compare, compare.r, -compare.g;\n");
> +        shader_addline(&buffer, "KIL compare;\n");
> +        shader_addline(&buffer, "MOV result.color, color;\n");

If you're checking for an exact match, that doesn't look right. You
probably want something like:

...
SGE compare.r, color.a, color_key.a;
SGE compare.g, -color.a, -color_key.a;
MUL compare, compare.r, compare.g;
...

I might have misunderstood the point of the shader though, if that's
the case then ignore this.