WINED3D: Color fixups for vertex shaders

[H. Verbeet]

On 10/03/06, Roderick Colenbrander <thunderbird2k at gmx.net> wrote:
> I have walked through the code. This small demo (dolphinvs from dx8.1 sdk)
> uses the following vertex declaration for the seafloor:
> D3DVSD_STREAM( 0 ),
> D3DVSD_REG( 0, D3DVSDT_FLOAT3 ), // Position of first mesh
> D3DVSD_REG( 3, D3DVSDT_FLOAT3 ), // Normal
> D3DVSD_REG( 6, D3DVSDT_FLOAT2 ), // Tex coords
> D3DVSD_END()
>
> It stores texture coordinates in register 6 (D3DVSD_SPECULAR). Registers
> should be general purpose I think and this should work fine. Your patch
> assumes that 'D3DVSD_SPECULAR' is allways used for colors which should be
> swizzled.
Well, no. It assumes that
arrayUsageMap[WINED3DSHADERDECLUSAGE_SPECULAR] contains the register
number for the register that is used to store specular colors, or -1
if there are no specular colors. loadNumberedArrays() in drawprim.c
uses arrayUsageMap in the same way.

> Because we are storing texture coordinates in them they shouldn't
> be swizzled but we don't know what is stored in them. I'm not sure how to
> fix this bug but I think the code should be moved to a later stage.
I don't think that will work. We really need to know if a certain
block of data is going to be used for colors or not, or we can't fix
the data, in the shader or otherwise. Also, we have to fix the data
before it gets passed to the shader or on input, we can't do it on
output because other components (with the correct color order) might
have been added in the shader.

Either way, arrayUsageMap doesn't get initialised correctly. If we
don't know what's in a register, we shouldn't pass it through
arrayUsageMap, but use some other way.

I won't be around next week, but after that I'll probably try to have
a look at d3d8 shaders.