wined3d: Take abs() of vertex z coordinate as FFP fog coordinate and remove oFog clamp for vertex shaders

[Stefan Dösinger]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Am 2014-10-24 23:03, schrieb Joachim Priesner:
> That flag doesn't exist prior to d3d9. The tests differ on d3d8
> too, and I haven't found a matching flag there. (If you think it's
> worth a look: What is the value of this flag for your AMD cards?)
All my Radeon cards set it.

If it's not too much effort I suggest this: Modify
dlls/wined3d/directx.c to set the flag in a separate one-liner patch.
Make the d3d9 tests test expect colors according to the flag and make
d3d8 accept both results. If there are any other unforeseen issues
ignore the vertex clamping behavior.

> By the way, I found this regarding clamping: 
> http://msdn.microsoft.com/en-us/library/windows/hardware/ff539405%28v=vs.85%29.aspx
>
>
> 
"Because the driver clamps the fog intensity value on a per-pixel
> basis, the runtime for DirectX 9.0 and later no longer clamps the
> fog intensity value before sending it to the driver."
I think this only affects the fixed function vertex pipeline and not
shaders. There are two ways to pass custom fog factors in the fixed
function pipeline. One is the specular alpha, the other one is a
separate D3DFVF_FOG field. We don't support D3DFVF_FOG, and I don't
know about any game that uses it.

MSDN talks about software vertex shaders on this page. I don't really
know how they work, but I suspect from a driver point of view they
pass pretransformed vertices (D3DFVF_XYZRHW or D3DDECLUSAGE_POSITIONT).

> This suggests that the clamping behavior should be different for
> d3d8 and d3d9. The D3DPMISCCAPS_FOGINFVF flag could probably be
> used to differentiate the behavior for d3d9, but again it does not
> exist on d3d8.
My guess is that prior to shader model 2, the register used to pass
fog values was a fixed point register that supported values from 0.0
to 1.0, so it was a hardware limitation that enforced the clamp. With
Shader Model 2 the register turned into a float register, so d3d9
needed a way for the driver to specify the behavior, maybe with the
idea that old hardware clamps and new hardware does not. In that sense
it would be a bug that Nvidia does not clamp when d3d8.dll is used
with d3d9+ hardware and that AMD clamps when d3d9.dll is used with
d3d9+ hardware.

For the sake of curiosity it would be interesting to know what Shader
Model 1 Nvidia hardware (Geforce 3/4) does, but I don't have such a
card. I also don't think it is too important because we didn't run
into a game that depends on either behavior. But we also didn't run
into a game that needs the fixed function abs() before...

Shader model 3 does away with the fixed varying semantics. It has just
10 general purpose unclamped float4 registers.

> Definitely, so thanks for your help. The tests for abs() should
> now be sufficient [in theory -- e.g. I have no idea what to do
> about them crashing on your Radeon 9000].
Don't worry about the 9000. I'll investigate this when I have time.

> Testing the oFog clamp behavior is a nice extra -- I wouldn't be 
> unhappy if a version of the patch that leaves that out could be 
> committed.
I'll do some investigation regarding vertex fog + a fragment
processing setup that just replaces the color, but otherwise I agree.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQIcBAEBAgAGBQJUTjmeAAoJEN0/YqbEcdMwyWMP/1Q397z6qC+Ao/whYAWQQjeh
lu1820mJBJk9Lvek/NMXZvXpnXBeGCe9BWIU+Pn8Pixu1pvbHTbzWY9n+mQdNr35
oIr2z3OdTI57ZxDFriGQizgvI3LczuuL6rIAuih5fQQhxNz8xidiacRxRYd/D7rf
XXOA/JXb4FdMWlMoCqm3iDKH3eGbf1CGygU+wxgD3AfU+NWL1bV16XzJtY93Wzf0
Q0RSOIBDHFf9CO2+aBeM/XwaNvWIy6MRhUbWPVezaP3fNt69Qck6Eq/Jl+UTH9G2
3NlOs622d20B2/OIzyGqOX8KUI3GpOCq5CKm9ym6Gsrxqn5TenHu9eFZi2n4wqDT
iLHBGv/fK5jMgZucq8SLHJ4ft+oK8H04TC0P852GH6gXoqsLfn89mVYLqa7aOdGq
wSb9/g2kQYdXWgILY8xaIU2Q3VFni57SegZCmNVnZ0BvtLHyaZNqpRY+3DGfBbSV
7xThRE1Mpcvo4DQhtT86D2GVJ1pe4QxS1XfkHB3jXo/KY6K+weTlljPZXlfTF5k/
H1p9dTaezUvqwueCn+SOpE0zxQ0rbfkNyN0KQWEr7WeYohMoVtrej8AMzCKjbD02
Ln3kQ6+VW3+uJFKpl+9GvAQ3VdiDJJq1VdxUkswbxYuVhvp0pSkHBOinq480h9Og
+okc0UPMBzSLlbwCIgt9
=pMud
-----END PGP SIGNATURE-----