fog with pixel shaders < 3.0

Stefan Dösinger stefandoesinger at
Mon Mar 12 06:56:54 CDT 2007

Am Montag 12 März 2007 10:35 schrieb Fabian Bieler:

Ok, I am not so much into the whole fog + shader thing, so I am not sure if I 
understand the problem correcty. The vertex shader writes out fog values, 
which go into the fog equation. With just a vertex shader only it works, see 
the dolphin sdk demo.

With pixel Shaders < SM 3.0, the fixed function fog calculation is still 
performed in Direct3D, but not in opengl? So we have to add a fog emulation 
to our pixel shaders?

One thing to consider is that there are 2 different fog types in d3d: Fog 
Calculated per Vertex, and per Fragment fog(or something like this). In gl 
the difference is just the fog niceness hint(GL_NICEST, GL_FASTEST), but in 
d3d they seem to be handled very differently with vertex shaders and 
transformed vertices. And last but not least there is the No-Fog[1] fog which 
has both vertex and table fog set to NONE but is still quite foggy.

Enabling and disabling fog isn't much of a problem once we have a fog 
calculation in the shader I think. We can just set fog start, end, exp 
parameters which effectively eliminate fog.

There will be more problems with selecting the fog equation. It does not apply 
to the no-fog fog(Linear, start = 255(or 1.0), end = 0), or per vertex fog 
with vshaders or transformed vertices(simmilar to no-fog fog). With per 
fragment fog we have linear, exp and exp2. With glsl we can make it a 
parameter, but for arb shaders we have to hardwire it in the shader. But I 
think that applications will not switch fog formulas like mad because the fog 
types won't mix well. So I think the hardwire and recompile approach is 

What we need for the start I think is the ability to recompile the shader. 
(Destroying the arb / glsl programs, removing it from the linked glsl shader 
table and reset the shader object). Henri, do you have any patches for that 
handy or should I take a look? With that I can then also add the signedness 
fixup for the bump map formats if no suitable extension is available.

If we want to manage multiple compiled shaders with fog on/off then we should 
extend that to the whole parameter set. We already have sampler type(2D, 3D, 
cube), texture transform flags(D3DTTF_PROJECTED) and now fog and pixel format 
coming. My concern is that managing this will get too expensive and grow the 
amount of linked glsl programs vastly.

> 3) Add Fog calculation to every shader, enabling or disabling it at runtime
> via a constant factor of 1.0 or 0.0 depending on the fog state which is
> supplied as a shader constant.
> Pro: cpu-load wise cheap
> Con: Adds some more instructions to fragment shader code.
Somehow you will have to get the fog parameters into the shader. Start and End 
won't be of much issue, but the formula will be a problem. Not so much with 
glsl, but arb will be hard.

I think you can just access the fixed function fog states from shaders. But it 
will use up one or more uniforms.

> Fabian
> P.S: I noticed that fog-state management is not yet included in the state
> table. Should a patch that fixes fog with pixel shaders wait until this is
> done, or is this of no importance?
No, it should be there :-) See STATE_RENDER(WINED3DRS_FOGENABLE), and states 
linked to it. Or did I miss anything? The apply handler should be state_fog.

[1] I don't know how MS calls it. With the fixed function pipeline fog 
weightings are stored in the alpha component of the specular vertex color. I 
think with vshaders it is linear, start = 1.0, end = 0.0 .
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url :

More information about the wine-devel mailing list