WineD3D: Correctly draw transformed and untransformed vertices in the same scene

[Stefan Dösinger]

Direct3D apps can draw pre-transformed vertices and completely bypass vertex 
transformation and lighting. Opengl doesn't have that concept, so we set the 
transformation matrix to idendity and disable all sorts of vertex processing 
effects to archieve the same. However, games expect to bypass clipping too, 
which can't be disabled in opengl. Because of that we scale the depth 
coordinate of the vertices, which we can do because transformed vertices are 
drawn in orthogonal space, so the size does not change.

This works for scenes containing only transformed vertices, but if an app 
mixes transformed and untransformed vertices to coordinate systems won't 
match correctly and the depth test will fail to remove hidden primitives. 
This patch disables the depth range scaling when transformed primitives are 
drawn, so the z order will be correct. If the app draws only transformed 
vertices the scaling is done to avoid loosing geometry

The correct solution would be to untransform the vertices and process them 
like untransformed vertices, but this would have serious performance impact 
and requires a lot of work for a software pipeline.
-------------- next part --------------
From nobody Mon Sep 17 00:00:00 2001
From: Stefan Dösinger <stefan at codeweavers.com>
Date: Mon Aug 21 18:02:34 2006 +0200
Subject: [PATCH] WineD3D: Correctly draw transformed and untransformed vertices in the
same scene

---

 dlls/wined3d/drawprim.c        |   10 ++++++++--
 dlls/wined3d/wined3d_private.h |    1 +
 2 files changed, 9 insertions(+), 2 deletions(-)

50f52b5802262bbf1ec5d46fea563389442459c0
diff --git a/dlls/wined3d/drawprim.c b/dlls/wined3d/drawprim.c
index 360efdd..f1b1a1a 100644
--- a/dlls/wined3d/drawprim.c
+++ b/dlls/wined3d/drawprim.c
@@ -182,8 +182,13 @@ void d3ddevice_set_ortho(IWineD3DDeviceI
         width  = This->stateBlock->viewport.Width;
         minZ   = This->stateBlock->viewport.MinZ;
         maxZ   = This->stateBlock->viewport.MaxZ;
-        TRACE("Calling glOrtho with %f, %f, %f, %f\n", width, height, -minZ, -maxZ);
-        glOrtho(X, X + width, Y + height, Y, -minZ, -maxZ);
+        if(!This->untransformed) {
+            TRACE("Calling glOrtho with %f, %f, %f, %f\n", width, height, -minZ, -maxZ);
+            glOrtho(X, X + width, Y + height, Y, -minZ, -maxZ);
+        } else {
+            TRACE("Calling glOrtho with %f, %f, %f, %f\n", width, height, 1.0, -1.0);
+            glOrtho(X, X + width, Y + height, Y, 1.0, -1.0);
+        }
         checkGLcall("glOrtho");
 
         /* Window Coord 0 is the middle of the first pixel, so translate by half
@@ -258,6 +263,7 @@ static void primitiveInitState(
     } else {
 
         /* Untransformed, so relies on the view and projection matrices */
+        This->untransformed = TRUE;
 
         if (!useVS && (This->last_was_rhw || !This->modelview_valid)) {
             /* Only reapply when have to */
diff --git a/dlls/wined3d/wined3d_private.h b/dlls/wined3d/wined3d_private.h
index 0010062..59f042b 100644
--- a/dlls/wined3d/wined3d_private.h
+++ b/dlls/wined3d/wined3d_private.h
@@ -507,6 +507,7 @@ typedef struct IWineD3DDeviceImpl
     UINT                    alphafunc;
     BOOL                    texture_shader_active;  /* TODO: Confirm use is correct */
     BOOL                    last_was_notclipped;
+    BOOL                    untransformed;
 
     /* State block related */
     BOOL                    isRecordingState;
-- 
1.2.4