D3D8 Software Vertex Processing

[Jan Havran]

Bingo!

I finally managed to install Windows 7 on my laptop and tried some 
debugging (I was comparing it to Wine). You was right, I have found that 
the game calls some D3D8 function where it gets D3DCAPS8 structure. In 
that structure it tests MaxVertexBlendMatrixIndex (offset 0xAC) and 
depend on that value it executes one or another branch. On Windows 
MaxVertexBlendMatrixIndex is set always to 255 as it says MS Dev Center 
for SWVP [1] (well, it is for D3D9, but i think it should be the same as 
D3D8). But the problem is that this value is set to 0 in Wine ([1] says 
that 0 means following: "device does not support indexed matrices"). As 
I have never heard about this issue on Windows, I guess that indexed 
matrices are always supported on Windows. I tested SWVP and HWVP on both 
Windows and Wine and here are the results:

Vietcong (Windows) with SW Vertex: 255
Vietcong (Windows) with HW Vertex: 126
Vietcong (Wine)    with SW Vertex: 0
Vietcong (Wine)    with HW Vertex: 126

So the game expects (was tested) 255 in SWVP but it gets 0. The minimum 
value of MaxVertexBlendMatrixIndex for which the application runs fine 
is 127, while anything below it (like 126 for HWVP or 0 for Wine 
SWVP/HWVP) will execute some broken branch (but for certain soldier 
figures only).

When I set this value to 255 in dlls/d3d8/device.c the game do not crash 
anymore, but instead of that there are some rendering problems (as you 
predicted, so you was right again). See attachment in [2].

How hard would be to fix it for D3D8 (and D3D9)?

[1] 
https://msdn.microsoft.com/en-us/library/windows/desktop/bb206316(v=vs.85).aspx
[2] https://bugs.winehq.org/show_bug.cgi?id=9337#c44


On 6.8.2017 22:25, Jan Havran wrote:
>
>
> On 6.8.2017 20:36, Stefan Dösinger wrote:
>> On 06/08/17 17:19, Jan Havran wrote:
>>> Can I simply edit some code in Wine (some D3D flag)
>>> so the game will think that it use SW vertexes, so after checking some
>>> flags by game it will not jump into some unfinished branches for HW
>>> vertexes? I just want to test if it is really caused by this. From
>>> application view, what is difference between using SW and HW vertexes,
>>> do you think that using HW vertexes with code done for SW vertexes will
>>> work?
>> This behavior should already be in place. If the app requsts a software
>> vertex processing device and asks us for the device description later we
>> should report that this is a software vertex processing device. We even
>> do some software-like things if the app wants SWVP, like ignore
>> D3DLOCK_DISCARD on buffers.
>>
>> The game may check some derived properties though and expect some
>> capability that SWVP has, but HWVP does not. The most likely case is
>> support for more than 256 vertex shader constants. Modern GPUs support
>> much more than that in OpenGL, but not in d3d8/9 on Windows. Because of
>> this we filter this in dlls/d3d8/device.c, d3dcaps_from_wined3dcaps().
>> Look for D3D8_MAX_VERTEX_SHADER_CONSTANTF at the end. It is likely that
>> this restriction is not in place on Windows with SWVP. You can try to
>> just disable this line.
>
> Wine is big magic to me right now, so thank you for your tips, I will 
> try to test that.
>
>>
>> Another possible culprit is fixed function vertex blending using matrix
>> indices. You can try to set MaxVertexBlendMatrices and
>> MaxVertexBlendMatrixIndex to 4 and 256 respectively (check on Windows if
>> those values are accurate, I might misremember them). This functionality
>> is unimplemented in Wine, so expect rendering problems if the game uses
>> this. It should be possible to implement this feature in our fixed
>> function replacement shader though.
>
> Ok, I will test it. Unfrotunately I didnt find piece of code where the 
> game decide which vertex processing it will use yet - maybe I will 
> install Windows so I will be able to compare execution of branches 
> between Windows and Linux.
>
>> The bug report doesn't have a crash dump. The similarity to windows HWVP
>> might mislead you. Try to compare your crash to the one of windows HWVP.
>> in particular, see if the crash address is the same.
>
> Well, I was talking about my issue with Brchi and we came to the 
> conclusion that it was crashing in the same function for both Linux 
> (Wine) and Windows (with enabled HWVP). Unfortunately, there is 
> version of his util without HWVP enabled (which doesnt work in Wine as 
> I said, because the game use branches for HWVP here) and after that 
> version with HWVP enabled only, but with fixed game code (by memory 
> patching) so there is no way how can I test it on Windows (because I 
> dont have version of the util where it is crashing for Windows, since 
> there is no version with HWVP enabled and without fix of this issue at 
> the same time).
>
> Anyway, this game has plenty of problems on newer Windows, but I never 
> heard about this issue even on Windows 10.
>
> I will try some tests you described and report back as soon as possible.
>
> Thanks for your time.
>
>