[PATCH 6/6] d3dx9_36/tests: fixed order of parameters

[Rémi Bernon]

On 9/29/21 11:48 AM, Henri Verbeet wrote:
> On Wed, 29 Sept 2021 at 09:19, Eric Pouech <eric.pouech at gmail.com> wrote:
>>
>> to be complete:
>> gcc11 complains with:
>> In function 'unit_vec2',
>>      inlined from 'attempt_line_merge' at /home/eric/work/wine/dlls/d3dx9_36/mesh.c:5542:5:
>> /home/eric/work/wine/dlls/d3dx9_36/mesh.c:5511:12: warning: 'lastdir' may be used uninitialized [-Wmaybe-uninitialized]
>>   5511 |     return D3DXVec2Normalize(D3DXVec2Subtract(dir, pt2, pt1), dir);
>>        |            ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>> In file included from /home/eric/work/wine/include/d3dx9.h:31,
>>                   from /home/eric/work/wine/dlls/d3dx9_36/d3dx9_private.h:32,
>>                   from /home/eric/work/wine/dlls/d3dx9_36/mesh.c:31:
>> /home/eric/work/wine/dlls/d3dx9_36/mesh.c: In function 'attempt_line_merge':
>> /home/eric/work/wine/include/d3dx9math.h:352:21: note: by argument 2 of type 'const D3DXVECTOR2 *' to 'D3DXVec2Normalize' declared here
>>    352 | D3DXVECTOR2* WINAPI D3DXVec2Normalize(D3DXVECTOR2 *pout, const D3DXVECTOR2 *pv);
>>        |                     ^~~~~~~~~~~~~~~~~
>> /home/eric/work/wine/dlls/d3dx9_36/mesh.c:5527:25: note: 'lastdir' declared here
>>   5527 |     D3DXVECTOR2 curdir, lastdir;
>>        |                         ^~~~~~~
>>
>> as D3DXV2Normalize isn't symetrical in its arguments definition (first arg is const ptr as input, while second arg is ptr as output)
>> so gcc11 complains about passing as first arg (being a const ptr) an uninitialized object (it has no way of knowing that both args are aliased)
>>
> I don't think that's exactly what's happening. In particular, note
> that the message complains about the second argument to
> D3DXVec2Normalize(), not the first. My guess is that it doesn't notice
> that the D3DXVec2Subtract() call initialises "dir" before the call to
> D3DXVec2Normalize() actually happens. That's admittedly a little
> tricky; the straightforward translation would first push "dir" to the
> stack, which points to uninitialised memory at that point, then call
> D3DXVec2Subtract() which initialises the vector pointed to by "dir",
> push the return value of D3DXVec2Subtract(), and finally call
> D3DXVec2Normalize().
> 

The problem actually comes from D3DXVec2Subtract which can return NULL 
if any of the input pointer is.

It's a little bit convoluted, but although the code takes the address of 
the vector, it's still possible to end up with NULL pointers.

For instance if outline->items == NULL and pt_index == 0, pt or prevpt 
(if pt_index == 1, or wraps around) may be NULL. Then, as pos is the 
first member, &pt->pos (or &prevpt->pos) may be NULL as well.

Of course, outline->items is probably not NULL, but that may be too 
complicated to infer.

Changing pos to be second point2d member solves the warning too.
-- 
Rémi Bernon <rbernon at codeweavers.com>