[PATCH vkd3d v2] vkd3d-shader/hlsl: Return the shape of the left operand when appropriate.

[Giovanni Mascellani]

This supersedes 217302. Also, 217464, from the same patch set, is revoked.

Thanks, Giovanni.


Il 19/10/21 09:32, Giovanni Mascellani ha scritto:
> The assumption about the size of matrices is not correct: it is
> legitimate to compose a matrix 2x2 with a vector of length 4, in which
> case it appears that the result has the shape of the first (leftmost)
> operand. Even for matrices 1xN or Nx1, the result is not always a
> vector: in general it has the shape of the first operand again.
> 
> Signed-off-by: Giovanni Mascellani <gmascellani at codeweavers.com>
> ---
>   libs/vkd3d-shader/hlsl.y | 13 +------------
>   1 file changed, 1 insertion(+), 12 deletions(-)
> 
> diff --git a/libs/vkd3d-shader/hlsl.y b/libs/vkd3d-shader/hlsl.y
> index bfe2bbb9..c42e372e 100644
> --- a/libs/vkd3d-shader/hlsl.y
> +++ b/libs/vkd3d-shader/hlsl.y
> @@ -971,18 +971,7 @@ static bool expr_common_shape(struct hlsl_ctx *ctx, struct hlsl_type *t1, struct
>       }
>       else
>       {
> -        /* Two vectors or a vector and a matrix (matrix must be 1xn or nx1) */
> -        unsigned int max_dim_1, max_dim_2;
> -
> -        max_dim_1 = max(t1->dimx, t1->dimy);
> -        max_dim_2 = max(t2->dimx, t2->dimy);
> -        if (t1->dimx * t1->dimy == t2->dimx * t2->dimy)
> -        {
> -            *type = HLSL_CLASS_VECTOR;
> -            *dimx = max(t1->dimx, t2->dimx);
> -            *dimy = 1;
> -        }
> -        else if (max_dim_1 <= max_dim_2)
> +        if (t1->dimx * t1->dimy <= t2->dimx * t2->dimy)
>           {
>               *type = t1->type;
>               *dimx = t1->dimx;
>