Test system for the Widl compiler

[Nikolay Sivov]

On 6/15/22 18:20, Bernhard Kölbl wrote:
> Hey everyone,
>
> during the course of implementing WinRT / UWP stuff, I regularly come
> across missing IDL 3.0 features in our Widl compiler.
> Unfortunately, Widl's code seems to have grown in a rater unpleasant
> direction and is sometimes a real burden to understand and maintain. I
> think some refactoring to its code would be very well fitting and make
> things a lot easier for everyone, but refactoring can also bring
> breakage and makes tracking down errors with Git harder. Additionally,
> as the code is already not that small, errors have an easy way to slip
> in without noticing.
> So to me, the ideal solution seems to add a testing system to the
> compiler, like most of Wine's components already have.
>
> Rémi and I already outlined some ideas:
> - Port the Winetest system to Unix programs
> - Make a libwidl or Widl.exe and test against that with our current test system
> - Add the tests into Widl itself as callable parameter
>
> What do you think is the best solution?
> Getting some suggestions would be great. :)

There are options. First it depends whether you want to validate against 
midl or not. If you do then you'll need something like widl.exe to run 
on Windows.

Next question is what your test data would be, and how to compare the 
output reliably with expected output. For example we don't really need 
exact matching for generated headers, or even marshaling format strings, 
as long as they achieve their purpose. For typelibs however, it might be 
useful to go for exact binary match of selected sections.

If testing against midl is not a goal, we could have tests for widl 
itself, even in form of scripts, to run on host system, and not through 
wine. This way you'll need expected test data that you trust, and 
maintain in a good shape without validating it on Windows. Even if it 
doesn't match midl output exactly, it's still potentially useful to 
catch regressions.

>
> Thanks,
> Bernhard
>