Test system for the Widl compiler

[Rémi Bernon]

On 6/15/22 17:20, Bernhard Kölbl wrote:
> Hey everyone,
> 
> during the course of implementing WinRT / UWP stuff, I regularly come
> across missing IDL 3.0 features in our Widl compiler.
> Unfortunately, Widl's code seems to have grown in a rater unpleasant
> direction and is sometimes a real burden to understand and maintain. I
> think some refactoring to its code would be very well fitting and make
> things a lot easier for everyone, but refactoring can also bring
> breakage and makes tracking down errors with Git harder. Additionally,
> as the code is already not that small, errors have an easy way to slip
> in without noticing.
> So to me, the ideal solution seems to add a testing system to the
> compiler, like most of Wine's components already have.
> 
> Rémi and I already outlined some ideas:
> - Port the Winetest system to Unix programs
> - Make a libwidl or Widl.exe and test against that with our current test system
> - Add the tests into Widl itself as callable parameter
> 
> What do you think is the best solution?
> Getting some suggestions would be great. :)
> 
> Thanks,
> Bernhard
> 

Hi, I thought a bit more about this topic.

What we currently have, from the use of IDLs in Wine, as well as from 
the existing tests, are checks that validate that WIDL output is ABI 
compatible with MIDL output, but not really source compatible.

What we additionally need in order to make WIDL more robust and changes 
easier to review, are tests which:

1) Compare what MIDL supports in term of syntax to what we do, to check 
whether WIDL parsing changes are correct and required.

2) Compare what MIDL generates from a given IDL to what we do, to check 
whether WIDL code generation changes are correct.


In order to do that I think we actually have to write tests against 
native MIDL.

I think 1) is the most critical part, and it's actually the easiest. We 
can simply write some IDL snippets and run them through MIDL, checking 
its success or failure, and do the same with WIDL.

For 2), it's a bit more tricky, but I think with a bit of 
post-processing and string matching it should be possible to check 
whether the generated files have the expected constructs.


I've made a POC implementation of such tests for 1), in the attached 
patches. They introduces a new builtin midl.exe, that could ultimately 
support the same command-line options as MIDL and act as a drop-in 
replacement, as well as associated tests which require the Windows SDKs 
to be installed to find native midl.exe to run against.

(I'm only not sure that the use of PARENTSRC to share sources between 
tools/widl and the new programs/midl is the best way to do it.)


Cheers,
-- 
Rémi Bernon <rbernon at codeweavers.com>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0001-midl.exe-Add-new-program.patch
Type: text/x-patch
Size: 9729 bytes
Desc: not available
URL: <http://www.winehq.org/pipermail/wine-devel/attachments/20220706/b0423440/attachment.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0002-midl-tests-Add-tests-for-midl.exe.patch
Type: text/x-patch
Size: 10920 bytes
Desc: not available
URL: <http://www.winehq.org/pipermail/wine-devel/attachments/20220706/b0423440/attachment-0001.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0003-midl-tests-Add-support-for-multiple-source-files.patch
Type: text/x-patch
Size: 9906 bytes
Desc: not available
URL: <http://www.winehq.org/pipermail/wine-devel/attachments/20220706/b0423440/attachment-0002.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0004-midl-tests-Add-support-for-generating-output-files.patch
Type: text/x-patch
Size: 9450 bytes
Desc: not available
URL: <http://www.winehq.org/pipermail/wine-devel/attachments/20220706/b0423440/attachment-0003.bin>