Test system for the Widl compiler

[Zebediah Figura]

On 6/15/22 13:21, Rémi Bernon wrote:
> On 6/15/22 20:04, Zebediah Figura wrote:
>> On 6/15/22 12:49, Nikolay Sivov wrote:
>>>
>>>
>>> On 6/15/22 18:20, Bernhard Kölbl wrote:
>>>> Hey everyone,
>>>>
>>>> during the course of implementing WinRT / UWP stuff, I regularly come
>>>> across missing IDL 3.0 features in our Widl compiler.
>>>> Unfortunately, Widl's code seems to have grown in a rater unpleasant
>>>> direction and is sometimes a real burden to understand and maintain. I
>>>> think some refactoring to its code would be very well fitting and make
>>>> things a lot easier for everyone, but refactoring can also bring
>>>> breakage and makes tracking down errors with Git harder. Additionally,
>>>> as the code is already not that small, errors have an easy way to slip
>>>> in without noticing.
>>>> So to me, the ideal solution seems to add a testing system to the
>>>> compiler, like most of Wine's components already have.
>>>>
>>>> Rémi and I already outlined some ideas:
>>>> - Port the Winetest system to Unix programs
>>>> - Make a libwidl or Widl.exe and test against that with our current 
>>>> test system
>>>> - Add the tests into Widl itself as callable parameter
>>>>
>>>> What do you think is the best solution?
>>>> Getting some suggestions would be great. :)
>>>
>>> There are options. First it depends whether you want to validate 
>>> against midl or not. If you do then you'll need something like 
>>> widl.exe to run on Windows.
>>>
>>> Next question is what your test data would be, and how to compare the 
>>> output reliably with expected output. For example we don't really 
>>> need exact matching for generated headers, or even marshaling format 
>>> strings, as long as they achieve their purpose. For typelibs however, 
>>> it might be useful to go for exact binary match of selected sections.
>>
>> We already have tests for typelibs and marshalling, in 
>> oleaut32:typelib and oleaut32:tmarshal respectively. I see no reason 
>> that's not sufficient already.
>>
>> As far as headers are concerned, we could probably take a similar 
>> approach in some ways. I don't know what exactly needs testing, but we 
>> could do things like
>>
>> ok(offsetof(mystruct.field) == 16,
>>          "wrong offset %u\n", offsetof(mystruct.field));
>>
>> or (for a compile-time assertion)
>>
>> extern void test(ITestInterface *obj)
>> {
>>      HRESULT (*myfunc)(type1 arg1, type2 *arg2, ...);
>>
>>      myfunc = obj->lpVtbl->myfunc;
>> }
>>
>> Ultimately we may not need to mess with widl, or write any extra test 
>> infrastructure.
>>
> 
> I don't agree and I think it'd be good to have widl parsing tests, to 
> validate parser grammar additions, expected parser errors, and the error 
> messages we print if we want to make them better. It will improve the 
> tool quality and help validating any widl change.

I'm not sure what you "don't agree" with, I was just proposing a way to 
test header contents, and I didn't immediately see any other things that 
needed testing.

Testing that certain IDLs fail to compile seems reasonable, although 
checking the exact message strikes me as more restrictive than necessary...