GSoC Idea: Tools - Winetest Scripting Interface

[Zhenbo Li]

Hello, Henri

Thank you for your reply.

2016-03-04 20:26 GMT+08:00 Henri Verbeet <hverbeet at gmail.com>:
> On 4 March 2016 at 09:47, Zhenbo Li <litimetal at gmail.com> wrote:
>> Alice is developing graphics driver, and wine's test for
>> d3d(8,9,10,11) would be a great test case. She was following this
>> workflow.
>> 1. Edit graphics driver
>> 2. cd /path/to/wine; cd dlls/d3dx9_36/tests; make test;
>> 3. cd /path/to/wine; cd dlls/d3d9/tests; make test;
>> (test other graphics related dlls)
>> 4. Edit graphics driver again.
>> 5. Run wine tests again, and compare them manually.
>>
>> With improved wine test suite, her workflow can be
>> 1. Edit graphics driver
>> 2. wine_test_suite --test_dll d3dx9_36 d3d9 d3d8 etc. --output result.txt
>> 3. Edit graphics driver again
>> 4. wine_test_suite --test_dll d3dx9_36 d3d9 d3d8 etc. --compare result.txt
>> And the script will tell Alice whether she had introduced, or fixed a bug.
>>
> winetest can do this.
>

Do you mean setting the filter in winetest?

>> B.
>> Bob is writing a patch for wine headers. After modification, he typed
>> 'make test' and left the computer for lunch. When he came back, he
>> found that dlls/comdlg32/tests failed(it always happens on my PC), and
>> the make script stopped. He needed to spend another hour to test the
>> remaining components.
>> With the script, he could type
>> wine_test_suite --test_all --output result.txt
>> And Bob can get everything he wants after his lunch.
>>
> winetest can do this.
>
Thank you for mentioning that.


>> C.
>> Charlie is working on wininet. He was afraid that his patch might
>> broke other dlls which depend on wininet. With this script, he can run
>> wine_test_suite --group net
>> And all cases related to net can be tested(net is pre-configured in the script)
>>
> It might be more interesting if the test suite could figure out the
> tests potentially affected by a range of commits on its own. I.e., if
> I make a change to wined3d that potentially affects d3drm and d2d1,
> among others.

This sounds good. A straight approach is to scan Makefile.in, and
generate a dependence tree from IMPORT and DELAYIMPORTS. However, I'm
wondering how to handle the dynamic dependencies. For example, in
dlls/mshtml/xmlhttprequest.c, we have
    hres = CoCreateInstance(&CLSID_DOMDocument, NULL,
CLSCTX_INPROC_SERVER, &IID_IXMLDOMDocument, (void**)&xmldoc);
And the development for msxml really affected mshtml.dll's behavior.

I did a roughly estimation
$ git grep CoCreateInstance|grep -v spec|grep -v tests| wc -l
162

Maybe we can write a dependency detector, and add some exceptions for
critical dynamic dependencies?

Thank you.

-- 
Have a nice day!
Zhenbo Li