GSoC Idea: Tools - Winetest Scripting Interface

[Francois Gouget]

On Mon, 7 Mar 2016, Zhenbo Li wrote:
[...]
> So we can do
> 
> winetest -c -t <tag> -m <email> -o result_with_old_driver.txt d3dx9_36
> d3d9 d3d8 etc.
> blah...
> winetest -c -t <tag> -m <email> -o result_with_new_driver.txt d3dx9_36
> d3d9 d3d8 etc.
> 
> Then compare these two results with external tools(like diff or vimdiff)

Yes. The missing part is the comparison where it's mostly going to be 
diff. The TestBot's WineSendLog script has a CompareLogs() function to 
only report new failures to patch submitters (behind the scenes it uses 
Algorithm::Diff). Maybe that can be useful to you:

https://source.winehq.org/git/tools.git/blob/HEAD:/testbot/bin/WineSendLog.pl#l139


> > > With the script, he could type
> > > wine_test_suite --test_all --output result.txt
> > > And Bob can get everything he wants after his lunch.
> > >
> > winetest can do this.

Actually it's more that WineTest does this by default. But make can do 
it too:
    make -k test


[...]
> Will it be too complicated if we allow developers to declare dynamic
> dependency manually?
> For example, I can add a line
> ## DEPENDENCY msxml3
> in dlls/mshtml/Makefile.in
> to declare that mshtml.dll will depend on msxml3.dll dynamically.

Something like could be useful if the TestBot is to ever run the tests 
on Wine too. The alternative is to always run all the tests and either 
trust that they have zero failures by default anyway (which is the 
goal), or that a diff will find new ones.

-- 
Francois Gouget <fgouget at free.fr>              http://fgouget.free.fr/
May your Tongue stick to the Roof of your Mouth with the Force of a Thousand Caramels.