GSoC Idea: Tools - Winetest Scripting Interface

[Zhenbo Li]

On 02/29/2016 01:00 AM, Stefan Dösinger wrote:
 > The testbot is mostly able to do those things. I don't think it is
 > able to match line number to function name though.
 >
 > Most of the tests in wine are not interesting to mesa. I expect them
 > to care about d3d and GL related stuff, maybe gdi, but something like
 > riched20 or wininet isn't really related to the GL library.

I know that d3d is more concerned in mesa community. I used riched20 
because it has less dependencies, and my test results can be easier to 
be reproduced.

On 02/29/2016 01:50 AM, Henri Verbeet wrote:
> On 28 February 2016 at 18:00, Stefan Dösinger <stefandoesinger at gmail.com> wrote:
>> The interesting thing is splitting up the output of the big d3d tests
>> in e.g. dlls/d3d9/tests/visual.c into reasonable chunks, preferably
>> matching the functions the tests are in. So that you go from
>> "d3d9/visual is failing" to "d3d9/visual/depth_bounds_test failed,
>> d3d9/visual/test_vshader_input successful".
>>
>> Ideally this should be stable with regard to changes to the tests.
>> E.g. if 10 lines of code are added at the top of the file it shouldn't
>> break your output processing because the failing ok() lines are in
>> different lines of the file now.

Yeah, line number is not reliable. Look at such code

static void test_foo()
{
     // Some code always changes
     a = foo();
     ok(a, "Test failed\n");
     b = foo();
     ok(b, "Test failed\n");
}

It's very common for developers to run a bisect on test_foo. Our script 
should distinguish ok(a) and ok(b). I haven't thought up an idea for 
this situation yet.

> It shouldn't be terribly hard to add a "function" argument to
> winetest_set_location(), although there may also be something to be
> said for introducing e.g. a winetest_begin_subtest() function instead.
>

With GCC's extension __FUNCTION__ (or C99)[1], it's easy to print 
function name[2], if we don't have some tricky code like

#define test_foo(a) _test_foo(l,a)
static void _test_foo(line, a) {
     ok_(line, foo(a) == S_OK, "foo failed\n");
}
static void test_bar()
{
     a = bar();
     test_foo(a);
}

Such code is very common in wine, and in logic, if foo(a)!=S_OK, we 
should report "test_bar" failed, instead of "_test_foo". Maybe we can 
provide a python-generator-style macro, like

#define generator(foo)   some_magic_here
static void generator(foo)(real_func_name, a)
{
     ok_magic(foo(a) == S_OK, real_func_name " failed\n" );
}
static void test_bar()
{
     a = bar();
     generator(foo)(a);
}

And the when ok_magic failed, developers can know that this error is 
related to test_bar.

However, I don't think this approach makes sense. Maybe no developers, 
except my script would like such code.

[1]: https://gcc.gnu.org/onlinedocs/gcc-3.1/gcc/Function-Names.html
[2]: https://github.com/Endle/wine-gsoc/tree/func_name