Your weekly winetest update

[Francois Gouget]

On Tue, 13 Feb 2018, Zebediah Figura wrote:

> On 11/02/18 18:11, Francois Gouget wrote:
> > Anyway, we might have to increase the limit at some point, but keeping 
> > the log sizes reasonable is important too. So I sent a patch that marks 
> > tests that output more than 32KB of traces as failed on test.winehq.org 
> > in order to push for a review / refactoring of those tests. If the patch 
> > is accepted I can send the same one for the TestBot so patches that 
> > bloat the traces are caught early.
> > 
> 
> Apologies if this has been proposed before, but would it perhaps make
> sense to run the tests with -q, so traces and todos are filtered out?

That would make the tests harder to debug, particularly for intermittent 
failures and non-TestBot results where it's not as easy as just 
re-running the test.

Maybe a compromise would be to have WineTest filter out the traces if 
the test succeeds. I'd be wary of relying on just re-running the test 
with traces on, again because of intermittent failures and because tests 
that fail may cause the next run to fail differently.

But even so we may still lose too much as comparing the traces on a 
successful test with those of a failing one can yield insight into where 
and how they are diverging.


-- 
Francois Gouget <fgouget at codeweavers.com>