[Tools 6/6] winetest: Issue a better description than '.' for the special failure modes. Write them as a note so they stand out better.
Francois Gouget
fgouget at free.fr
Mon Mar 17 12:18:08 CDT 2008
On Mon, 17 Mar 2008, Paul Vriens wrote:
[...]
> We currently have these single '.' also when there are multiple failures for
> an OS that are actually the same failures.
Yes, I've noticed that. Essentially we lose information when merging bug
reports. Fix That's next on my todo list. Here's my (optimistic) todo
list:
* Fix merging report data so the group-level information is more
accurate.
* Collect statistics about the number of unit tests that failed and
their percentage, both for individual reports and for groups.
* For each build, collect global statistics about the group-level
results, how many reports of each type are present, etc. Store that
in a new machine readable file (a bit like the individual report's
summary.txt files).
* Add a new script that takes these build-level summaries and generates
a proper top-level index file showing the percentage of failed unit
tests and the count of reports for each group.
That last bit would make progress more visible, and would make it easier
to find a build which has been tested on a given platform.
> For example the missing dll's. There are several dll's missing on win95 and
> win98 so it would be nice to have this reflected in the group piece of the
> page. I think green with a blue border looks much nicer than the single '.' we
> are currently having. Other thoughts?
Well, if, for a given test, all the reports in the group have the
same state ('crashed', 'timed out', 'errors', etc.), then the group's
cell should look exactly like each of the corresponding report cells. So
in the missing Win95 dlls, this would indeed be a 'pass also-skip' cell.
But if they all crashed, then it should be a white cell with
the 'crashed' message.
If the cells have different states, for instance one says 'crashed' and
the other 'timed out', or one says 'not run' and the other says errors
occurred, then I think the group cell should say something like 'mixed'
and be on a white background.
Maybe the 'filelimit' and 'truncated' cells should be ignored when
merging, though we should really not have any of these either.
> The new page layout is now present and I must say it looks better although
> Wine looks a bit worse now because all the green is gone :-).
Yeah :-/ But I think it more accurately reflects the state of Wine.
Hopefully it's also more motivating for the people fixing Wine to
eliminate todos.
--
Francois Gouget <fgouget at free.fr> http://fgouget.free.fr/
La terre est une b
More information about the wine-devel
mailing list