[winetest] Show missing tests in single/group results
Paul Vriens
paul.vriens.wine at gmail.com
Sun Aug 5 14:55:14 CDT 2007
Paul Vriens wrote:
> Hi,
>
> (I've just remembered that there is a limit on the email size, so I'm
> doing it a
> bit differently now. The other email, including the screenshots, will
> probably come through as well)
>
> Before sending this to wine-patches I would like people to have a look
> first.
>
> This patch makes sure that missing tests are shown in the test results,
> both
> when there is only one system in a group (f.e. Win98) or multiple
> systems in a
> group.
>
> On 'http://www.xs4all.nl/~pvriens/200708031000.old.single/' you can see the
> current state of the test results. When a test is missing a '.' is shown
> (this is of course different from a missing test where we don't have
> results at all but that's shown as a '.' as well).
>
> On 'http://www.xs4all.nl/~pvriens/200708031000.old.group/' you can see the
> current state when a group of systems is involved. When tests are
> missing for a
> group, a '.' is shown with a link to the group overview. There is
> however no
> missing test output in the group overview (for example gdiplus and pdh)
>
> 'http://www.xs4all.nl/~pvriens/200708031000.new.single/' shows the new
> results
> page for single tests. Instead of a single '.' there is a failure shown
> (red
> box) and a skip border (as these tests have not run). There is still a
> '.' which
> could maybe better be a '?'. There is also a 'onMouseOver' hint when you
> go over
> the '.'.
>
> 'http://www.xs4all.nl/~pvriens/200708031000.new.group/' shows the new
> results
> when a group is involved. The main summary is still the same as in the
> old one,
> but the group results now also show the missing tests (see gdiplus and pdh
> again). Again there is a 'onMouseOver' hint in the group results.
>
> Remember that this is still without winetest producing output about
> missing dll's.
>
> Next patches will:
>
> - add a missing dll as one of the reasons for missing tests
> - add some better clue's on the main summary for multiple systems in a
> group
> (not a single '.' but maybe the same 'failed and skipped').
>
> As said, one of the thoughts I had was to replace the '.' with a '?' in
> case of
> missing tests for an unknown reason but I'm not sure about that yet.
>
> Comments/remarks etc.. are welcome.
>
> I'd also like to hear about other idea's for this page to make it useful
> for
> everybody. Apart from the fact that we probably should have more
> exposure to the
> fact that we have such a page/tool.
> We saw an immediate rise (slowed down again) in test results as soon as
> winetest
> was mentioned on
> 'http://tweakers.net/meuktracker/15817/Wine-0.9.42.html' (in
> Dutch). There it was also mentioned that some machines crashed because
> of the
> tests etc.. (maybe a wine-winetest component on bugzilla could help with
> that
> which we could reference on the main test.winehq.org/data/*/index.html
> pages).
>
> Cheers,
>
> Paul.
>
Forgot to mention that only the index.html file is on the mentioned pages. No
result files (to cut down on size for me to push).
I also forgot to attach the patch. So here it is.
Cheers,
Paul
-------------- next part --------------
--- /wine/tools/winetest/gather 2007-08-03 17:20:09.000000000 +0200
+++ gather 2007-08-05 14:02:42.000000000 +0200
@@ -121,6 +121,22 @@
push @{$group->{tests}}, $testref;
}
+# Find missing tests. After this exercise all testresults are available
+# for all systems.
+foreach my $group (@groups) {
+ next unless exists $group->{tests};
+ foreach my $test (@{$group->{tests}}) {
+ foreach my $testname (sort keys %alltests) {
+ if (!exists $test->{results}->{$testname}) {
+ # Make sure missing tests are shown in the group results
+ $group->{digests}->{$testname} = "differ";
+ # Mark this test as missing for an unknown reason
+ $test->{results}->{$testname} = ["test missing", "-", "-", "-"];
+ }
+ }
+ }
+}
+
# Write out the tables
my ($header); # same in thead and tfoot
@@ -201,28 +217,30 @@
# Output a single cell of a test
sub singletest {
my ($test, $testname, $groupname) = @_;
-
- if (!exists $test->{results}->{$testname}) {
- print OUT " <td class=\"note\">.</td>\n";
+ my $file = "$test->{dir}/$testname.txt";
+ my ($count, $todo, $error, $skipped) = @{$test->{results}->{$testname}};
+ if ($count eq "failed") {
+ my $msg = $todo eq "-"?"crash":
+ $todo == 258?"timeout":"failed";
+ my $fail = -r "$datadir/$build/$file"?"<a href=\"$file\">$msg</a>":$msg;
+ print OUT " <td class=\"note\">$fail</td>\n";
+ } elsif ($count eq "test missing") {
+ print OUT <<"EOF";
+ <td class="skip_fail"><a
+ title="Test did not run for an unknown reason"
+ onMouseOver="Test did not run for an unknown reason";
+ >.</a></td>
+EOF
} else {
- my $file = "$test->{dir}/$testname.txt";
- my ($count, $todo, $error, $skipped) = @{$test->{results}->{$testname}};
- if ($count eq "failed") {
- my $msg = $todo eq "-"?"crash":
- $todo == 258?"timeout":"failed";
- my $fail = -r "$datadir/$build/$file"?"<a href=\"$file\">$msg</a>":$msg;
- print OUT " <td class=\"note\">$fail</td>\n";
- } else {
- my $class = $error?"fail":"pass";
- my $skip = $skipped?"skip_":"";
- print OUT <<"EOF";
+ my $class = $error?"fail":"pass";
+ my $skip = $skipped?"skip_":"";
+ print OUT <<"EOF";
<td class="$skip$class"><a
href="$file"
title="$count tests, $todo TODO, $error errors, $skipped skipped"
onMouseOver="refresh('$testname','$groupname $test->{tag}',$count,$todo,$error,$skipped);"
>$error</a></td>
EOF
- }
}
}
More information about the wine-devel
mailing list