Conformance tests

[Dimitrie O. Paun]

On Thu, 21 Aug 2003, Ferenc Wagner wrote:

> recent discussion on the list ignited my interest again.
> Francois has a very nice page
> <http://fgouget.free.fr/wine/tests-en.shtml>, let's try to
> make better use of it!

Good, we need a new infusion of enthusiasm for this project :)
 
> 1. The precompiled binaries offered here would mean a great
>    help, were they updated regularly.

To achive this I think we need to refine the process to the
point where we build one file (an .exe), which is downloaded
automatically to some address. When we get there, we can
enlist a bunch of people to regularly run this process. But
if we require more manual intervention from them, they will
do it a few times and give up.

> 2. Maybe better, a framework could be developed which would
>    run all the tests, pack the results up and send it back
>    to Francois for easy integration into his page.

Yes! I proposed such a shell some time ago. Look for
"Winetest Shell" at the bottom of the page:
    http://winehq.org/site/fun_projects

> 3. The table on the page is too big for convenient use.  I
>    do not know about much structure to see; maybe a table is
>    not even needed.  Just version, date of test, if it is up
>    to date with the binaries on the page, and a list of
>    failing tests as links to the error messages would be
>    enough.  If it is not, we can still embed it into the
>    giant API database at http://24.229.94.2 (why is it not
>    linked from WineHQ?) and have a general SQL interface! :)
>    Hmm, some diffing capabilities might be useful indeed...

Hope you're kiddin', right? :) We should simplify things,
rather than complicate them. In general, all these entries
should be green, we a few red ones. The red ones should
be hyperlinks to a page listing all the problems (a cute
little popup window would be nice). The cells in the big
table should be rather small, so that everything fits on
the screen.
 
> 4. We could recruit people with access to the different
>    platforms and send them mails asking for a rerun whenever
>    the new tests are compiled.  Or advertise it on the devel
>    list only...

See 1. above...

-- 
Dimi.