Improving http://tests.winehq.org/data/

[James Hawkins]

On 10/25/07, Francois Gouget <fgouget at free.fr> wrote:
>
> Here are some things I noticed while using this site. Let me know if I
> it would help to make bug reports for these:
>
>  * Some result entries are red with a dash in them and a blue border.
> See the Windows 98 results for http://test.winehq.org/data/200710241000/
>    I assume these means that the test did not load. However we should
> distinguish two cases there:
>
>  - if it did not run because the tested dll did not exist at all, then
> it's not a test failure and thus the background should be green.
>    A typical case would be the crypt32 tests on Windows 98.
>
>  - if the dll was there but the test still did not run, typically
> because the dll is missing an export, then that's a bug in the test: it
> should dynamically load that function so the other checks it performs
> can be run.
>    A typical case is the gdi32 tests on Windows 98.
>    Ideally we'd even have a log showing the missing API but that's
> probably too tricky to do on Windows.
>
>  * Downloading the log file for a given test run gives me a file that's
> called 'report'. It would be nice if it was called something like
> 'vmware-win98-report' instead so that saving a couple of them in a
> directory leads to fewer collisions.
>
>  * It would be nice if the /data page looked more like a calendar with
> the most current date easily accessible, and a less like the directory
> listing it currently is. Then a / page would be nice too.
>
>    I'm pretty unlikely to fix these, but at least here is a starting if
> someone is inclined to do so: you will find the source for the
> test.winehq.org website in the tools.git repository:
>    http://source.winehq.org/git/tools.git/?a=tree;f=winetest
>
>    To get hold of the sources, see the instructions there:
>    http://www.winehq.org/site/git#modules
>

Looking at the test data, all of the msi:install tests timeout.  I
just ran the install tests in XP running under vmware on a 3ghz
machine.  The tests took 9m41s.  That completely blows away the 2min
timeout.  There's nothing wrong with the tests, they just take a long
time.   I don't think we should extend the timeout, because it's very
subjective and more tests will be added, meaning we'll have to change
the timeout eventually.  I do think we should have a flag or variable
that allows the timeout to be ignored for certain tests.  Any
opinions?

-- 
James Hawkins