[Winetest RFI] What do you want out of the test suite?

[Paul Vriens]

Detlef Riekenberg wrote:
> On Fr, 2007-08-10 at 12:15 +0200, Paul Vriens wrote:
>> I just want to get people's opinion(s) about the needs/requirements/whishes for 
>> both the conformance tests themselves (including winetest) as well as the webpages.
> 
Hi Detlef

> * Please make the links "Previous build" / "Next build" working!

This has been bothering me for ages. I only ever looked briefly into this, but 
either the links should be correct or be removed.

> 
> * I really like to Filter the Results by DLL:

What do you mean by filter? Currently the results are 'filtered' by test. Would 
you like a page just for your DLL?

>  - Try to find the Test-Result, when a new Failure popped up 
>    in a specific Windows Version.
>    (Current Example is oleauth32/vartest for w2k)

'New failure' = 'different result than the previous run' ?
> 
>  - Try to find the most recently Results for a Test.
>    (For Example, when a new Test was added)
>    Find this for a specific Windows Version.

The most recent results are in the last run. Or do you mean something else. I 
would like to have a 'latest' link on test.winehq.org/data as a very simple way 
of getting to the last version run.

> 
> * http://bugs.winehq.org/show_bug.cgi?id=3187
> 
> 
> * Can we add the selected locale to the results?
>   (Example issue are the current failures in oleauth32/vartest)

Do you want this in the report (text) or on the html page?

> 
> 
> * Can we add the tested dll-version to the results?
>   (This is a shortcut for the installed versions of:
>    IE, msi, DirectX, ...)

Again in the report or on the page? And only for the main dll (main dll equals 
the root directory of the test)? For msi that would be msi.dll, for the other 2 
you mention it will be more difficult as they are comprised of several dll's. So 
how would like to see that?

> 
> * winetest.exe: Allow "/?" as an alias for "-h"
> * winetest.exe: Allow other character in the TAG (underscore as example)
> * winetest.exe: Enable the "Stop" button
> * winetest.exe: make the progress-bars the same size
> * winetest.exe: Be more verbose during extract / test / upload 
>   (add "x of y" somewhere; add used time somewhere)
> 
> Can we optional collect the results with a tempfile to replace the Pipe?
> (set WINETEST_OUTPUT to the Filename as Example)
> 
> This will make it possible with an updated wine/test.h to add win16
> tests in the testsuite.
> 
> I do not expect to run winetest on win3.x, but just prevent the 16Bit
> API to fail.
> 
> 

These last ones I only glanced at, but will look deeper soon.

 From the first items in you list it looks you'd like some sort of search/input 
page that generator output. Is that correct?

Cheers,

Paul.