[ANN] Conformance testing campaign

[Dimitrie O. Paun]

On Tue, 2 Sep 2003, Ferenc Wagner wrote:

> > The winetests shell is comming along nicely, I don't think
> > it's worth dupicating effort. It will do all the decompression,
> > sending of results, cleanup.
> 
> Maybe you misunderstood me.  The question is about
> collecting the MSVC compiled test binaries.  Or is that
> covered, too?

Yes, the winetests.exe should contain the compiled test
binaries as well.

> > We should include Wine as one of the Windows version.
> 
> Nobody submitted that...

Right, I did not realize. To make it clearer, maybe we should
control the columns manually, rather than dinamically. We
know what we expect, and an empty column will also give an
indication of what tests we're missing.

> > I suggest the cells be coloured green, yellow and red. The
> > colour is based on the following:
> >   -- green: all tests received passed
> >   -- yellow: some tests received failed
> >   -- red: all tests received failed
> 
> What is in the cells?

That's a good question. Maybe number of errors? Number of
tests we received? I dunno...

> > The yellow and red cells should be clickable and a popup
> > should detail the problems.
> 
> Now an error cell is clickable if there is some output.
> Even passed tests can produce some traces, although rarely.

Right, we are on same wavelenght. It's just that I find the
output a bit too heavy, with the 3 numbers in there. A minor
point though. But having less in the cells will also make them
smaller, and we can fit more on the screen. This will be even
better when most of them are green :)

> > Ideally we should be able to extract them out of the email
> > regardless of how it was included/attached.
> 
> Not a serious problem.  We have munpack and file and a
> handful of possibilities.
> 
> > But all this is low priority because 99.99% of them will
> > be sent by winetests I suspect, so we know exactly how
> > they will be included.
> 
> Maybe the report formats should converge then.  I find mine
> quite convenient, and have a parser for that... :)

For sure. The two efforts are very much complementary, and
we should poll our efforts. winetests will handle just the
client part, so if you need the output in a different format,
just say so.

-- 
Dimi.