[ANN] Conformance testing campaign

[Dimitrie O. Paun]

On Mon, 22 Sep 2003, Ferenc Wagner wrote:

> "Dimitrie O. Paun" <dpaun at rogers.com> writes:
> 
> >   -- for the ME case, how can we have have some
> >      results (up to kernel32.dll:codepage) and then
> >      have no results? Doesn't that mean that they failed?
> 
> No, this means that when the console test hung the tester
> killed the DOS box and thus did not run further tests.
> Jakob might implement a timeout or we could explain more.

Right, this was my point: it's more of a failure, than
not having run the test (displayed as "."). Maybe we should
say "timeout" for these?

> >   -- It would be nice to make the tests name links to
> >      the test file. [...] This is tricky though, as it is
> >      not clear how to figure this thing out.
> 
> Yes, it must be included in the output.  I gave it a try,
> winsock and kernel32 are tricky, for example.

Yes, I've noticed, it's cool (however, as you say, some
of the links are wrong also).

> >   -- How do you assign the name to different reports
> >      for the same OS?
> 
> It is the name of the directory the data comes from.  In
> principle, testers could provide their tags.

How do you make sure they don't collide? If we are to
install this on WineHQ and have it run automatically,
I guess we'll have to create temp dirs. At which point,
that stuff is useless anyway, so I think we should just
drop it altogether. That is, instead of having stuff like:

           Win95  Win95dimi  Wine95joe .etc
	<rest of the table .... >

We should just drop that from the column header, and
say
Win95:
	<rest of the table .... >


> >   -- Also, in the "XXX differences" section, shouldn't we
> >      have the exact version & ServicePack displayed between
> >      the OS name and the reporter link, as we are dealing
> >      with only one instance?
> 
> Sure, but I do not have the information.  Noted, though.

How come? I thought Jakob includes a dump of the OS version
structure, like so:
Operating system version:
dwMajorVersion=5
dwMajorVersion=0
dwBuildNumber=2195
dwPlatformId=2
szCSDVersion=Service Pack 3
wServicePackMajor=3
wServicePackMinor=0
wSuiteMask=0
wProductType=1
wReserved=30

> >   -- A few links at the top of the page would be nice:
> > 	1. To the main Testing page, something like
> > 		/site/testing
> 
> I am not sure what this main Testing page is...

We will need a Testing status page on WineHQ no? We need a
master page which explains where to get the latest tests,
what you need to do, link to the results, etc. Not done
yet, but needed :)
 
> I was on holiday last week and will possibly leave again,
> but will surely come back, just wait...

Cool, I can't wait to have this done and integrated in WineHQ.

-- 
Dimi.