current wine status pr-1

[Paul Millar]

Hi Tom,


On Mon, 2 Sep 2002, tom wrote:
> Paul Millar wrote:
> >On Sun, 1 Sep 2002, tom wrote:
> >>I have a pre-1 hacked together and request any feed back that you can give.
> >
> >I'm doing rolling CVS regression/conformance testing and reporting back
> >whenever something breaks.  How about adding an extra line under the "Wine
> >Regression Test Suite"?
> >
> Hi Paul,
> 
> Is this ** Part ** of the Regression Test Suite or a different
> category to add ? 

It could probably be argued either way ... :) In the limited meaning of 
Regression Test Suite, then the answer is no.

> I am not a developer so please look over me if this 
> is a dumb
> question ... If it is part of the Regression Test Suite I will just add 
> you in as a worker
> with CodeWeavers .

Well, I don't claim to have written anything in the Regression Test Suite
(only one completely trivial patch), so I'd rather my name wasn't in the
"workers" section.

> If it is a Different category I think it shold be like this
> 
> Rolling cvs testing
> Documentation = 
> workers = Paul Millar
> % = 10%

Hmmm, yes. On the whole, I think this is better. You can put me down for
Documentation too. I'm not sure what (if any) documentation is really
needed, but I think there's a reasonable amount at:
  http://www.astro.gla.ac.uk/users/paulm/WRT/
and
  http://www.astro.gla.ac.uk/users/paulm/WRT/help.php

Its difficult to gauge a percent-completeness for the rolling regression
testing.  I'm quite happy with the functionality as it stands now
(although I've got a few ideas for improvements). Its been quite stable
(apart from some problems with thread deadlocks) for quite a while now, so
I'd say its probably around 90% or so.


> Is the ( Regression Test Suite ) a program to test the Regression % or
> the % of the Regression ? As one is a app the other is a %
> If it is a app do you know the current % done  ? 

I don't know if this answers your question, but the Regression Test Suite
is a series of explicit statements of what is correct behaviour of the
Windows API.  Calling function Foo() with parameter "Bar" should return
"Baz" : that sort of thing.  Information is gleaned from MS's official
documentation and from direct testing of various Windows platforms.  
There's a little more info on the first URL above.

Ideally we want all the functions implemented by wine to precisely match
the result from Windows under all circumstances (or at least all
circumstances where it matters for a program). This behaviour should then
be tested to make sure nothing breaks at a later date.

For calculating a percentage, the script calculates the number of APIs
implemented, and the number of those functions that appears in at least
one regression test. The idea is divide one by the other and you have a
measure of what percent of wine is tested by the Regression Suite (hence
how complete it is).

Unfortunate the estimate almost certainly an over-estimate. The calculated
number of implemented API is via a crude grep. The estimate also assumes
that, just because a function is mentioned in a regression test, the
function should be counted as "tested". Both bias towards an increased
percentage, so the 10.3% value should be taken with a generous pinch of
salt :)

Cheers,

Paul.

----
Paul Millar