RESENT: running confirmance tests, question

[Francois Gouget]

On Fri, 3 Jan 2003, Sylvain Petreolle wrote:

> Looking at http://fgouget.free.fr/wine/tests-en.shtml,
> I downloaded & ran the precompiled tests.
> Are them updated to ensure results in sync with the current CVS ?

I have just uploaded a new version of the test binaries to the above
location. Before that version the binaries contained lots of patches
that were not yet in CVS because I compiled them from my standard Wine
tree. Now I am using a separate Wine tree so the binaries are compiled
from a clean CVS tree. Also, now the page will tell you when the
winetests.zip file has last been updated. Currrently it reads:

winetests.zip -- Sunday, 05-Jan-2003 04:32:51 MET -- 1.1M

So the tests should match Wine's CVS as of 2003/01/05, and the file is
1.1MB in size.


> Is there a way to compile tests under Cygwin ? Looking into doc doesnt
> say that.

I'm not sure about Cygwin but it's theoretically possible to
cross-compile them on Linux to generated Windows binaries. However there
were issues with that last time I tried:
 * the binaries failed to work on Win9x
 * just tried now and I get:

i586-mingw32msvc-gcc dsound.cross.o testlist.cross.o -o
dsound_crosstest.exe -L../../../dlls  -ldsound -luser32 -lkernel32 -lm
../../../dlls/libmsvcrt.a(ds00425.o)(.text+0x0): multiple definition of
`atexit'
/usr/lib/gcc-lib/i586-mingw32msvc/3.2/../../../../i586-mingw32msvc/lib/crt2.o(.text+0x230):/home/ron/devel/debian/mingw32-runtime/mingw32-runtime-2.2/build_dir/src/mingw-runtime-2.2/crt1.c:241:
first defined here
../../../dlls/libmsvcrt.a(ds00305.o)(.text+0x0): multiple definition of
`_onexit'
/usr/lib/gcc-lib/i586-mingw32msvc/3.2/../../../../i586-mingw32msvc/lib/crt2.o(.text+0x250):/home/ron/devel/debian/mingw32-runtime/mingw32-runtime-2.2/build_dir/src/mingw-runtime-2.2/crt1.c:249:
first defined here


> What data should be reported when seeing "invalid test" in current
> results on the web page (running XP here) ?

Good question. Thanks for asking<g>. I added the following to the page:

> These tests are conformance tests which means their main function is
> to describe the behavior of Windows APIs, and then to allow us to
> verify that Wine behaves like the test says. So obviously if a test
> fails on Windows its description of the Windows API behavior is
> incorrect and is useless as far as testing the correctness of Wine.
> Thus in the 'Wine' column I am marking any test that fails on a
> Windows platform as 'incorrect'. It must be fixed first. Then we can
> consider the results of running the test on Wine.

So basically what needs to be done for 'incorrect tests' (previously
'invalid tests') is to fix the test so that it works on Windows and send
the patch to wine-patches. Then the tests can be run again on the
various Windows platforms and if it checks out on all of them its result
under Wine can be considered.

-- 
Francois Gouget         fgouget at free.fr        http://fgouget.free.fr/
                Linux: It is now safe to turn on your computer.