We *really* need a development model change !
Jeremy White
jwhite at codeweavers.com
Wed Jan 2 13:12:18 CST 2002
>
>
>* its exit code
> * text output on either or both of stdout and stderr, both of which are
>normally redirected to a file called 'xxx.out'.
>
> A test succeeds if:
> * its exit code is 0
> * and its output, 'xxx.out' matches the reference output according to
>the rules described later.
>
I think that it would be handy to use stderr for status/diagnostics, and
only use
stdout for reference checking. Chatty people like me get their statii,
but the tests remain clean. Perhaps a WINETEST_DEBUG env variable
would be a good addition as well (and since Alexandre controls the commits,
my guess is it'll default to off <g>).
>Otherwise this file is either called:
> * 'xxx.ref'
> * or 'xxx.win95' or 'xxx.win98' ... if the output depends on the
>Windows version being emulated. The winever-specific file takes
>precedence over the '.ref' file, and the '.ref' file, which should
>exist, serves as a fallback.
>
I hope that the cases where this would be needed would be few enough
that we wouldn't need to build in a general purpose exception;
we can just have a mytest.win95.test and a mytest.win98.test for
the cases where it's needed.
[snipping chunks largely agreed with]
>
>Test coverage
>-------------
>
> Each test should contain a section that looks something like:
>
I started trying to tweak Alexandre's patch to create some sample
tests, and I learned the following:
1. Cygwin installation has *dramatically* improved.
Getting a full working toolchain is no longer
a big pain in the rear end, it's actually pretty easy.
2. Having '.test' files be implicitly Perl scripts is too
limiting, IMHO. I hate to add another format, but
I've been toying with YAFF (yet another file format)
so that a '.test' file describes a test, as follows:
# Comment lines
script=name_of_my_perl_test_script
invoke=name_of_c_test_or_shell_script
status=-eq 0
pattern=.*OK
compare=name_of_my_ref_file
where one of script or invoke is required, and
status, if given, is a test expression that $0 is compared to,
pattern is a regexp applied to stdout, if given,
and compare is the name of a .ref.out file to compare
the output with.
The default would be just a case of 'status=-eq 0'.
The nice thing about this approach is that you can
handle the multiple version testing just by creating
a new reference file and a new .test file.
>In Windows:
>
> Hmmm, not sure how that is done. Run 'winetest.exe'?
>
IMO, we should have a script that creates a 'winetest.zip',
with simple batch files to make it easy to run a single
test.
But, (as a new convert to the ease of use of Cygwin), I think
we can just stick with 'make tests' for the full deal
on Windows.
Jer
More information about the wine-devel
mailing list