Perl/Wine extension for perusal

Alexandre Julliard julliard at winehq.com
Wed Feb 21 17:23:26 CST 2001


Eric Pouech <Eric.Pouech at wanadoo.fr> writes:

> So, I had rather in mind to have a two passes approach
> pass 1 : let the scripts run and output the results
> pass 2 : analyze the results. This could be either comparing with a 
> manually edited reference case, or reference cases could be generated from
> running the program under Win <put any version here>.

Yes, that's the idea; the test script will output a bunch of results
which can then be compared against a reference run generated under
Windows. We probably also want a standard config file (or a few of
them to handle version differences) to ensure the test environment is
the same for everybody.

> So, this imposes to have the test programs also (compile) and run under
> windows... This could even allow (from source code for example) to have
> 3 test caes :
> 1/ compiled and running under windows
> 2/ compiled under windows, but run with wine
> 3/ compiled as a winelib app

The idea of using an interpreter like Perl is precisely that you don't
need to compile anything to run tests. I think this is important
because not everybody has a Windows compiler. It also allows using the
exact same test script under Windows and Wine, so that you don't have
to worry whether your Windows binary exactly matches your Winelib
binary.

> in order to help the pass 2 (and even produce all possible reports), I
> had in mind (but didn't give a good look at it) to let the test scripts
> produce an XML output. This would allow the test harness to spit out any
> information (like used DLLs native/builtin, version of those, version of
> windows (emulated by wine or native...), plus the output by itself
> analysis and report should be (partially) driven by some XSL/XSLT tools

I don't think we need anything fancy like that. The output should be
simple ASCII that can be automatically compared with diff, and the
test is considered a failure if diff finds any difference against the
reference output. Everything should be automated as much as possible.

> for developpers, the make test should be made more fine grain (IMO). Like
> (not necessarly thru make) test a DLL (if some modifications have been
> made to it) or even an entry point... (or a set of entry points).

Yes, there should be one set of test scenarios for each dll, and each
scenario should test one entry point (or a few related ones); this way
you can either run a single test, or do a make test in a dll directory
to run the tests for this dll, or make test at the top-level which
will simply iterate through all the dlls. The tests should ideally run
fast enough that you can do a make test in the dll dir everytime you
change something.

> It's the way I started splitting the test cases (DLL / set of features...).
> This has drawbacks. For example, it's rather hard to test, say, Unicode
> support accross all DLLs

But you should never need to do that. If you change something in a
core Unicode function in kernel, this will be tested by the kernel
test scenarios. If the test scenarios don't find a problem but tests
for higher level dlls fail, then your kernel test scenarios are buggy
since they didn't spot the change.

-- 
Alexandre Julliard
julliard at winehq.com




More information about the wine-devel mailing list