Perl/Wine extension for perusal
Eric.Pouech at wanadoo.fr
Wed Feb 21 15:36:48 CST 2001
> I think it would be great if we could start to define
> (and build) a test harness. I think that there are a lot of people
> who would help write test scripts, who might otherwise be unable
> to help with Wine development. The more the merrier, I always say...
> What did you have in mind? Our thinking, beyond building the
> Perl based winetest, hasn't gone very far. I think we've imagined
> the following:
> make test Performs an automated, non interactive regression
> test and returns true or false.
well, I started thinking on that... there could several uses of a test
harness for Wine. We could either define this is a good result or not.
But this is a bit deeper than that. For examples, some cases should work
in Win98 emulation but fail for Win2k...
So, I had rather in mind to have a two passes approach
pass 1 : let the scripts run and output the results
pass 2 : analyze the results. This could be either comparing with a
manually edited reference case, or reference cases could be generated from
running the program under Win <put any version here>.
So, this imposes to have the test programs also (compile) and run under
windows... This could even allow (from source code for example) to have
3 test caes :
1/ compiled and running under windows
2/ compiled under windows, but run with wine
3/ compiled as a winelib app
normally, there shouldn't be too many differences between 2 and 3... but,
we all know what "normally" means...
Of course, we don't need to start with all that
in order to help the pass 2 (and even produce all possible reports), I
had in mind (but didn't give a good look at it) to let the test scripts
produce an XML output. This would allow the test harness to spit out any
information (like used DLLs native/builtin, version of those, version of
windows (emulated by wine or native...), plus the output by itself
analysis and report should be (partially) driven by some XSL/XSLT tools
this may be a bit overkill to start but could be a reasonable target
on the test cases basis, I wrote a few of them (as any of us has done more
or less), basically tackling the multimedia DLLs
however, it's also a good basis to reverse engineer at the API level (or
better understand some un^^ poorly documented features of the APIs)
for developpers, the make test should be made more fine grain (IMO). Like
(not necessarly thru make) test a DLL (if some modifications have been
made to it) or even an entry point... (or a set of entry points).
It's the way I started splitting the test cases (DLL / set of features...).
This has drawbacks. For example, it's rather hard to test, say, Unicode
support accross all DLLs
> make interactive-test Runs through interactive tests that require
> user interaction with the system; user
> confirms or denies their results.
> Both of these test runs could also be done on a Windows box, so that
> an exact 1:1 comparison could be made.
> Of course, the hard part if figuring out how to do the interactive-test.
> We've thought about trying to automate it (screen shots, simulating
> mouse movements, all that good stuff), but I don't think we have an
> obvious or brilliant idea. We looked into tools for that, but found
> no good free ones; there is also a suspicion that an X based
> test tool won't be enough for Wine.
no many thoughts on this one either ... did you try the windows messages
journaling API ? IIRC, some early MS test programs did use it.
Eric Pouech (http://perso.wanadoo.fr/eric.pouech/)
"The future will be better tomorrow", Vice President Dan Quayle
More information about the wine-devel