We *really* need a development model change !

Andriy Palamarchuk apa3a at yahoo.com
Tue Jan 8 08:26:20 CST 2002


--- Alexandre Julliard <julliard at winehq.com> wrote:
> Andriy Palamarchuk <apa3a at yahoo.com> writes:
>
> I frankly don't see the need of a complex
> test harness to
> launch tests and print reports. Maybe when we have
> 1000 tests we will
> want some of that complexity, but for now it's only
> a waste of time.

Almost all complexity you are talking about is already
implemented for us. Usage of the framework is very
simple, do not require from test writers anything.
They are only required to correctly use Test::Simple
(or Test::More). They don't need to remember about
Test::Harness.

There are reasons to use Test::Harness:
1) control of TODO tests - I really want to use this
feature.
2) control of SKIP tests - very useful for
Wine-specific tests, choosing behavior, depending on
Windows versions, etc. I need this feature too.
3) we already need to manage the test output. I'd
estimate number of checks for my existing
SystemParametersInfo unit test as:
25 (number of implemented actions) * 10 (minimal
number of checks for each action) = 250 - 350 tests
We'll definitely have huge number of tests. Why not
pick up scaleable approach from very beginning?

> Let's not debate licenses again. Anything that is
> included in the Wine
> distribution has to be under the Wine license. If
> you want a GPLed
> test suite you'll have to distribute it separately
> from Wine; but I
> think that would be a mistake.

IMHO you can distribute them together if these are 2
separate applications, right? I don't have big
preferences about the license. Wine license will work
fine.


We spent a lot of our valuable time and resources on
the issue. Lets finalize the decisions and start to
work.

Suggest decisions from the discussion:
1) unit tests are very important for the project
2) mixed C/Perl environment will be used for the tests
development. Choosing the tool is a matter of personal
preferences.

Comment: Alexandre, basing on the discussion above I
think we both came to this. Do I understand you
correctly?
TODO:
  - create support in the build process

3) Test::Harness will be used to compile report for
test batches

Comment - TODO:
  - fix problems in using Test::Harness (and some
other modules, BTW) with winetest Perl framework
  - create simple framework for C to print report in
format, suitable for Test::Harness. (I already started
to work on this.)
  - change Test::Harness to run not only Perl scripts,
but also other types of executables (can do this)

4) The unit test will be a separate application

Comment:
Alexandre, we explicitely did not agree on this
decision yet. You preferred to have unit tests
spreaded over the Wine directory tree. The main
argument for this was possibility of running subsets
of test. I suggest to change Test::Harness (or create
new module, based on it) to use tree-like tests
organization. If you need to test module wine/dll/foo,
you request to run tests hierarchy "wine/dll/foo".

TODO:
  - customize Test::Harness to use tree-like test
suits structure (I can do this).

5) Existing Wine license will be used for the Unit
test application

Please, let me know if I missed something or your
point of view is not reflected.

Looking forward for your comments.
Andriy Palamarchuk


__________________________________________________
Do You Yahoo!?
Send FREE video emails in Yahoo! Mail!
http://promo.yahoo.com/videomail/




More information about the wine-devel mailing list