We *really* need a development model change !

[Alexandre Julliard]

Andriy Palamarchuk <apa3a at yahoo.com> writes:

> Always succeed *under Windows*. Do you really, really,
> really think all the tests will succeed under Wine
> from day 1 and we will be able to maintain them
> failure-free?

Absolutely. There's a very simple way of enforcing that: I'm not
comitting anything that causes make test to fail.

> The value of unit tests is exactly in failures! The
> more failures of unit tests we have - the better test
> developers do their work.

Well, I guess it's a philosophical point, but for me the value of the
tests is for regression testing. If you allow the tests to fail you'll
pretty soon have 90% of the tests fail somewhere, and this is
completely useless except maybe as a list of things we still have to
do. While if the tests usually succeed, as soon as something fails you
know there's a regression and you have to fix it.

> Wrong. In make files you can check only some
> compilation dependencies. You can't do anything about
> logical dependencies. I'm for running the whole suite
> of the tests by developers before submitting *any*
> patch and from time to time centrally.

What you can do with my make test patch is run make test -k first, let
it go through everything, and then run make test again and it will
only run the tests that failed the first time, or that have been
modified. This is a major gain. It could certainly be done some other
way too without using make, but AFAICS your test harness would force
to run all tests all the time (or to disable them manually). This is
not acceptable once you have a large number of tests.

> This is why I want very much to keep it in separate
> directory. Even the tests structure, probably, should
> correspond more to logical Win32 API structure, not to
> Wine directory tree.

Well, the Wine directory tree corresponds (mostly) to the Win32 API
structure. But this is again a philosophical point.  I don't view the
test suite as an API specification, but as a regression test suite for
Wine.

> The Win32 unit test suite has huge value by itself.
> All Win32 projects can contribute to it on the same
> rights. The ideal solution, probably, whould be to
> move completely separate project. We can do this later
> if the need arises.

This is like the documentation: yes it would be great to have good
documentation and good specifications of the Win32 API. It's not going
to happen, because there's no motivation behind it except "would be
nice to have".

OTOH a Wine test suite can happen (I hope) because this is something
Wine developers need when they write code, so there is at least some
motivation for them to write tests. We can do it in a way that the
Wine test suite is useable for more than testing Wine, but this has to
be the main focus.

-- 
Alexandre Julliard
julliard at winehq.com