Babystep: Testing framework

[Alexandre Julliard]

Jeremy White <jwhite at codeweavers.com> writes:

> Um...the way I constructed it, there is no additional effort required
> in the Makefile whatsoever.  Now, if you get rid of my .test file,
> yes, then, doing this all in the Makefile is a pain in the neck.

There is no effort unless you want to get the dependencies right; then
you are in for a lot of trouble. With your proposal the only
dependency is on the .test file, which will most likely never change
over the lifetime of a test. OTOH the Perl/C source will change
frequently, but make test will never notice it and won't re-run the
test. That defeats the whole purpose of using make.

> If the goal is to make it easy for a newcomer to add a test, then requiring
> them to hack the Makefile is going to be a barrier.  Asking them to
> copy and cut/paste a 'samples/foo.test' file will not be.

Sorry if I wasn't clear. My comment was about your example of adding a
new category of tests, like support for .so tests. That's when you
need some serious hacking. If all you do is add a Perl or C test, you
only have to add a line in the Makefile, just like with your solution.

> I think either choice is valid, but let me lay down the gauntlet:
> replace my 'winetest/samples' directory with a Make based equivalent
> that allows for the same sorts of tests, and makes it similarly obvious
> to a newcomer how to add a new test.  You are better with
> Make than I am; you may persuade me readily with a patch.

It would look pretty much the same. Just replace this:

TESTS = sample1.test sample2.test sample3.test

in your makefile by:

PLTESTS = sample2.pl
CTESTS = sample3.c

and delete the .test files (there is no SHTESTS for shell tests, but I
think that's a feature if we want portability).

Creating a new test involves simply adding its name on the right line;
no .test to write, no .spec file, no new Makefile.

(if you'd like to try it I have committed the rules for PLTESTS so
this should work now; CTESTS will still need a bit more work)

-- 
Alexandre Julliard
julliard at winehq.com