[PATCH 1/5] cmd/tests: add more tests for command line parsing

[Frédéric Delanoy]

2011/8/30 Martin Wilck <mwilck at arcor.de>:
> Hi Frédéric,
>
>> Some of those tests are already present in the test_builtins.cmd*
>> (e.g. the spacing tests)
>> You should probably integrated your tests in there: that makes it
>> easier to spot regressions and helps prevent tests duplication
>
> I can certainly do that if you prefer (and I will), but I don't agree
> that having all tests in a single big file is an advantage. Splitting
> the unit tests over multiple files would make it much easier to relate
> output lines to script lines and do focused work on certain areas.

Cmd test suite would probably eventually benefit from this on the long
term, but it's still very incomplete in a number of areas (thanks for
helping improve it BTW).
IMHO we should probably split it only when it's reasonably complete,
where we'd have a broader view.
(it also makes viewing regression easier for the devs since the parser
is rather fragile, and everything is a bit intertwined)

Also, test_parsing name is probably a bit too generic (not that
test_builtins isn't ;) )

But you can still submit it in the meantime, maybe AJ will accept it.
Just make sure there's no duplication

> For
> anyone who doesn't work on it regularly, test_builtins.cmd is a nightmare.

Just look at cmd parser code if you really want to be afraid ;)

I'm a bit responsible for this, but the current parser has numerous
shortcomings/bugs that sometimes force to use ugly workarounds.
Also, when you have to make tests work on NT4->2k8 *and* wine at *all*
times, you sometimes have to make your hands dirty.
(e.g. a restriction is that you need the same # of lines of the
processed cmd file, and the expected file, on all windowses and wine.
There's no support for skip() or win_skip() either like is available
in the other regular C unit tests.
I guess that eating one's own dogfood...)

Just ask if you don't understand the @keywords at . Doc is a bit lacking;
I might add a quick wiki help page if desired.
Basically the most important ones are @todo_wine@ and @or_broken@
(respectively to indicate the test fails on wine, or have alternate
results in other windowses)

Wine testbot can also help (https://testbot.winehq.org/index.pl)

>> Also, see http://www.winehq.org/pipermail/wine-devel/2011-August/091817.html
>
> I'm not sure why you're telling me that.
Well the mentioned link was about splitting tests ("Cmd tests timeout
and splitting tests")
The discussion thread was about splitting the test file into distinct
subfiles, such as 1 per builtin or so do a timeout on some linux
machines.
(Cmd's testsuite run in a matter of seconds on windowses, but much
longer on distinct linux/bsd test machines)

> My patches aren't about
> performance but correctness. They won't affect performance negatively,
> either.
Of course not, since it's only in the test proc

> Regards,
> Martin

Nice job by the way.

Frédéric

> PS: any idea why my patches didn't make it to wine-patches?

Did you correctly register via
http://www.winehq.org/mailman/listinfo/wine-patches?