Sufficient 1.2 release criterion: passing all tests on all platforms?

[Dan Kegel]

On Sun, Mar 8, 2009 at 9:27 PM, Austin English <austinenglish at gmail.com> wrote:
>> Even without any new features, it seems to me that
>> passing all tests on all platforms might all on its own
>> merit a new stable release.
>
> By 'all platforms', do you mean all Windows versions, or Linux/OS X/BSD/Solaris?

I meant Windows.

But now that you ask, we do have a lot of platforms to consider.  We
simply can't provide the same level of support for them all.
The gcc project defines three tiers of support.  If we did that, it
might look like this:
We would define tiers for Windows conformance test validation, CPUs,
and host operating systems, and maybe graphics cards.
1st tier: we run tests regularly, and all tests must pass for release.
2nd tier: we might run tests occasionally or regularly, but we will
tolerate some failures.
3rd tier: we won't test ourselves, and will tolerate failures, but
will accept bugfixes from advocates.

Here's one possible set of definitions:

For Windows conformance test validation:
1st tier:  Win XP 32 bit, Win 2003 32 bit, Win Vista 32 and 64 bit,
Win 2008 32 bit
2nd tier: Win XP 16 bit, Win 95, Win 98, Win ME, Win 7 32 and 64 bit
3rd tier: Win 3.1, DOS

For CPUs:
1st tier: whatever our developers use, but mostly < 2 year old Intel
and AMD chips, running apps in all three modes, 16, 32, and 64 bit (as
supported by hw)
2nd tier: none
3rd tier: power pc, sparc, other less-common pentium-compatible chips

For host OS:
1st tier: Linux
2nd tier: Mac OS X
3rd tier: Solaris, FreeBSD

For graphics cards:
1st tier: Nvidia 8400 or higher
2nd tier: < 4 year old Nvidia, < 2 year old ATI, < 2 year old Intel
3rd tier: older nvidia

I'm sure that will need adjusting, but it's a good starting point for
discussion.
- Dan