Sufficient 1.2 release criterion: passing all tests on all platforms?

[Austin English]

On Sun, Mar 8, 2009 at 4:02 PM, Dan Kegel <dank at kegel.com> wrote:
> But now that you ask, we do have a lot of platforms to consider.  We
> simply can't provide the same level of support for them all.
> The gcc project defines three tiers of support.  If we did that, it
> might look like this:
> We would define tiers for Windows conformance test validation, CPUs,
> and host operating systems, and maybe graphics cards.
> 1st tier: we run tests regularly, and all tests must pass for release.
> 2nd tier: we might run tests occasionally or regularly, but we will
> tolerate some failures.
> 3rd tier: we won't test ourselves, and will tolerate failures, but
> will accept bugfixes from advocates.

+1

> Here's one possible set of definitions:
>
> For Windows conformance test validation:
> 1st tier:  Win XP 32 bit, Win 2003 32 bit, Win Vista 32 and 64 bit,
> Win 2008 32 bit
> 2nd tier: Win XP 16 bit, Win 95, Win 98, Win ME, Win 7 32 and 64 bit
> 3rd tier: Win 3.1, DOS

Not sure exactly what you mean by Win XP 16-bit? The Win16 test suite on XP?

> For CPUs:
> 1st tier: whatever our developers use, but mostly < 2 year old Intel
> and AMD chips, running apps in all three modes, 16, 32, and 64 bit (as
> supported by hw)
> 2nd tier: none
> 3rd tier: power pc, sparc, other less-common pentium-compatible chips

No argument there. Perhaps move 64-bit to 2nd tier, and move it up to
1st once we've got better support for it.

> For host OS:
> 1st tier: Linux
> 2nd tier: Mac OS X
> 3rd tier: Solaris, FreeBSD

Having tested these often, I'd say OS X is more broken than FreeBSD.
I'd swap those two around to be honest. Solaris/OpenBSD/NetBSD are
tier 3 though.

-- 
-Austin