Generating stats on regression bugs in bugzilla

[Austin English]

On Fri, Apr 24, 2009 at 8:04 PM, Ken Sharp <kennybobs at o2.co.uk> wrote:
>
>
> Austin English wrote:
>>
>> On Fri, Apr 24, 2009 at 6:40 PM, Scott Ritchie <scott at open-vote.org>
>> wrote:
>>>
>>> I had a theory that we might be getting better at preventing regressions
>>> now
>>> than a year or so ago due to the expansion of things like AppDB and the
>>> test
>>> suite.  It'd be nice to have some sort of data though.
>>>
>>> Would it be reasonable to comb through the bugzilla database for all the
>>> bugs tagged regression and analyze them?  I'm interested in:
>>>
>>> 1) quantity and frequency of bugs tagged regression that were filed at
>>> various dates since the release of 1.0.  I'm not sure whether it's better
>>> to
>>> use the version number field in bugzilla as a proxy for "first version
>>> with
>>> regression" or to just guess that from the date.
>>> 2) how long regression bugs took to be fixed
>>>
>>> Thanks,
>>> Scott Ritchie
>>>
>>>
>>>
>>
>> It'd be better to use the SHA1SUM of the bad patch, which will give
>> you the day the regression was added.
>>
>> If the fix has the SHA1SUM listed in there as well, it's even easier for
>> you.
>>
>
>
> Some bugs contain the wrong regression before mentioning the correct one.
>  It would be necessary to look for the last SHA1SUM mentioned in a bug to
> get the correct one.
>
> But if the last comment is the fix... you see what I'm saying.
>

You'll also want to account for when a regression is reported/fixed.
E.g., some regressions were introduced in, say, 1.1.12, but not
reported until 1.1.15.

And if the fix is put in immediately, but not marked fixed until
1.1.18, how would we know? Without the SHA1SUM, it's hard to say.

Not trying to discourage you, as it would be _very_ interesting to
see, but I don't think it'll be very easy at all.

-- 
-Austin