RFC: TestBot priorities

[Francois Gouget]

On Fri, 3 Apr 2020, Gabriel Ivăncescu wrote:
[...]
> An alternative is to just print a summary of warnings and not actually fail
> the test because of them. Whoever is interested in the warnings can then
> easily spot them. It requires more work, though.

I think that's more or less the way to go.

1. The builds would happen without -Werror.

2. GetLogLineCategory() would identify the warnings and add them to 
   LogInfo but distinguish them from errors (replace IsNew with a 
   Category field?).

3. Compare to the reference build to distinguish new warnings from old 
   ones. This is necessary in case the compiler used by the TestBot 
   produces warnings on the unpatched Wine tree. It will require storing 
   the build logs in latest too.

4. The warnings would be saved in the errors files. The new line types 
   could be 'w' and 'W' for old and new warnings respectively.

5. This will let the WineRun*() scripts know about both warnings and 
   errors. They can then decide what to do depending on the type of 
   job and whether the warnings are new or not.
   - Old warnings would always be ignored.
   - If a build task has new warnings, instead of marking it as 
     'badbuild' it could be marked as 'completed' as usual but with a 
     non-zero number of failures. This would probably allow running the 
     tests on Windows but I think the patch would still be considered 
     bad since the job would end up with a non-zero number of failures.
   - For Wine tasks the number of warnings would be added to the 
     Failures field for the same overall result.
   - So in both cases the non-zero number of failures would only impact
     wine-devel patches.

6. The JobDetails page would show the warnings in summary mode and 
   highlight them in the full build log.


-- 
Francois Gouget <fgouget at codeweavers.com>