Francois Gouget : testbot: Cache the log error lists in .errors files.

[Alexandre Julliard]

Module: tools
Branch: master
Commit: 99dbbf84b0191d0827a71e9c7a32a9e40e3ce1d3
URL:    https://source.winehq.org/git/tools.git/?a=commit;h=99dbbf84b0191d0827a71e9c7a32a9e40e3ce1d3

Author: Francois Gouget <fgouget at codeweavers.com>
Date:   Sun Feb  9 18:43:56 2020 +0100

testbot: Cache the log error lists in .errors files.

Prior to this patch the log files needed to be parsed every time the
error list is needed. Parsing the reference report and diffing its
errors also needs to be repeated every time the list of new errors
is needed. Both of these have a significant impact on the JobDetails
page load time.
So this patch lets the task scripts (WineRun*) extract the log file
errors when they complete, detect the new errors, and saves the result
in a .errors file stored next to the original log file. All that's
needed then is to read the small .errors file to get the full errors
list along with their status (old or new) which speeds up WineSendLog
and JobDetails.
However the task scripts cannot generate the errors cache file for the
testbot.log file because that file is complete only once the task
scripts exits. So the errors cache file for it is generated by the
WineSendLog script once the job completes.
Furthermore the .err files were used to store the validation errors
detected by the ParseWineTestReport() function. These are now stored
as the '.Extra' error group in the .errors files, making the .err files
redundant.
Then instead of showing the content of the xxx and xxx.err file in
succession, the scripts either just iterate over all the error groups,
or present the full content of the log first and then show the 'extra'
error groups, that is those that are not tied to a specific log line
number.

Note: This requires running UpdateTaskLogs to create the new .errors
files and to delete the obsolete .err files.

Wine-Bug: https://bugs.winehq.org/show_bug.cgi?id=48035
Signed-off-by: Francois Gouget <fgouget at codeweavers.com>
Signed-off-by: Alexandre Julliard <julliard at winehq.org>

---

 testbot/bin/Janitor.pl                |   2 +-
 testbot/bin/UpdateTaskLogs            | 136 ++++++++++-------
 testbot/bin/WineRunBuild.pl           |   2 +
 testbot/bin/WineRunReconfig.pl        |   2 +
 testbot/bin/WineRunTask.pl            |  17 ++-
 testbot/bin/WineRunWineTest.pl        |  16 +-
 testbot/bin/WineSendLog.pl            | 111 +++++++++-----
 testbot/lib/WineTestBot/LogUtils.pm   | 275 +++++++++++++++++++++++++++++-----
 testbot/lib/WineTestBot/StepsTasks.pm |   7 -
 testbot/lib/WineTestBot/Tasks.pm      |   4 +
 testbot/web/JobDetails.pl             |  91 +++++------
 11 files changed, 445 insertions(+), 218 deletions(-)

Diff:   https://source.winehq.org/git/tools.git/?a=commitdiff;h=99dbbf84b0191d0827a71e9c7a32a9e40e3ce1d3