Upcoming TestBot patches

[Francois Gouget]

On Wed, 18 Oct 2017, Francois Gouget wrote:
> 
> For those that are interested I'm trying to regularly push my set of 
> 'might be ready to send' patches on GitHub:
> 
> https://github.com/fgouget/wine-tools/tree/future

I have updated the GitHub repository with a new revision of the patches.

https://github.com/fgouget/wine-tools/commits/future-20171208

I'm sending the first few minor patches now and if all goes well I will 
start sending the first set of major patches next monday (so I have 
Alexandre on hand if it breaks the TestBot).

This branch contains the following patchsets:

* First is a group of 10 patches related to tracking the TestBot 
  activity. This adds a new Activity page where one can see exactly what 
  is going on at a given time and in the past.
  This probably will not change much for you, Wine developers, but 
  it's really useful to me to see what the TestBot is doing, identify 
  places where it behaves in some unexpected way and know which part of 
  the log to scrutinize, and to check that the job scheduler works as 
  expected.

  testbot: Keep a record of the status of the VMs.
  testbot: Save the status of all VMs on startup.
  testbot/web: Use the TestBot records to show its activity.
  testbot: Add a history Record when the Engine starts or stops.
  testbot/web: Show TestBot start/stop events on the Activity page.
  testbot/web: Show the TestBot's mispredictions on the Activity page.
  testbot: Add history records for mutable task results.
  testbot: Add a history record when a task is canceled.
  testbot: Add a history record when a child process died.
  testbot/web: Show failed Tasks on the Activity page.


* Second is a single patch that leverages the previous infrastructure to 
  provide TestBot statistics such as average revert times, Wine update 
  times, number of jobs and tasks per hour, etc. It will be interesting 
  to see how these evolve with some of the next changes.

  testbot/web: Add a statistics page.


* Then comes a rewrite of the job scheduler. The goal was to 
  add support for multiple test configurations per VM (see below).
  In the process I found flaws in the existing scheduler and also that 
  it lacked the flexibility to be extended in the way I needed. So this 
  patch is the new scheduler without the multi-test-configuration 
  support.
  As the FIXME implies there are still a couple of points I want to 
  review but I've been using it for the past few weeks and it seems to 
  be working fine.

  FIXME testbot: Make the job scheduler more extensible.


* Finally is a patchset extending the job scheduler to support multiple 
  test configurations per VM. This patch will impact the life of Wine 
  developers: in particular it should make it possible to duplicate all 
  the test configurations I run on my home VMs on the TestBot.
  The first application will be adding tests for more locales using 
  the TestBot's Ultimate windows editions: in addition to Hebrew 
  and Japanese we will be able to test Arabic, Korean, French, etc.
  Eventually it should be possible to duplicate the configuration of any 
  of my home VMs to the TestBot and thus make those easier to debug.
  That will increase the load on the TestBot and this is where the new 
  statistics page will be interesting: for instance it should show if 
  the load is spread well across the VM hosts.
  I have also been using this patchset over the past weeks without 
  issue but I feel it still needs some more testing in corner cases 
  hence the FIXME.

  testbot: Check the current snapshot in LibvirtTool checkidle.
  testbot: Check the current snapshot in LibvirtTool monitor.
  testbot: Replace poweroff with checkoff in the Engine's Cleanup().
  testbot: Avoid race conditions between LibvirtTool monitor and other tools.
  FIXME testbot: Allow multiple VMs to share access to a hypervisor domain.


* Currently the TestBot overrides the umask in various places. Even 
  though all the TestBot processes are single-threaded that does not 
  seem right to me; even more so since we set the umask in the 
  configuration file.
  It seems to me it would be cleaner to rely on the umask set in the 
  configuration file so I may send this patch at some point.
  Does anyone care to comment?

  testbot: Individual modules and scripts should not override the umask.


-- 
Francois Gouget <fgouget at codeweavers.com>