TestBot status / VMs todo list

[Francois Gouget]

Part 3: It's full of VM (or a bit less so).

TL;DR;
- The TestBot is operating on a reduced set of VMs.
- A lot of VMs need to be restored or updated.


* debiant
  - This new Debian Testing VM is meant to solve some Mesa issues that
    plague the debian10 VM.
  - It has somewhat better results for win32 and somewhat worse for
    wow64.
  -> Let me know if it needs tweaking.
  -> Should it replace debian10 as the main Wine testing platform?
  -> Should debian10 be kept around?

* wxppro_2scr
  - Windows was blue-screening shortly after reverting to the live 
    dual-screen snapshot (reverting to the single-screen snapshot still 
    works). This looks like a QEmu issue and recreating the snapshot did 
    not help.
  - So I changed that configuration to use a powered off snapshot. This 
    means it has to go through the Windows boot every time it is used 
    which should reliable enough these days. This is slower but it is 
    not one of the main test VM configurations so it is not used that 
    often.

* wvista was not producing usable WineTest results anymore
  - It was also not restored when vm1's disks got replaced.
  - Restore from backup.
  - Set up autologin and TestAgentd autostart.
  - Run Windows Update.
  - Make sure all troublemakers like Windows Update, Defender, etc are
    disabled.
  - Check that WineTest runs fine and investigate if not.

* wvistau64 was not producing usable WineTest results anymore
  - It was also not restored when vm1's disks got replaced.
  - Vista's support for Unicode scripts is no good so there it no
    point testing languages such as Arabic or Hebrew on it.
  - So restore wvistau64 from a backup made before the languages were
    installed. This should reduce the qcow2 size and maybe reduce
    revert times.
  - Set up autologin and TestAgentd autostart.
  - Run Windows Update.
  - Make sure all troublemakers like Windows Update, Defender, etc are
    disabled.
  - Reinstall a few simple languages as those should still be testable:
    en-CA, fr-FR, fr-CA, pt-BR, ru-RU
    -> Send me your wish list.
  - Check that WineTest runs fine and investigate if not.

* w7pro64 tended to lose network access
  - Its role got dropped to winetest, i.e. wine-devel patches are not
    run on it.
  - Symptom:
    network read timed out (wait2/connect:AgentVersion.h:0/9)
    The test VM has crashed, rebooted or lost connectivity (or the
    TestAgent server died)
  - Figure out why or just restore from backup.
  - Set up autologin and TestAgentd autostart.
  - Run Windows Update.
  - Add a dual-screen configuration for Zhiyi.
  - Check that WineTest runs fine and investigate if not.

* w7u tended to lose network access
  - It caused false positives and thus got temporarily retired.
  - Symptom:
    unable to open 'exe32.report' for reading: No such file or directory
    The test VM has crashed, rebooted or lost connectivity (or the
    TestAgent server died)
  - Figure out why or just restore from backup.
  - TestAgentd is too old (1.6), preventing capturing cmd errors.
  - Set up autologin and TestAgentd autostart.
  - Run Windows Update.
  - Install some languages, at the very least all the vistau64 ones:
    en-CA, fr-FR, fr-CA, pt-BR, ru-RU
  - If the scripts support allows it add more complex languages.
  - Check that WineTest runs fine and investigate if not.

* w8 crashes while running WineTest
  - Figure out why or just restore from backup.
  - Set up autologin and TestAgentd autostart.
  - Run Windows Update.
  - Check that WineTest runs fine and investigate if not.

* w2008s64 needs an update for msvcr80
  - It lacks the latest version of msvcr80.
  - Restore from backup.
  - Set up autologin and TestAgentd autostart.
  - Run Windows Update -> upgrades msvcr80.
  - Figure out what to do for the missing mfplat components:
    https://bugs.winehq.org/show_bug.cgi?id=48430
    This could also be fixed by taking care of bug 48208.
  - Check that WineTest runs fine and investigate if not.

* w1064
  - The 1607+ updates were installed too late so that by the time they
    were added failures had accumulated. So while 1909 is likely to end
    up above the 70 test unit failures limit it's better to install it
    as soon as possible.
  - Restore from the pre-language backup.
  - Install 1909, preferably from ISO (not available yet) as this limits
    the size increase of the qcow2 file.
  - Run Windows Update.
  - Do not reinstall the languages (see below).
  - Check that WineTest runs fine and investigate if not.

* w1064 bis
  - Get a new Windows 10 license and create a new Windows 10 VM.
  - The reason for this is that we are starting to have too many test
    configurations for w1064: 5 versions + dual-screen + 8+ languages.
    Because they are all on the same VM we have to run them all on the
    same host one at a time, leading to long job run times. Splitting
    them accross two VMs will allow us to double the test rate.
  - Make sure the new VM is on the latest Windows 10 version. Don't keep
    snapshots of the other versions.
  - Disable all the troublemakers such as Windows Defender, automatic
    defragmentation, etc. Then take a 'base' snapshot.
  - Then set up the new VM with the languages that got removed from the
    above other w1064 VM and some more:
    https://bugs.winehq.org/show_bug.cgi?id=31785
    ar-EG, en-CA, fr-FR, fr-CA, he-IL, ja-JP, ko-KO, pt-BR, ru-RU, zh-CN
  - Also add Thai, Tibetan, a Deseret script language, a Sinhala script
    language and Canadian Aboriginal syllabic language.
  - This still makes a lot of configurations for a single VM. So some
    locales may be offloaded to the vistau64 or w7u VMs and kept with a
    winetest or extra role on the w1064b VM so one can always check the
    behavior on the latest Windows version.
  - Check that WineTest runs fine and investigate if not.
  -> The TestBot still seems to keep up with the current set of VMs so
     the priority is lower than restoring VMs that have been taken out.
     But as VMs are restored this may become more urgent.

* build
  - The build VM has become somewhat redundant: the Wine VMs are totally
    capable of producing the Windows binaries. Plus most jobs will
    require running the tests on Wine which means the Windows binaries
    will be available 'for free'.
    But the Wine VM is more likely to have build failures after the
    daily Wine commits or to timeout when building or running the tests.
    So it's probably worth keeping a VM dedicated to producing the
    Windows binaries.
  - However instead of being a specific VM it could be a clone of the
    'main' Wine VM with just a different IP address and role. If so
    virt-sysprep may allow automating the conversion of a Wine VM to a
    build VM and vice-versa.
  - If not cloned from the current Wine test VM, upgrade it to at least
    run the same Debian version. Then also reconfigure it to autostart
    TestAgentd on boot.
  -> But for now the build VM still seems to work ok so this is low
     priority.

* cw-gtx560 + Wine
  - Sometimes has way too many failures in Wine. This seems to mostly
    happen with the wow32 and wow64 builds these days (yet another
    nouveau issue?). These need to be investigated and fixed. If the
    fix is not system-specific it should be replicated to the other
    systems.
    https://test.winehq.org/data/errors.html
  -> Not really part of the TestBot so priority is low.

* cw-rx460 + Wine
  - The Wine test results were so bad it no longer is running them.
    Again this needs to be investigated, etc.
    https://bugs.winehq.org/show_bug.cgi?id=48092
    https://bugs.winehq.org/show_bug.cgi?id=48093
  -> Not really part of the TestBot so priority is low.


-- 
Francois Gouget <fgouget at codeweavers.com>