Make test drill, next steps, call for help with Winetest
Francois Gouget
fgouget at free.fr
Sat Oct 27 07:03:02 CDT 2007
On Mon, 15 Oct 2007, Hans Leidekker wrote:
[...]
> I have written attached script to facilitate automated winetest runs.
> It needs a VirtualBox virtual machine with either Windows or a Wine platform
> configured to run autorun.inf files.
> worry about winetest screwing up your installation.
[...]
That's something I've wanted to do for some time and this finally
spured me to action. So here's a script that will do essentially the
same thing but with VMware Workstation.
Like your script it downloads winetest using wget, then generates an ISO
with an autorun.inf file so the tests will start automatically in the
VM. It also uses snapshots so the state of the virtual machine is
preserved.
Typically you would use it like this:
* I have not found a way to 'insert' the CD if the cdrom device is not
already connected in the VM. So you must first ensure that's the case.
You can do so in a 'winetest' snapshot, the script will automatically
switch to that snapshot.
* Creating a winetest snapshot is probably a good idea anyway, if only
to ensure it's a relatively clean environment.
* Then run the script as follows:
winetest /path/to/virtual/machine.vmx --tag mytests --no-submit
The --no-submit option is so that the tests don't submit their
results to http://test.winehq.org/. The goal here is to avoid sending
lots of duplicate results while you're experimenting. Once you're
confident things work just as expected, remove the option.
There are other options to specify which version of winetest.exe to
grab, to set the timeout on the tests execution, etc.
Things still on the todo list:
* the script also grabs the winetest.exe signature and attempts to
verify it. But I don't know where to find the corresponding public key
(651FD487) so I don't know if this really works.
* the script gets the 'latest' winetest.exe version by default, but
when there are compilation problems I guess this may stay stuck at the
same revision for some time. So it would be nice if the script could
know which version it's getting so it could skip the tests if that
version is too old, or has already been tested.
* I wonder if this works with VMware Server? It would be nice as Server
is free. The script uses snapshots quite a bit though so it may not
work. Let me know how it goes if you try.
--
Francois Gouget <fgouget at free.fr> http://fgouget.free.fr/
Any sufficiently advanced Operating System is indistinguishable from Linux
-------------- next part --------------
#!/bin/sh
# Copyright (C) 2007 Francois Gouget
#
# This program is free software; you can redistribute it and/or
# modify it under the terms of the GNU General Public
# License as published by the Free Software Foundation; either
# version 2.1 of the License, or (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
# General Public License for more details.
#
# You should have received a copy of the GNU General Public
# License along with this library; if not, write to the Free Software
# Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301, USA
name0=`basename "$0"`
verbose=""
base_url="http://www.astro.gla.ac.uk/users/paulm/WRT/CrossBuilt"
vmware="/usr/local/opt/vmware/bin"
if [ -z "$DISPLAY" ]
then
DISPLAY=":0.0"
export DISPLAY
fi
verbose()
{
if [ -n "$verbose" ]
then
echo "$@"
fi
}
vm_is_busy()
{
vm="$1"
test -f "$vm.WRITELOCK"
return $?
}
vm_is_running()
{
# For VM matching purposes, only keep the last two elements of the path
# to avoid trouble with symbolic links
vm_match=`dirname "$opt_vm"`
vm_match=`basename "$vm_match"`/`basename "$opt_vm"`
vm_match=`echo "$vm_match" | sed -e 's/\([^a-zA-Z0-9_ /]\)/[\1]/g'`
"$vmware/vmrun" list | egrep "$vm_match" >/dev/null
return $?
}
### Main
opt_vm=""
opt_tag=""
opt_version=""
opt_check_sig=""
opt_submit=""
opt_timeout=""
opt_autorun=""
opt_snapshot=""
opt_cddevice=""
opt_usage=""
while [ $# -gt 0 ]
do
arg="$1"
shift
case "$arg" in
--tag)
if [ -n "$opt_tag" ]
then
echo "$name0:error: --tag can only be used once" >&2
opt_usage="2"
break
elif [ $# -eq 0 ]
then
echo "$name0:error: missing argument for the --tag option" >&2
opt_usage="2"
break
else
opt_tag="$1"
shift
fi
;;
--version)
if [ -n "$opt_version" ]
then
echo "$name0:error: --version can only be used once" >&2
opt_usage="2"
break
elif [ $# -eq 0 ]
then
echo "$name0:error: missing argument for the --version option" >&2
opt_usage="2"
break
else
opt_version="$1"
shift
fi
;;
--check-sig)
opt_check_sig="1"
;;
--no-check-sig)
opt_check_sig="0"
;;
--submit)
opt_submit="1"
;;
--no-submit)
opt_submit="0"
;;
--timeout)
if [ -n "$opt_timeout" ]
then
echo "$name0:error: --timeout can only be used once" >&2
opt_usage="2"
break
elif [ $# -eq 0 ]
then
echo "$name0:error: missing argument for the --timeout option" >&2
opt_usage="2"
break
elif echo "$1" | egrep '^[0-9][0-9]*$' >/dev/null
then
opt_timeout="$1"
shift
else
echo "$name0:error: the timeout value must be a decimal number" >&2
opt_usage="2"
break
fi
;;
--autorun)
opt_autorun="1"
;;
--no-autorun)
opt_autorun="0"
;;
--snapshot)
if [ -n "$opt_snapshot" ]
then
echo "$name0:error: --snapshot can only be used once" >&2
opt_usage="2"
break
elif [ $# -eq 0 ]
then
echo "$name0:error: missing argument for the --snapshot option" >&2
opt_usage="2"
break
else
opt_snapshot="$1"
shift
fi
;;
--cddevice)
if [ -n "$opt_cddevice" ]
then
echo "$name0:error: --cddevice can only be used once" >&2
opt_usage="2"
break
elif [ $# -eq 0 ]
then
echo "$name0:error: missing argument for the --cddevice option" >&2
opt_usage="2"
break
else
opt_cddevice="$1"
shift
fi
;;
--verbose)
verbose="1"
;;
--help|-help|-h|-?)
opt_usage="0"
break
;;
*)
if [ -z "$opt_vm" ]
then
opt_vm="$arg"
else
echo "$name0:error: unknown option '$arg'" >&2
opt_usage="2"
break
fi
esac
done
if [ -z "$opt_usage" ]
then
if [ -z "$opt_vm" ]
then
echo "$name0:error: you must specify a VM path" >&2
opt_usage="2"
elif [ -z "$opt_tag" ]
then
opt_tag=vmware-`basename "$opt_vm" | sed -e 's/\.[cv][fm][gx]$//' -e 's/ //g'`
fi
[ -n "$opt_version" ] || opt_version="latest"
if [ -z "$opt_snapshot" ]
then
opt_snapshot="winetest"
fi
[ -n "$opt_autorun" ] || opt_autorun="1"
if [ "$opt_autorun" = "0" -a -n "$opt_submit" ]
then
echo "$name0:error: --(no-)submit can only be used with the --autorun option" >&2
opt_usage="2"
fi
[ -n "$opt_submit" ] || opt_submit="1"
if [ "$opt_submit" = "0" -a -z "$opt_timeout" ]
then
opt_timeout="0"
fi
[ -n "$opt_timeout" ] || opt_timeout="30"
[ -n "$opt_cddevice" ] || opt_cddevice="ide1:0"
fi
if [ -n "$opt_usage" ]
then
if [ "$opt_usage" != "0" ]
then
echo "$name0:error: try '$name0 --help' for more information" >&2
exit $opt_usage
fi
echo "Usage: $name0 [--help] [--version VERSION] [--check-sig] [--tag TAG]"
echo " [--timeout TIMEOUT] [--submit|--no-submit]"
echo " [--autorun|--no-autorun] [--cddevice DEVICE]"
echo " [--snapshot SNAPSHOT] /path/to/vm.vmx"
echo
echo "Downloads the latest winetest.exe, and runs it in a VMware virtual machine by"
echo "putting it on a virtual CD. The state of the virtual machine is left"
echo "undisturbed."
echo
echo "Options:"
echo " --version VERSION Specifies which version of winetest.exe to download."
echo " By default this is 'latest'."
echo " --check-sig Specifies that winetest.exe should not be run if its"
echo " signature cannot be checked. By default, the tests are"
echo " only aborted if winetest.exe does not match its"
echo " signature. If --no-check-sig is used, then the"
echo " signature is not checked at all'"
echo " --tag TAG The tag to use when submitting the results. By default"
echo " this is the basename of the virtual machine file."
echo " --timeout TIMEOUT Specifies how long, in minutes, to give to winetest.exe"
echo " to run. By default this is 30 minutes."
echo " --submit Submit the test results to http://test.winehq.org/data/"
echo " This is the default. Use --no-submit to not submit the"
echo " results, and remove the 30 minutes timeout."
echo " --autorun Generate an autorun.inf file so the tests are started"
echo " automatically. This is the default. If you use"
echo " --no-autorun, then you will need to make your own"
echo " arrangements to start the tests from the CD."
echo " --cddevice DEVICE The name of the virtual machine's device for the cdrom"
echo " drive. By default this is 'ide1:0'."
echo " --snapshot SNAPSHOT The snapshot to use to run the tests. By default this"
echo " is 'winetest'. Make sure that, in this snapshot, the"
echo " cdrom device is connected. If you use the --autorun"
echo " option, then autorun should also be enabled."
echo " /path/to/vm.vmx This is the path to the virtual machine's vmx or cfg"
echo " file."
echo " --verbose Print some extra informational messages."
echo " --help Show this help message."
exit 0
fi
### Check that the VM is usable
verbose "Checking the VM..."
if vm_is_busy "$opt_vm"
then
echo "$name0:error: '$opt_vm' is already opened or running." >&2
if [ $opt_timeout -eq 0 ]
then
echo "$name0:error: Aborting." >&2
exit 1
fi
count=`expr $opt_timeout + 2`
while vm_is_busy "$opt_vm"
do
if [ $count -eq 0 ]
then
echo "$name0:error: '$opt_vm' is still in use. Giving up." >&2
exit 1
fi
count=`expr $count - 1`
sleep 60
done
fi
"$vmware/vmrun" listSnapshots "$opt_vm" | egrep "^$opt_snapshot\$" >/dev/null
if [ $? -ne 0 ]
then
echo "$name0:error: the '$opt_snapshot' snapshot does not exist. Aborting" >&2
exit 1
fi
### Grab winetest and prepare the ISO
verbose "Downloading winetest.exe..."
[ -n "$TMPDIR" ] || TMPDIR="/tmp"
tmpdir="$TMPDIR/$name0-$$"
if ! mkdir -p "$tmpdir/master"
then
echo "$name0:error: unable to create the '$tmpdir/master' directory" >&2
exit 1
fi
logfile="$tmpdir/$name0.log"
wget -O "$tmpdir/master/winetest.exe" "$base_url/winetest-$opt_version.exe" >"$logfile" 2>&1
if [ $? -ne 0 ]
then
echo "$name0:error: unable to download winetest.exe" >&2
echo
cat "$logfile"
exit 1
fi
wget -O "$tmpdir/master/winetest.exe.sig" "$base_url/winetest-$opt_version.exe.sig" >"$logfile" 2>&1
if [ $? -ne 0 -a "$opt_check_sig" != "0" ]
then
echo "$name0:error: unable to download winetest.exe.sig" >&2
echo
cat "$logfile"
exit 1
fi
### Check the winetest.exe signature
if [ "$opt_check_sig" != "0" ]
then
verbose "Checking the winetest.exe signature..."
type gpg >/dev/null 2>&1
if [ $? -eq 0 ]
then
gpg --verify "$tmpdir/master/winetest.exe.sig"
rc=$?
if [ $rc -eq 0 ]
then
echo "Signature checked."
elif [ $rc -eq 1 ]
then
echo "$name0:error: it looks like winetest.exe has been tampered with. Aborting" >&2
exit 1
elif [ "$opt_check_sig" = "1" ]
then
echo "$name0:warning: your system does not seem to be set up to check the signature. Aborting." >&2
exit 1
else
echo "$name0:warning: your system does not seem to be set up to check the signature. Continuing anyway..." >&2
fi
elif [ "$opt_check_sig" = "1" ]
then
echo "$name0:error: gpg is not installed. Aborting." >&2
exit 1
else
echo "$name0:warning: unable to check the winetest.exe signature because gpg is not installed. Continuing anyway..." >&2
fi
fi
### Create the CD image
verbose "Creating the CD image..."
if [ "$opt_autorun" = "1" ]
then
options=""
if [ "$opt_submit" = "0" ]
then
options="-o c:\\winetest.log"
fi
cat >"$tmpdir/master/autorun.inf" <<_EOF_
[autorun]
open=winetest.exe -q -t $opt_tag $options
_EOF_
fi
mkisofs -o "$tmpdir/winetest.iso" "$tmpdir/master" >"$logfile" 2>&1
if [ $? -ne 0 ]
then
echo "$name0:error: unable to create the CD image" >&2
echo
cat "$logfile"
exit 1
fi
### Prepare the VM
tmp_snapshot="$name0-`date +%Y%m%d`"
"$vmware/vmrun" listSnapshots "$opt_vm" | egrep "^$tmp_snapshot\$" >/dev/null
if [ $? -eq 0 ]
then
# This is not our snapshot so don't delete it after we're done
delete_snapshot=0
else
"$vmware/vmrun" snapshot "$opt_vm" "$tmp_snapshot"
if [ $? -ne 0 ]
then
echo "$name0:error: an error occurred while saving the VM's current state" >&2
exit 1
fi
delete_snapshot=1
fi
"$vmware/vmrun" revertToSnapshot "$opt_vm" "$opt_snapshot"
if [ $? -ne 0 ]
then
echo "$name0:error: an error occurred while switching to the '$opt_snapshot' snapshot" >&2
exit 1
fi
### Run the test
verbose "Running the tests..."
"$vmware/vmware" -s "$opt_cddevice.fileName=$tmpdir/winetest.iso" \
-s "$opt_cddevice.deviceType=cdrom-image" \
-x -q "$opt_vm" &
count=0
while [ $opt_timeout -eq 0 -o $count -lt $opt_timeout ]
do
sleep 60
if vm_is_running "$opt_vm"
then
count=`expr $count + 1`
else
break
fi
done
if [ $opt_timeout -ne 0 -a $count -gt $opt_timeout ]
then
"$vmware/vmrun" stop "$opt_vm"
fi
### Restore the VM and cleanup
verbose "Cleaning up..."
"$vmware/vmrun" revertToSnapshot "$opt_vm" "$tmp_snapshot"
if [ $? -ne 0 ]
then
echo "$name0:error: an error occurred while restoring the VM's to its initial state ($tmp_snapshot)" >&2
exit 1
fi
if [ $delete_snapshot -ne 0 ]
then
"$vmware/vmrun" deleteSnapshot "$opt_vm" "$tmp_snapshot"
fi
rm -rf "$tmpdir"
exit 0
More information about the wine-devel
mailing list