WineHQ
WineHQ

3.2. How to do regression testing using Git

A problem that can happen sometimes is "it used to work before, now it doesn't anymore...". Here is a step by step procedure to try to pinpoint when the problem occurred. This is NOT for casual users.

  1. Clone the "Git" repository from winehq. It's more than 100Mb, so it may take some time with a slow Internet connection.

  2. If you found that something broke between wine-1.1.42 and wine-1.1.44 (these are [WWW] release tags). To start regression testing we run:

    git bisect start
    git bisect good wine-1.1.42
    git bisect bad wine-1.1.44

    If you have exact date/time instead of a release you will need to use sha1 IDs from git log.

  3. Having told Git when things were working and when they broke, it will automatically "position" your source tree to the middle. So all you need to do is build the source:

    ./configure && make clean && make depend && make
    ./wine 'c:\test.exe'

    If the version of Wine that Git picked still has the bug, run:

    git bisect bad
    and if it does not, run:
    git bisect good

    When you run this command, Git will checkout a new version of Wine for you to rebuild, so repeat this step again. When the regression has been isolated, git will inform you.

    To find out what's left to test, try:

    git bisect visualize.

  4. When you have found the bad patch and want to go back to the current HEAD run:

    git bisect reset

  5. If you find the patch that is the cause of the problem, you have almost won; report about it to Wine Bugzilla or subscribe to wine-devel and post it there. There is a chance that the author will jump in to suggest a fix; or there is always the possibility to look hard at the patch until it is coerced to reveal where is the bug :-)