No subject

several dozen different tests, for various functions of wine.
Obviously, I'd like to start with platinum rated applications, to
prevent regressions. From there, the current list of
applications/functions to target are:
1. Firefox3 - useful for testing lots of other things, e.g.,
java/flash, etc. Easy to test.
2. Microsoft Office - We've broken this a few times in recent months,
so adding a test in daily git for it would help catch regressions
sooner.
3. World of Warcraft - I don't play, but A) It's really popular, and
B) also gives us a chance to test wininet for regressions.
4. iTunes - this installers been broken a couple times. The bonjour
service should allow for testing services related regressions.
5. Adobe suite - there haven't been too many regressions here that I
can remember, but it's definitely an area we should watch out for.

Of course, we don't want to only test the installer's keystrokes/mouse
inputs, so SHA1 checksumming of installed files would verify against
corrupted files/setupapi regressions/etc. Once a few installer tests
are in and working well, I plan to extend the tests further to test
running the application, e.g, for photoshop by opening a file,
inserting a circle, and saving it. This is a bit harder to pull off in
Autohotkey in a way that's portable, so I'm saving that sort of thing
for later.

Also interesting would be a real application's test suite. Dan
suggested using Google Chrome's, since it's squeaky clean after using
lots of Valgrind and Purify. That would be good, but I'd put that
after getting the test suite to pass in Wine.

This should be very doable using Autohotkey's scripting functionality
(pretty extensive - http://www.autohotkey.com/docs/Functions.htm,
http://www.autohotkey.com/docs/Variables.htm) and good old fashioned
shell scripting. I'm very familiar with bourne scripting, and
currently maintain winetricks while Dan's taking a Wine vacation. I'm
currently working on a build script to build Wine on several OS's, as
well as running and submitting the conformance tests
(http://winezeug.googlecode.com/svn/trunk/build_script/daily.sh). [I
got tired of maintaining the script over several different
OS's/computers]. The upside is it would be easy to add on appinstall
to this daily script. I typically set it running before leaving home
for work/school. It spends an hour or so building wine and running the
tests in a few different configurations. Adding appinstall would allow
my computer to daily test several applications for installation,
running, etc. This of course isn't as useful as Patchwatcher's testing
of each single patch, but I don't currently have a spare machine that
it can test on. Getting a script and tests ready to do so, however,
would be great for Wine. Besides, I can easily setup the script to
upload the results to a webserver, so we can daily see that each test
is passing. If a test is failing, the information would then be
publicly available for anyone to run 'git bisect'.

Wow, this turned into a really long e-mail, and I've got an exam to
study for. I'd appreciate any comments, and let me know if there are
any questions.

-- 
-Austin

--000e0cd403322f81e40465e5d055
Content-Type: application/octet-stream; name=appinstall
Content-Disposition: attachment; filename=appinstall
Content-Transfer-Encoding: base64
X-Attachment-Id: f_fsp78wyo1

IyEvYmluL2Jhc2gKIyBRdWljayBhbmQgZGlydHkgc2NyaXB0IHRvIGRvd25sb2FkIGFuZCBpbnN0
YWxsIHZhcmlvdXMgCiMgcmVkaXN0cmlidXRhYmxlIHJ1bnRpbWUgbGlicmFyaWVzCiMKIyBDb3B5
cmlnaHQgMjAwOC0yMDA5IEF1c3RpbiBFbmdsaXNoCiMgQmFzZWQgb2ZmIFdpbmV0cmlja3MgQ29w
eXJpZ2h0IEdvb2dsZSAoRGFuIEtlZ2VsLCBkYW5rQGtlZ2VsLmNvbSkKCiMgQ2xlYW4gdXAgYSBm
ZXcgdmFyaWFibGVzOgojIEZvciBub3csIGRvbid0IGFsbG93IHVzZXIgZGVmaW5lZCBwcmVmaXhl
czoKdW5zZXQgV0lORVBSRUZJWAoKIyBEZWZhdWx0IHZhbHVlcyBmb3IgaW1wb3J0YW50IHNldHRp
bmdzIGlmIG5vdCBhbHJlYWR5IGluIGVudmlyb25tZW50LgojIFRoZXNlIHNldHRpbmdzIHNob3Vs
ZCBub3QgbmVlZCBlZGl0aW5nIGhlcmUuCldJTkU9JHtXSU5FOi13aW5lfQpXSU5FUFJFRklYPSR7
V0lORVBSRUZJWDotJEhPTUUvLndpbmUtYXBwaW5zdGFsbH0KV0lORUNMRUFOUFJFRklYPSR7V0lO
RVBSRUZJWDotJEhPTUUvLndpbmUtYXBwaW5zdGFsbC1jbGVhbn0KV0lORVRSSUNLUz0ke1dJTkVU
UklDS1M6LXdpbmV0cmlja3N9CkFQUElOU1RBTExfQ0FDSEU9JEhPTUUvLmFwcGluc3RhbGwtY2Fj
aGUKQVBQSU5TVEFMTF9DQUNIRV9XSU49JFdJTkVQUkVGSVgvYXBwaW5zdGFsbF9jYWNoZQoKVkVS
U0lPTj0iQXBwaW5zdGFsbCAwLjAuMSAgKEMpIEF1c3RpbiBFbmdsaXNoIC0gTEdQTCAyLjEiCgp0
ZXN0IC1kICRBUFBJTlNUQUxMX0NBQ0hFIHx8IG1rZGlyICRBUFBJTlNUQUxMX0NBQ0hFCgojIFNv
bWUgaGVscGVyIGZ1bmN0aW9uczoKdHJ5KCkgewogICAgIyAiVkFSPWZvbyB0cnkgY21kIiBmYWls
cyB0byBwdXQgVkFSIGluIHRoZSBlbnZpcm9ubWVudAogICAgIyB3aXRoIHNvbWUgdmVyc2lvbnMg
b2YgYmFzaCBpZiB0cnkgaXMgYSBzaGVsbCBmdW5jdGlvbj8hCiAgICAjIEFkZGluZyB0aGlzIGV4
cGxpY2l0IGV4cG9ydCB3b3JrcyBhcm91bmQgaXQuCiAgICBleHBvcnQgV0lORVBSRUZJWAogICAg
Y2QgJEFQUElOU1RBTExfQ0FDSEUKICAgIGVjaG8gRXhlY3V0aW5nICIkQCIKICAgICIkQCIKICAg
IHN0YXR1cz0kPwogICAgaWYgdGVzdCAkc3RhdHVzIC1uZSAwCiAgICB0aGVuCiAgICAgICAgZGll
ICJOb3RlOiBjb21tYW5kICckQCcgcmV0dXJuZWQgc3RhdHVzICRzdGF0dXMuICBBYm9ydGluZy4i
CiAgICBmaQp9CgojIHZlcmlmeSBhbiBzaGExc3VtCnZlcmlmeV9zaGExc3VtKCkgewogICAgd2Fu
dHN1bT0kMQogICAgZmlsZT0kMgogICAKICAgIGdvdHN1bT1gJFNIQTFTVU0gPCAkZmlsZSB8IHNl
ZCAncy8gLiovLydgCiAgICBpZiBbICIkZ290c3VtInggIT0gIiR3YW50c3VtInggXQogICAgdGhl
bgogICAgICAgZGllICJzaGExc3VtIG1pc21hdGNoISAkZ290c3VtIDw+ICR3YW50c3VtICEgIFJl
bmFtZSAkZmlsZSBhbmQgdHJ5IGFnYWluLiIKICAgIGZpCn0KCiMgRG93bmxvYWQgYSBmaWxlCiMg
VXNhZ2U6IHBhY2thZ2UgdXJsIFtzaGExc3VtIFtmaWxlbmFtZV1dCiMgQ2FjaGVzIGRvd25sb2Fk
cyBpbiBhcHBpbnN0YWxsY2FjaGUvJHBhY2thZ2UKZG93bmxvYWQoKSB7CiAgICBpZiBbICIkNCJ4
ICE9ICIieCBdCiAgICB0aGVuCiAgICAgICAgZmlsZT0iJDQiCiAgICBlbHNlCiAgICAgICAgZmls
ZT1gYmFzZW5hbWUgIiQyImAKICAgIGZpCiAgICBjYWNoZT0iJEFQUElOU1RBTExfQ0FDSEUvJDEi
CiAgICBta2RpciAtcCAiJGNhY2hlIgogICAgaWYgISB0ZXN0IC1mICIkY2FjaGUvJGZpbGUiCiAg
ICB0aGVuCiAgICAgICAgY2QgIiRjYWNoZSIKICAgICAgICAjIE1hYyBmb2xrcyB0ZW5kIHRvIGhh
dmUgY3VybCByYXRoZXIgdGhhbiB3Z2V0CiAgICAgICAgIyBPbiBNYWMsICd3aGljaCcgZG9lc24n
dCByZXR1cm4gZ29vZCBleGl0IHN0YXR1cwogICAgICAgICMgTmVlZCB0byBqYW0gaW4gLS1oZWFk
ZXIgIkFjY2VwdC1FbmNvZGluZzogZ3ppcCxkZWZsYXRlIiBlbHNlCiAgICAgICAgIyByZWRoYXQu
Y29tIGRlY29tcHJlc3NlcyBsaWJlcmF0aW9uLWZvbnRzLnRhci5neiEKICAgICAgICBpZiBbIC14
ICJgd2hpY2ggd2dldGAiIF0KICAgICAgICB0aGVuCiAgICAgICAgICAgIyBVc2UgLW5kIHRvIGlu
c3VsYXRlIG91cnNlbHZlcyBmcm9tIHBlb3BsZSB3aG8gc2V0IC14IGluIFdHRVRSQwogICAgICAg
ICAgICMgWypdIC0tcmV0cnktY29ubnJlZnVzZWQgd29ya3MgYXJvdW5kIHRoZSBicm9rZW4gc2Yu
bmV0IG1pcnJvcmluZwogICAgICAgICAgICMgc3lzdGVtIHdoZW4gZG93bmxvYWRpbmcgY29yZWZv
bnRzCiAgICAgICAgICAgIyBbKl0gLS1yZWFkLXRpbWVvdXQgaXMgdXNlZnVsIG9uIHRoZSBhZG9i
ZSBzZXJ2ZXIgdGhhdCBkb2Vzbid0CiAgICAgICAgICAgIyBjbG9zZSB0aGUgY29ubmVjdGlvbiB1
bmxlc3MgeW91IHRlbGwgaXQgdG8gKGNvbnRyb2wtQyBvciBjbG9zaW5nCiAgICAgICAgICAgIyB0
aGUgc29ja2V0KQogICAgICAgICAgIHRyeSB3Z2V0IC1uZCAtYyAtLXJlYWQtdGltZW91dD0zMDAg
LS1yZXRyeS1jb25ucmVmdXNlZCAtLWhlYWRlciAiQWNjZXB0LUVuY29kaW5nOiBnemlwLGRlZmxh
dGUiICIkMiIKICAgICAgICBlbHNlCiAgICAgICAgICAgIyBjdXJsIGRvZXNuJ3QgZ2V0IGZpbGVu
YW1lIGZyb20gdGhlIGxvY2F0aW9uIGdpdmVuIGJ5IHRoZSBzZXJ2ZXIhCiAgICAgICAgICAgIyBm
b3J0dW5hdGVseSwgd2Uga25vdyBpdAogICAgICAgICAgIHRyeSBjdXJsIC1MIC1vICRmaWxlIC1D
IC0gLS1oZWFkZXIgIkFjY2VwdC1FbmNvZGluZzogZ3ppcCxkZWZsYXRlIiAiJDIiCiAgICAgICAg
ZmkKICAgICAgICBjZCAiJG9sZGRpciIKICAgIGZpCiAgICBpZiBbICIkMyJ4ICE9ICIieCBdCiAg
ICB0aGVuCgl2ZXJpZnlfc2hhMXN1bSAkMyAgIiRjYWNoZS8kZmlsZSIKICAgIGZpCn0KCiMgQ2hl
Y2sgZm9yIG5lZWRlZCBleHRlcm5hbCBzdHVmZgp3aGljaCAiJFdJTkUiID4gL2Rldi9udWxsCmlm
IHRlc3QgIiQ/IiAtbmUgIjAiCnRoZW4KICBkaWUgIkNhbm5vdCBmaW5kIFdpbmUgKCRXSU5FKSIK
ZmkKCndoaWNoICIkV0lORVRSSUNLUyIgPiAvZGV2L251bGwKaWYgdGVzdCAiJD8iIC1uZSAiMCIK
dGhlbgogIGRpZSAiQ2Fubm90IGZpbmQgV2luZXRyaWNrcyAoJFdJTkVUUklDS1MpIgpmaQoKIyBN
YWMgZm9sa3MgdGVuZCB0byBub3QgaGF2ZSBzaGExc3VtLCBidXQgd2UgY2FuIG1ha2UgZG8gd2l0
aCBvcGVuc3NsCmlmIFsgLXggImB3aGljaCBzaGExc3VtYCIgXQp0aGVuCiAgIFNIQTFTVU09InNo
YTFzdW0iCmVsc2UKICAgU0hBMVNVTT0ib3BlbnNzbCBkZ3N0IC1zaGExIgpmaQoKIyBQbGFuczoK
IyBPbiBmaXJzdCBydW4sIGNoZWNrIGZvciB+Ly53aW5lLWFwcGluc3RhbGwsIGlmIGV4aXN0cywg
bnVrZSwgaWYgbm90LCBydW46CiMgV0lORVBSRUZJWD1+Ly53aW5lLWFwcGluc3RhbGwgd2luZXRy
aWNrcyBhdXRvaG90a2V5CiMgVG8gaW5zdGFsbCBhdXRvaG90a2V5LCB0aGVuIHNldHVwIGEgbGlu
ayB0byBvdXIgY2FjaGUgZGlyZWN0b3J5OgojIGxuIC1zICRIT01FLy5hcHB0ZXN0Y2FjaGUgJEhP
TUUvLndpbmUtYXBwaW5zdGFsbC9kcml2ZV9jL2FwcHRlc3RjYWNoZQojIFRoZW4gYmFja3VwIG91
ciBzZXR1cCwgc28gd2UgZG9uJ3QgaGF2ZSB0byBlYWNoIHRpbWU6CiMgY3AgLXIgJEhPTUUvLndp
bmUtYXBwaW5zdGFsbCAkSE9NRS8ud2luZS1hcHBpbnN0YWxsLWNsZWFuCiMgU28gdGhhdCBmb3Ig
ZWFjaCBhcHAsIHdlIGNhbiBydW46CiMgY3AgLXIgJEhPTUUvLndpbmUtYXBwaW5zdGFsbC1jbGVh
biAkSE9NRS8ud2luZS1hcHBpbnN0YWxsICYmCiMgd2luZSAiQzpcUHJvZ3JhbSBGaWxlc1xBdXRv
SG90a2V5XEF1dG9ob3RrZXkuZXhlIiBhcHBfc2NyaXB0LmFoayAmJgojIGdvX3RvX25leHQgfHwg
ZWNobyBmYWlsICYmIGdvX3RvX25leHQKCiMgUHJlZml4IHN0dWZmOgoKY3JlYXRlcHJlZml4KCkg
ewogICAgZWNobyAnQ3JlYXRpbmcgY2xlYW4gV0lORVBSRUZJWC4nCiAgICAjIEZpcnN0LCB3aW5l
Ym9vdCwgc28gd2UgaGF2ZSBhIGRpcmVjdG9yeSB0byBzeW1saW5rLCBldGMuIE5PVEU6IEFkZCBk
aWUgIndpbmVib290IGZhaWxlZCBoZXJlIgogICAgd2luZSB3aW5lYm9vdCAmJgoKICAgICMgTmV4
dCwgc2V0dXAgYSBzeW1saW5rIHRvIG91ciB0ZXN0aW5nIGNhY2hlOgogICAgbG4gLXMgJEFQUElO
U1RBTExfQ0FDSEUgJEFQUElOU1RBTExfQ0FDSEVfV0lOCgogICAgIyBUaGVuLCByZW1vdmUgWjpc
IHN5bWxpbmssIHRvIGlzb2xhdGUgdGhlIHByZWZpeDoKICAgICMgRGlzYWJsZWQgZm9yIG5vdywg
c2VlbXMgdG8gYmUgcmVtb3ZpbmcgcmVjdXJzaXZlbHk/IE5lZWQgdG8gY2hlY2sgc2xhc2hlcy4K
ICAgICMgcm0gLWkgJFdJTkVQUkVGSVgvZG9zZGV2aWNlcy96XDoKCiAgICAjIE5vdywgaW5zdGFs
bCBBdXRvSG90S2V5IHNpbGVudGx5OgoKICAgIGVjaG8gYHB3ZGAKICAgIGRvd25sb2FkIC4gaHR0
cDovL3d3dy5hdXRvaG90a2V5Lm5ldC9wcm9ncmFtcy9BdXRvSG90a2V5MTA0NzA2X0luc3RhbGwu
ZXhlIDNkM2Q4ODQ1NDczZGVhNDc3ZDY5ODNkMDYzZjBhZmM5OTk5ZDg4MGYKICAgIHRyeSAkV0lO
RSBBdXRvSG90a2V5MTA0NzA2X0luc3RhbGwuZXhlICIvUyIKCiAgICAjIEZpbmFsbHksIGJhY2t1
cCB0aGlzIGNsZWFuIFdJTlBSRUZJWCB0byBzYXZlIGEgZmV3IHNlY29uZHMgb2YgdGltZSBvbiBl
YWNoIHJ1bjoKICAgIGNwIC1hICRXSU5FUFJFRklYICRXSU5FQ0xFQU5QUkVGSVgKCiAgICBlY2hv
ICdCYWNraW5nIHVwIGNsZWFuIFdJTkVQUkVGSVguJwp9CgoKZm9vY3JlYXRlcHJlZml4KCkgewp3
aW5ldHJpY2tzIGF1dG9ob3RrZXkKbG4gLXMgJEFQUElOU1RBTExfQ0FDSEUgJEFQUElOU1RBTExf
Q0FDSEVfV0lOCiMgRGlzYWJsZWQgZm9yIG5vdywgc2VlbXMgdG8gYmUgcmVtb3ZpbmcgcmVjdXJz
aXZlbHk/IE5lZWQgdG8gY2hlY2sgc2xhc2hlcy4KIyBybSAtdiAkV0lORVBSRUZJWC9kb3NkZXZp
Y2VzL3pcOgoKY3AgLWEgJFdJTkVQUkVGSVggJFdJTkVDTEVBTlBSRUZJWAp9CgpkaWUoKSB7CiAg
ZWNobyAiJEAiCiAgZXhpdCAxCn0KCm51a2VwcmVmaXgoKSB7CiAgcm0gLXJmICRXSU5FUFJFRklY
Cn0KCnRlc3QgLWQgIiRXSU5FUFJFRklYIiAmJiBudWtlcHJlZml4Cgp0ZXN0IC1kICIkV0lORVBS
RUZJWCIgfHwgY3JlYXRlcHJlZml4CgpjYXNlICIkMSIgaW4KLVZ8LS12ZXJzaW9uKSAKICBlY2hv
ICIkVkVSU0lPTiIKICBleGl0IDAKICA7Owplc2FjCgojLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t
LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLQoKdXNhZ2UoKSB7CiAgICBz
ZXQgK3gKICAgIGVjaG8gIlVzYWdlOiAkMCBbb3B0aW9uc10gcGFja2FnZSBbcGFja2FnZV0gLi4u
IgogICAgZWNobyAiVGhpcyBzY3JpcHQgYXV0b21hdGljYWxseSBpbnN0YWxscyBhIGZldyBXaW5k
b3dzIGFwcGxpY2F0aW9ucyIKICAgIGVjaG8gImluIFdpbmUgYXMgYSBmb3JtIG9mIHJlZ3Jlc3Np
b24gdGVzdGluZy4iCiAgICBlY2hvICIiCiAgICBlY2hvICJPcHRpb25zOiIKICAgIGVjaG8gIiAt
cSAgICAgICAgIHF1aWV0LiAgWW91IG11c3QgaGF2ZSBhbHJlYWR5IGFncmVlZCB0byB0aGUgRVVM
QXMuIgogICAgZWNobyAiIC12ICAgICAgICAgdmVyYm9zZSIKICAgIGVjaG8gIiAtViAgICAgICAg
IGRpc3BsYXkgVmVyc2lvbiIKICAgIGVjaG8gIkFwcHM6IgogICAgZWNobyAiIGZpcmVmb3gzICAg
ICAgRmlyZWZveCBWZXJzaW9uIDMiCn0KCiMgQXBwbGljYXRpb25zIHRvIGluc3RhbGwuIExldCdz
IGFpbSBmb3IgT3BlbiBzb3VyY2UsIHBsYXRpbnVtIGluc3RhbGxlcnMKIyBUaGVuIGJyYW5jaCBv
dXQgdG8gbW9yZSBjb21wbGV4IGluc3RhbGxlcnMsIHRlc3Rpbmcgc2VydmljZXMsIGV0Yy4KIyBp
VHVuZXMsIEknbSBsb29raW5nIGF0IHlvdS4KCiMtLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t
LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tCgpmaXJlZm94MygpIHsKICAgICMg
RmlyZWZveCAzCiAgICBkb3dubG9hZCAuICJodHRwOi8vcmVsZWFzZXMubW96aWxsYS5vcmcvcHVi
L21vemlsbGEub3JnL2ZpcmVmb3gvcmVsZWFzZXMvMy4wL3dpbjMyL2VuLVVTL0ZpcmVmb3ggU2V0
dXAgMy4wLmV4ZSIgOTFjZjk5ZWRiYzRlYjFlZjZhZjMxNzllNjEyNmFiYzY1OTMwOTI2NQogICAg
ICAgdHJ5ICRXSU5FICJGaXJlZm94IFNldHVwIDMuMC5leGUiIAp9CgojLS0tLS0tLS0tLS0tLS0t
LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLQoKd2hpbGUg
dGVzdCAiJDEiICE9ICIiCmRvCiAgICBQQUNLQUdFPSQxCiAgICBjYXNlICQxIGluCiAgICAtcSkg
QVBQSU5TVEFMTF9RVUlFVD0iL3EiCiAgICAgICAgQVBQSU5TVEFMTF9VTklYUVVJRVQ9Ii1xIgog
ICAgICAgIEFQUElOU1RBTExfU0lMRU5UPSIvc2lsZW50IgogICAgICAgIEFQUElOU1RBTExfUz0i
L1MiICAgICAgICAgICAgICAgICAjIGZvciBOU0lTIGluc3RhbGxlcnMKICAgICAgICBXSU5FREVC
VUc9JHtXSU5FREVCVUc6LSJmaXhtZS1hbGwifQogICAgICAgIGV4cG9ydCBXSU5FREVCVUcKICAg
ICAgICA7OwogICAgLXYpIHNldCAteDs7CgogICAgIGZpcmVmb3gzKSBmaXJlZm94Mzs7CgogICAg
KikgZWNobyBVbmtub3duIGFyZyAkMTsgdXNhZ2UgOyBleGl0IDE7OwogICAgZXNhYwogICAgIyBQ
cm92aWRlIGEgYml0IG9mIGZlZWRiYWNrCiAgICB0ZXN0ICIkQVBQSU5TVEFMTF9RVUlFVCIgPSAi
IiAmJiBjYXNlICQxIGluIAogICAgLXEpIGVjaG8gU2V0dGluZyBxdWlldCBtb2RlOzsKICAgIC12
KSBlY2hvIFNldHRpbmcgdmVyYm9zZSBtb2RlOzsKICAgICopIGVjaG8gIkluc3RhbGwgb2YgJDEg
ZG9uZSIgOzsKICAgIGVzYWMKICAgIHNoaWZ0CmRvbmUKCiMgY2xlYW51cApybSAtcmYgIiRBUFBJ
TlNUQUxMX1RNUCIvKgoKdGVzdCAiJEFQUElOU1RBTExfUVVJRVQiID0gIiIgJiYgZWNobyAkVkVS
U0lPTiBkb25lLiB8fCB0cnVlCg==
--000e0cd403322f81e40465e5d055--