Your weekly winetest update

Thu Feb 1 20:10:28 CST 2018

Hello folks, my name is Zebediah Figura, and I'll be taking over for
Jeremy White today.

I'd like to start out by thanking everyone who has been working on
fixing tests, with special notice to Alex Henrie who has fixed more
Windows tests recently than probably anyone else combined. This mail
however is not about Windows at all but Wine.

Item one: it seems to me—perhaps naïvely so, but it seems to me that we
are closer than ever to having an entirely green machine (well, orange
really) on the Linux end—so much so that I'm going to run through the
failures that are left. Please do take the time to at least skim, since
in some cases this is an RFC and what to do to fix a test isn't exactly
clear even though the cause is. I also have a few other comments, so I'd
appreciate your reading through to the end of this mail.

So, without further ado:

---

* kernel32:change is due to the asynchronous nature of inotify. How
should this be fixed? inotify gives us one creation event and two or
more notifications on the parent. It's not clear how to fix this in the
test. It certainly doesn't seem possible to make the server behave like
Windows.

* ole32:clipboard is explained and fixed by 140671 [1].

* quartz:filtergraph is a number of problems related to winegstreamer
teardown. I have a set of patches in my local tree which should clear up
most to all of these, and I'd like to start submitting these soon.

* shell32:shelllink needs some discussion, I think. The problem is that
winemenubuilder is holding the file when we check for leaks. On Windows
I would imagine all menu-building would be done synchronously, but on
Wine this is implausible since we need to create our own copy of the
link icon possibly after the link is created. At one point I suggested
that menu-building should occur on prefix shutdown, which, while not
really possible to do on top of the Windows API, might nevertheless be
the best solution.

* urlmon:protocol is explained and fixed by 121083 [2]. This solution is
less clean than is ideal, but my further attempts to synchronize more
cleanly were met with other failures. I'm willing to take another look
at this, although I'd appreciate help.

* user32:msg is now down to one consistent failure. This is explained
and fixed by 140924/140925 [3]/[4].

* user32:win, perhaps one of the most notorious offenders, has three
consistent failures left, and possibly some more intermittent ones.
  - Lines 3364-5 (test_SetForegroundWindow) and 9940-4
(test_LockWindowUpdate) were determined by jwhite to be the fault of the
hwndMain test window being minimized during the test. I further
investigated and traced this down, surprisingly enough, to a bug in
fvwm, which I've reported at [5]. The bug involves a lot of steps to
trigger, notably, and we can (and, I think, should) work around it by
modifying the test machine configuration: either by changing the focus
mode from MouseFocus, or by removing the RaiseTransient or
StackTransientParent style.
  - Line 6838 (test_EnableWindow) is explained and fixed by 141063 [6].

* wininet:ftp, which affects both Linux and Windows, is a recent
failure, as can be seen at [7]. But nothing was changed in the tests at
that point. It seems more likely that something changed with respect to
the ftp.winehq.org server itself, some time on or before 3 January.
Could someone with knowledge of wininet or of the FTP server be implored
to look into this?

Then there are the following failures. I haven't investigated any of
these at all, partly because I'm as yet completely unfamiliar with the
respective APIs:

* A number of Direct3D failures, all (?) of which seem to be dependent
on the driver used. I'm not going to enumerate them all here, but I will
make note of the fact that cw1 only needs d3d10core:device, d3d11:d3d11,
and d3d9:visual, and cw2 only needs d3d9:device. All of these are
consistent.
* mmdevapi:capture, intermittent and rare.
* mmdevapi:render, intermittent but less rare.
* mshtml:htmldoc, intermittent but almost consistent.
* urlmon:misc, intermittent.
* xaudio2_7:xaudio2, intermittent.

While I'm certainly willing to fix every last failure myself if need be,
just think how much easier it would be if people who were familiar with
the code took a look at it!

Then there's one very weird set of failures: every D3D-related test from
ddraw:ddraw2 onward (i.e. ddraw, dxdiagn, dxgi), on cw1-hd6800-wow32—and
only that architecture—fails to print its summary line. This has been
the case for since at least early November (unfortunately I lost the
exact date when this started happening, but it wasn't obviously the
result of any commit).

And, finally, the 64-bit failures:
* d3dx9_36:math is slightly off on the spherical harmonics values due to
the differences in FP math on 64-bit. Here's inviting someone who knows
the math to check all of our numbers and make sure they're correct, or
otherwise do as Alex Henrie did for spherical light [8].
All others are due to lack of typelib marshaling:
* ieframe:ie
* msxml3:domdoc
* oleaut32:tmarshal
* shell32:shelldispatch
* user32:dialog (indirectly, since msxml3:domdoc crashes and leaves the
winedbg window open)

Perhaps in these cases the best thing to do is to mark those
todo_wine_if (sizeof (void *) == 8), since properly implementing the
64-bit marshaller takes more work than anyone is willing to do.

---

The ultimate goal, of course, is that we finally get a way to run
testbot patches on [ideally multiple] Linux machines. Nobody needs me to
tell them how important this is, but of course it's also important that
we start from a clean slate, since these tests are going to be run for
*every* patch, not just those that modify the tests. So while Linux is
in a better state than some Windows versions at this point, it seems to
me that we really do want a wholly green machine.

Along these lines it's worth noting, by the way, that except for the D3D
failures, there seems to be no difference between cw1 and cw2. It seems
to me it would be a good thing if we started testing multiple window
managers as well (and in particular we may want to test those which see
common use such as KWin and Mutter).

Also along these lines: presumably setting up the testbot to run linux
tests is going to require a fair amount of work, and that work is going
to be implied to be François' responsibility. I'd just like to say that
I'm willing to help as much as I can, though.

Okay, so, all that said, we have another problem. I've tried running
winetest on my own machine recently, in the interest of rooting out some
of the rarer failures, and I've found that the reports it generates are
consistently about 1.7 MB long, which is of course over the 1.5 MB limit
that test.winehq.org accepts. I did some sed'ing and came up with the
worst offenders as being:

test		   bytes  % of report
d3d11:d3d11	  286861 16.27
ws2_32:sock	  108672  6.17
kernel32:virtual   75963  4.29
gdi32:font	   64700  3.61
user32:win	   62114  3.52
shell32:shlexec	   41235  2.34
user32:msg	   37126  2.11

This isn't, I assert, something wrong with my setup. The D3D team's
obviously admirable predisposition toward verbosity, and the need to
print a large number of floating-point values, combined with a total of
2167 tests marked todo, has taken over almost one-sixth of the report.
But even with d3d11 removed, the report just barely sneaks in under the
limit, at 1475812 bytes. One might blame an excess of trace lines, as
last time, but in all reality, I am inclined to think that 1.5 MB is no
longer a reasonable limit for a test report. If there is an inherent
limitation brought on by server storage space or processing power, well,
so be it, but if not we probably will want to either raise it—or
otherwise start investing energy into getting rid of existing todos.

Alright, that's finally it from me, I promise. At least for now ;-)

[1] https://source.winehq.org/patches/data/140671
[2] https://www.winehq.org/pipermail/wine-devel/2018-January/121083.html
[3] https://source.winehq.org/patches/data/140924
[4] https://source.winehq.org/patches/data/140925
[5] https://github.com/fvwmorg/fvwm/issues/51
[6] https://source.winehq.org/patches/data/141063
[7] http://test.winehq.org/data/tests/wininet:ftp.html
[8]
https://source.winehq.org/git/wine.git/commit/07f7022fccc1c51eb4ccb4b6735e76f6a5ec13b7