RFC: Allow marking unreliable tests as flaky.

Wed Jun 1 12:36:50 CDT 2022

Hi,

While running your changed tests, I think I found new failures.
Being a bot and all I'm not very good at pattern recognition, so I might be
wrong, but could you please double-check?

Full results can be found at:
https://testbot.winehq.org/JobDetails.pl?Key=115928

Your paranoid android.

=== w7u_2qxl (32 bit report) ===

Report validation errors:
d3d8:device has no test summary line (early exit of the main process?)
d3d8:device has unaccounted for skip messages

=== w7u_adm (32 bit report) ===

Report validation errors:
d3d8:device has no test summary line (early exit of the main process?)
d3d8:device has unaccounted for skip messages

=== w7u_el (32 bit report) ===

Report validation errors:
d3d8:device has no test summary line (early exit of the main process?)
d3d8:device has unaccounted for skip messages

=== w8 (32 bit report) ===

Report validation errors:
d3d8:device has no test summary line (early exit of the main process?)
d3d8:device has unaccounted for skip messages

=== w8adm (32 bit report) ===

Report validation errors:
d3d8:device has no test summary line (early exit of the main process?)
d3d8:device has unaccounted for skip messages

=== w864 (32 bit report) ===

Report validation errors:
d3d8:device has no test summary line (early exit of the main process?)
d3d8:device has unaccounted for skip messages

=== w1064v1507 (32 bit report) ===

Report validation errors:
d3d8:device has no test summary line (early exit of the main process?)
d3d8:device has unaccounted for skip messages

=== w1064v1809 (32 bit report) ===

Report validation errors:
d3d8:device has no test summary line (early exit of the main process?)
d3d8:device has unaccounted for skip messages

=== w1064 (32 bit report) ===

Report validation errors:
d3d8:device has no test summary line (early exit of the main process?)
d3d8:device has unaccounted for skip messages

=== w1064_tsign (32 bit report) ===

Report validation errors:
d3d8:device has no test summary line (early exit of the main process?)
d3d8:device has unaccounted for skip messages

=== w10pro64 (32 bit report) ===

Report validation errors:
d3d8:device has no test summary line (early exit of the main process?)
d3d8:device has unaccounted for skip messages

=== w7u_2qxl (32 bit report) ===

Report validation errors:
dinput:mouse has no test summary line (early exit of the main process?)

=== w7u_adm (32 bit report) ===

Report validation errors:
dinput:mouse has no test summary line (early exit of the main process?)

=== w7u_el (32 bit report) ===

Report validation errors:
dinput:mouse has no test summary line (early exit of the main process?)

=== w8 (32 bit report) ===

Report validation errors:
dinput:mouse has no test summary line (early exit of the main process?)

=== w8adm (32 bit report) ===

Report validation errors:
dinput:mouse has no test summary line (early exit of the main process?)

=== w864 (32 bit report) ===

Report validation errors:
dinput:mouse has no test summary line (early exit of the main process?)

=== w1064v1507 (32 bit report) ===

Report validation errors:
dinput:mouse has no test summary line (early exit of the main process?)

=== w1064v1809 (32 bit report) ===

Report validation errors:
dinput:mouse has no test summary line (early exit of the main process?)

=== w1064 (32 bit report) ===

Report validation errors:
dinput:mouse has no test summary line (early exit of the main process?)

=== w1064_tsign (32 bit report) ===

Report validation errors:
dinput:mouse has no test summary line (early exit of the main process?)

=== w10pro64 (32 bit report) ===

Report validation errors:
dinput:mouse has no test summary line (early exit of the main process?)

=== w864 (64 bit report) ===

Report validation errors:
dinput:mouse has no test summary line (early exit of the main process?)

=== w1064v1507 (64 bit report) ===

Report validation errors:
dinput:mouse has no test summary line (early exit of the main process?)

=== w1064v1809 (64 bit report) ===

Report validation errors:
dinput:mouse has no test summary line (early exit of the main process?)

=== w1064 (64 bit report) ===

Report validation errors:
dinput:mouse has no test summary line (early exit of the main process?)

=== w1064_2qxl (64 bit report) ===

Report validation errors:
dinput:mouse has no test summary line (early exit of the main process?)

=== w1064_adm (64 bit report) ===

Report validation errors:
dinput:mouse has no test summary line (early exit of the main process?)

=== w1064_tsign (64 bit report) ===

Report validation errors:
dinput:mouse has no test summary line (early exit of the main process?)

=== w10pro64 (64 bit report) ===

Report validation errors:
dinput:mouse has no test summary line (early exit of the main process?)

=== w10pro64_en_AE_u8 (64 bit report) ===

Report validation errors:
dinput:mouse has no test summary line (early exit of the main process?)

=== w10pro64_ar (64 bit report) ===

Report validation errors:
dinput:mouse has no test summary line (early exit of the main process?)

=== w10pro64_ja (64 bit report) ===

Report validation errors:
dinput:mouse has no test summary line (early exit of the main process?)

=== w10pro64_zh_CN (64 bit report) ===

Report validation errors:
dinput:mouse has no test summary line (early exit of the main process?)

=== w7u_2qxl (32 bit report) ===

mmdevapi:
render.c:1114: Test failed: Position 6495 too far after playing 100ms

Report validation errors:
mmdevapi:render has no test summary line (early exit of the main process?)
mmdevapi:render has unaccounted for failure messages
mmdevapi:render has unaccounted for skip messages

=== w7u_adm (32 bit report) ===

Report validation errors:
mmdevapi:render has no test summary line (early exit of the main process?)
mmdevapi:render has unaccounted for skip messages

=== w7u_el (32 bit report) ===

Report validation errors:
mmdevapi:render has no test summary line (early exit of the main process?)
mmdevapi:render has unaccounted for skip messages

=== w8 (32 bit report) ===

Report validation errors:
mmdevapi:render has no test summary line (early exit of the main process?)
mmdevapi:render has unaccounted for failure messages
mmdevapi:render has unaccounted for skip messages

=== w8adm (32 bit report) ===

Report validation errors:
mmdevapi:render has no test summary line (early exit of the main process?)
mmdevapi:render has unaccounted for skip messages

=== w864 (32 bit report) ===

mmdevapi:
render.c:1340: Test failed: GetBuffer large (20671) at iteration 2
render.c:1114: Test failed: Position 6656 too far after playing 100ms

Report validation errors:
mmdevapi:render has no test summary line (early exit of the main process?)
mmdevapi:render has unaccounted for failure messages
mmdevapi:render has unaccounted for skip messages

=== w1064v1507 (32 bit report) ===

Report validation errors:
mmdevapi:render has no test summary line (early exit of the main process?)
mmdevapi:render has unaccounted for skip messages

=== w1064v1809 (32 bit report) ===

mmdevapi:
render.c:1114: Test failed: Position 54464 too far after playing 100ms
render.c:1114: Test failed: Position 6494 too far after playing 100ms

Report validation errors:
mmdevapi:render has no test summary line (early exit of the main process?)
mmdevapi:render has unaccounted for failure messages
mmdevapi:render has unaccounted for skip messages

=== w1064 (32 bit report) ===

mmdevapi:
render.c:1114: Test failed: Position 58280 too far after playing 100ms
render.c:1114: Test failed: Position 6866 too far after playing 100ms

Report validation errors:
mmdevapi:render has no test summary line (early exit of the main process?)
mmdevapi:render has unaccounted for failure messages
mmdevapi:render has unaccounted for skip messages

=== w1064_tsign (32 bit report) ===

mmdevapi:
render.c:1114: Test failed: Position 58568 too far after playing 100ms
render.c:1340: Test failed: GetBuffer large (22500) at iteration 6
render.c:1114: Test failed: Position 7323 too far after playing 100ms

Report validation errors:
mmdevapi:render has no test summary line (early exit of the main process?)
mmdevapi:render has unaccounted for failure messages
mmdevapi:render has unaccounted for skip messages

=== w10pro64 (32 bit report) ===

mmdevapi:
render.c:1114: Test failed: Position 57944 too far after playing 100ms
render.c:1114: Test failed: Position 6922 too far after playing 100ms

Report validation errors:
mmdevapi:render has no test summary line (early exit of the main process?)
mmdevapi:render has unaccounted for failure messages
mmdevapi:render has unaccounted for skip messages

=== w864 (64 bit report) ===

mmdevapi:
render.c:1340: Test failed: GetBuffer large (20671) at iteration 5

Report validation errors:
mmdevapi:render has no test summary line (early exit of the main process?)
mmdevapi:render has unaccounted for failure messages
mmdevapi:render has unaccounted for skip messages

=== w1064v1507 (64 bit report) ===

mmdevapi:
render.c:1340: Test failed: GetBuffer large (20671) at iteration 6
render.c:1114: Test failed: Position 6728 too far after playing 100ms

Report validation errors:
mmdevapi:render has no test summary line (early exit of the main process?)
mmdevapi:render has unaccounted for failure messages
mmdevapi:render has unaccounted for skip messages

=== w1064v1809 (64 bit report) ===

mmdevapi:
render.c:1114: Test failed: Position 6421 too far after playing 100ms

Report validation errors:
mmdevapi:render has no test summary line (early exit of the main process?)
mmdevapi:render has unaccounted for failure messages
mmdevapi:render has unaccounted for skip messages

=== w1064 (64 bit report) ===

mmdevapi:
render.c:1340: Test failed: GetBuffer large (22500) at iteration 5
render.c:1114: Test failed: Position 6878 too far after playing 100ms

Report validation errors:
mmdevapi:render has no test summary line (early exit of the main process?)
mmdevapi:render has unaccounted for failure messages
mmdevapi:render has unaccounted for skip messages

=== w1064_2qxl (64 bit report) ===

mmdevapi:
render.c:1114: Test failed: Position 57872 too far after playing 100ms
render.c:1114: Test failed: Position 7054 too far after playing 100ms

Report validation errors:
mmdevapi:render has no test summary line (early exit of the main process?)
mmdevapi:render has unaccounted for failure messages
mmdevapi:render has unaccounted for skip messages

=== w1064_adm (64 bit report) ===

mmdevapi:
render.c:1114: Test failed: Position 58688 too far after playing 100ms
render.c:1340: Test failed: GetBuffer large (22500) at iteration 7
render.c:1340: Test failed: GetBuffer large (22500) at iteration 8
render.c:1114: Test failed: Position 7091 too far after playing 100ms
render.c:1340: Test failed: GetBuffer large (22500) at iteration 5

Report validation errors:
mmdevapi:render has no test summary line (early exit of the main process?)
mmdevapi:render has unaccounted for failure messages
mmdevapi:render has unaccounted for skip messages

=== w1064_tsign (64 bit report) ===

mmdevapi:
render.c:1114: Test failed: Position 58664 too far after playing 100ms
render.c:1114: Test failed: Position 6820 too far after playing 100ms

Report validation errors:
mmdevapi:render has no test summary line (early exit of the main process?)
mmdevapi:render has unaccounted for failure messages
mmdevapi:render has unaccounted for skip messages

=== w10pro64 (64 bit report) ===

mmdevapi:
render.c:1114: Test failed: Position 58624 too far after playing 100ms
render.c:1114: Test failed: Position 7522 too far after playing 100ms

Report validation errors:
mmdevapi:render has no test summary line (early exit of the main process?)
mmdevapi:render has unaccounted for failure messages
mmdevapi:render has unaccounted for skip messages

=== w10pro64_en_AE_u8 (64 bit report) ===

mmdevapi:
render.c:1114: Test failed: Position 56680 too far after playing 100ms
render.c:1114: Test failed: Position 8151 too far after playing 100ms

Report validation errors:
mmdevapi:render has no test summary line (early exit of the main process?)
mmdevapi:render has unaccounted for failure messages
mmdevapi:render has unaccounted for skip messages

=== w10pro64_ar (64 bit report) ===

mmdevapi:
render.c:1114: Test failed: Position 55992 too far after playing 100ms

Report validation errors:
mmdevapi:render has no test summary line (early exit of the main process?)
mmdevapi:render has unaccounted for failure messages
mmdevapi:render has unaccounted for skip messages

=== w10pro64_ja (64 bit report) ===

mmdevapi:
render.c:1114: Test failed: Position 55992 too far after playing 100ms
render.c:1114: Test failed: Position 7804 too far after playing 100ms

Report validation errors:
mmdevapi:render has no test summary line (early exit of the main process?)
mmdevapi:render has unaccounted for failure messages
mmdevapi:render has unaccounted for skip messages

=== w10pro64_zh_CN (64 bit report) ===

mmdevapi:
render.c:1114: Test failed: Position 57280 too far after playing 100ms
render.c:1114: Test failed: Position 7404 too far after playing 100ms

Report validation errors:
mmdevapi:render has no test summary line (early exit of the main process?)
mmdevapi:render has unaccounted for failure messages
mmdevapi:render has unaccounted for skip messages

=== w7u_2qxl (32 bit report) ===

Report validation errors:
schedsvc:atsvcapi has no test summary line (early exit of the main process?)

=== w7u_adm (32 bit report) ===

Report validation errors:
schedsvc:atsvcapi has no test summary line (early exit of the main process?)
schedsvc:atsvcapi has unaccounted for skip messages

=== w7u_el (32 bit report) ===

Report validation errors:
schedsvc:atsvcapi has no test summary line (early exit of the main process?)

=== w8 (32 bit report) ===

Report validation errors:
schedsvc:atsvcapi has no test summary line (early exit of the main process?)
schedsvc:atsvcapi has unaccounted for skip messages

=== w8adm (32 bit report) ===

Report validation errors:
schedsvc:atsvcapi has no test summary line (early exit of the main process?)
schedsvc:atsvcapi has unaccounted for skip messages

=== w864 (32 bit report) ===

Report validation errors:
schedsvc:atsvcapi has no test summary line (early exit of the main process?)
schedsvc:atsvcapi has unaccounted for skip messages

=== w1064v1507 (32 bit report) ===

Report validation errors:
schedsvc:atsvcapi has no test summary line (early exit of the main process?)
schedsvc:atsvcapi has unaccounted for skip messages

=== w1064v1809 (32 bit report) ===

Report validation errors:
schedsvc:atsvcapi has no test summary line (early exit of the main process?)
schedsvc:atsvcapi has unaccounted for skip messages

=== w1064 (32 bit report) ===

Report validation errors:
schedsvc:atsvcapi has no test summary line (early exit of the main process?)
schedsvc:atsvcapi has unaccounted for skip messages

=== w1064_tsign (32 bit report) ===

Report validation errors:
schedsvc:atsvcapi has no test summary line (early exit of the main process?)
schedsvc:atsvcapi has unaccounted for skip messages

=== w10pro64 (32 bit report) ===

Report validation errors:
schedsvc:atsvcapi has no test summary line (early exit of the main process?)
schedsvc:atsvcapi has unaccounted for skip messages

=== w864 (64 bit report) ===

Report validation errors:
schedsvc:atsvcapi has no test summary line (early exit of the main process?)
schedsvc:atsvcapi has unaccounted for skip messages

=== w1064v1507 (64 bit report) ===

Report validation errors:
schedsvc:atsvcapi has no test summary line (early exit of the main process?)
schedsvc:atsvcapi has unaccounted for skip messages

=== w1064v1809 (64 bit report) ===

Report validation errors:
schedsvc:atsvcapi has no test summary line (early exit of the main process?)
schedsvc:atsvcapi has unaccounted for skip messages

=== w1064 (64 bit report) ===

Report validation errors:
schedsvc:atsvcapi has no test summary line (early exit of the main process?)
schedsvc:atsvcapi has unaccounted for skip messages

=== w1064_2qxl (64 bit report) ===

Report validation errors:
schedsvc:atsvcapi has no test summary line (early exit of the main process?)
schedsvc:atsvcapi has unaccounted for skip messages

=== w1064_adm (64 bit report) ===

Report validation errors:
schedsvc:atsvcapi has no test summary line (early exit of the main process?)
schedsvc:atsvcapi has unaccounted for skip messages

=== w1064_tsign (64 bit report) ===

Report validation errors:
schedsvc:atsvcapi has no test summary line (early exit of the main process?)
schedsvc:atsvcapi has unaccounted for skip messages

=== w10pro64 (64 bit report) ===

Report validation errors:
schedsvc:atsvcapi has no test summary line (early exit of the main process?)
schedsvc:atsvcapi has unaccounted for skip messages

=== w10pro64_en_AE_u8 (64 bit report) ===

Report validation errors:
schedsvc:atsvcapi has no test summary line (early exit of the main process?)
schedsvc:atsvcapi has unaccounted for skip messages

=== w10pro64_ar (64 bit report) ===

Report validation errors:
schedsvc:atsvcapi has no test summary line (early exit of the main process?)
schedsvc:atsvcapi has unaccounted for skip messages

=== w10pro64_ja (64 bit report) ===

Report validation errors:
schedsvc:atsvcapi has no test summary line (early exit of the main process?)
schedsvc:atsvcapi has unaccounted for skip messages

=== w10pro64_zh_CN (64 bit report) ===

Report validation errors:
schedsvc:atsvcapi has no test summary line (early exit of the main process?)
schedsvc:atsvcapi has unaccounted for skip messages

=== debian11 (32 bit report) ===

Report validation errors:
d3d8:device has no test summary line (early exit of the main process?)
d3d8:device has unaccounted for failure messages
d3d8:device has unaccounted for todo messages
d3d8:device has unaccounted for skip messages
dinput:mouse has no test summary line (early exit of the main process?)
mmdevapi:render has no test summary line (early exit of the main process?)
mmdevapi:render has unaccounted for skip messages

=== debian11 (32 bit Arabic:Morocco report) ===

Report validation errors:
d3d8:device has no test summary line (early exit of the main process?)
d3d8:device has unaccounted for failure messages
d3d8:device has unaccounted for todo messages
d3d8:device has unaccounted for skip messages
dinput:mouse has no test summary line (early exit of the main process?)
mmdevapi:render has no test summary line (early exit of the main process?)
mmdevapi:render has unaccounted for skip messages

=== debian11 (32 bit German report) ===

Report validation errors:
d3d8:device has no test summary line (early exit of the main process?)
d3d8:device has unaccounted for failure messages
d3d8:device has unaccounted for todo messages
d3d8:device has unaccounted for skip messages
dinput:mouse has no test summary line (early exit of the main process?)
mmdevapi:render has no test summary line (early exit of the main process?)
mmdevapi:render has unaccounted for skip messages

=== debian11 (32 bit French report) ===

mmdevapi:
render.c:1114: Test failed: Position 54248 too far after playing 100ms

Report validation errors:
d3d8:device has no test summary line (early exit of the main process?)
d3d8:device has unaccounted for failure messages
d3d8:device has unaccounted for todo messages
d3d8:device has unaccounted for skip messages
dinput:mouse has no test summary line (early exit of the main process?)
mmdevapi:render has no test summary line (early exit of the main process?)
mmdevapi:render has unaccounted for failure messages
mmdevapi:render has unaccounted for skip messages

=== debian11 (32 bit Hebrew:Israel report) ===

Report validation errors:
d3d8:device has no test summary line (early exit of the main process?)
d3d8:device has unaccounted for failure messages
d3d8:device has unaccounted for todo messages
d3d8:device has unaccounted for skip messages
dinput:mouse has no test summary line (early exit of the main process?)
mmdevapi:render has no test summary line (early exit of the main process?)
mmdevapi:render has unaccounted for skip messages

=== debian11 (32 bit Hindi:India report) ===

Report validation errors:
d3d8:device has no test summary line (early exit of the main process?)
d3d8:device has unaccounted for failure messages
d3d8:device has unaccounted for todo messages
d3d8:device has unaccounted for skip messages
dinput:mouse has no test summary line (early exit of the main process?)
mmdevapi:render has no test summary line (early exit of the main process?)
mmdevapi:render has unaccounted for skip messages

=== debian11 (32 bit Japanese:Japan report) ===

Report validation errors:
d3d8:device has no test summary line (early exit of the main process?)
d3d8:device has unaccounted for failure messages
d3d8:device has unaccounted for todo messages
d3d8:device has unaccounted for skip messages
dinput:mouse has no test summary line (early exit of the main process?)
mmdevapi:render has no test summary line (early exit of the main process?)
mmdevapi:render has unaccounted for skip messages

=== debian11 (32 bit Chinese:China report) ===

Report validation errors:
d3d8:device has no test summary line (early exit of the main process?)
d3d8:device has unaccounted for failure messages
d3d8:device has unaccounted for todo messages
d3d8:device has unaccounted for skip messages
dinput:mouse has no test summary line (early exit of the main process?)
mmdevapi:render has no test summary line (early exit of the main process?)
mmdevapi:render has unaccounted for skip messages

=== debian11 (32 bit WoW report) ===

Report validation errors:
d3d8:device has no test summary line (early exit of the main process?)
d3d8:device has unaccounted for failure messages
d3d8:device has unaccounted for todo messages
d3d8:device has unaccounted for skip messages
dinput:mouse has no test summary line (early exit of the main process?)
mmdevapi:render has no test summary line (early exit of the main process?)
mmdevapi:render has unaccounted for skip messages

=== debian11 (64 bit WoW report) ===

Report validation errors:
d3d8:device has no test summary line (early exit of the main process?)
d3d8:device has unaccounted for failure messages
d3d8:device has unaccounted for todo messages
d3d8:device has unaccounted for skip messages
dinput:mouse has no test summary line (early exit of the main process?)
mmdevapi:render has no test summary line (early exit of the main process?)
mmdevapi:render has unaccounted for skip messages