comdlg32/tests: Use BOOL type where appropriate

[Marvin]

Hi,

While running your changed tests on Windows, I think I found new failures.
Being a bot and all I'm not very good at pattern recognition, so I might be
wrong, but could you please double-check?
Full results can be found at
https://newtestbot.winehq.org/JobDetails.pl?Key=2943

Your paranoid android.


=== w2008s64 (64 bit filedlg) ===
No test summary line found