Unable to start UTF-8 encoded executable in UTF-8 locale - one line fix included

[Alex Villací­s Lasso]

I so wanted to be the first to provide the fix to the Open File dialog 
not handling UTF-8, but Michael Jung beat me at it :-/

While trying Michael Jung's fix, I found this other issue: when wine is 
run in an UTF-8 locale, and a native PE executable exists with an UTF-8 
encoded filename (for example gatón.exe with an [U+00F3 LATIN SMALL 
LETTER O WITH ACUTE]), any attempt to run this executable will fail with 
the following message:

wine: cannot open builtin library for <your executable name>: f

This suggests that wine somehow thinks the UTF-8 encoded filename is a 
builtin library instead of an actual PE file. So I dug into 
dlls/kernel/process.c and found the following:

* Function __wine_kernel_init() calls find_exe_file() with exe_nameW set 
to the Unicode version of the exename, and expects main_exe_file to be 
set for a native exe, or NULL for a builtin.
* Function find_exe_file() is supposed to try loading the file as 
builtin, then native. For the builtin try, it calls 
open_builtin_exe_file() to test whether a builtin exists with the 
specified name, and expects the flag file_exists to be set on return.
* Function open_builtin_exe_file() does a couple of sanity checks on the 
filename, including checking that all characters of the filename are in 
the 7-bit ASCII range (0..127), then delegates the file-exists test to 
wine_dll_load_main_exe().

The problem arises because open_builtin_exe_file() fails to set the 
file_exists flag either way when a sanity check fails. The Unicode 
version of the UTF-8 encoded exename fails the 7-bit ASCII test, but the 
function leaves file_exists uninitialized. Therefore the rest of the 
code thinks that the filename references a builtin (since file_exists is 
nonzero) and then the loading fails. The one-line fix attached simply 
sets file_exists to false *before* doing anything else in 
open_builtin_exe_file(), so that a filename that fails the sanity checks 
is properly flagged as non-existing.

Changelog:
* Initialize file_exists to 0 at exe load test, prevents mistaking of 
UTF-8 encoded exenames as builtins.

Alex Villacís Lasso

-------------- next part --------------
A non-text attachment was scrubbed...
Name: wine-kernel-process_utf8-builtin-fix.patch
Type: text/x-patch
Size: 345 bytes
Desc: not available
Url : http://www.winehq.org/pipermail/wine-patches/attachments/20051124/0e17f3f1/wine-kernel-process_utf8-builtin-fix.patch