Path manipulation issue in Python - likely broken API in Wine

[Zhiyi Zhang]

Hi Sebastian,

Thanks for reporting. I opened a bug 47787 for this. I will take a look when I have time.
If any one wants to try fixing this, feel free to do so and assign the bug to themselves.
It's probably using some older path functions. With cpython source code available,
it should be rather easy to debug this.

Thanks,
Zhiyi

On 9/20/19 11:57 PM, Sebastian M. Ernst wrote:
> Damjan, Zhiyi, Jeff, all,
>
> thanks for looking into this and the quick patch. I very much appreciate it.
>
> I found one more odd thing and it might be closely related (if not the
> same bug). I tested different versions of CPython for Windows on Wine
> 4.14 and noticed that pytest (a Python test library) was somehow broken
> on CPython 3.5 and 3.6 but not on CPython 3.7. It can also be traced
> back to path manipulation:
>
> # CPython 3.5.4
> user at comp:/path/to/target> wine python.exe -c "from pathlib import Path;
> print(Path('.').resolve())"
> Traceback (most recent call last):
>   File "<string>", line 1, in <module>
>   File "pathlib.py", line 1109, in resolve
>   File "pathlib.py", line 186, in resolve
> FileNotFoundError: [WinError 2] Datei nicht gefunden: '.'
>
> # CPython 3.6.3
> user at comp:/path/to/target> wine python.exe -c "from pathlib import Path;
> print(Path('.').resolve())"
> .
>
> # CPython 3.7.4
> user at comp:/path/to/target> wine python.exe -c "from pathlib import Path;
> print(Path('.').resolve())"
> Z:\path\to\target
>
> CPython 3.7 is correctly resolving the path '.' while CPython 3.5 and
> 3.6 fail in different ways. All three examples work correctly on Windows
> 10. "Datei nicht gefunden" translates to "file not found".
>
> Best regards,
> Sebastian
>
>
> Am 15.09.19 um 17:51 schrieb Zhiyi Zhang:
>> I will take a look at this. https://bugs.winehq.org/show_bug.cgi?id=47766
>>
>> On 9/15/19 5:21 PM, Damjan Jovanovic wrote:
>>> Confirming that it's dlls/kernelbase PathAllocCanonicalize() that breaks it.
>>>
>>> I can't debug this further at the moment. Please feel free to continue.
>>>
>>>
>>> On Sun, Sep 15, 2019 at 11:10 AM Damjan Jovanovic <damjan.jov at gmail.com>
>>> wrote:
>>>
>>>> Confirming with Python 3.7.4 on FreeBSD.
>>>>
>>>> You don't need PYTHONHOME. It is sufficient to symlink any directory in
>>>> the path leading up to Python's installation directory into a "."-prefix
>>>> link, and to run python.exe with CWD coming through that link:
>>>>
>>>> (With CWD = ~/.wine/drive_c/users/me/Local Settings/Application
>>>> Data/Programs/.py/Python37-32)
>>>> wine python.exe -c "import sys; print(sys.executable)"
>>>> C:\users\me\Local Settings\Application
>>>> Data\Programspy\Python37-32\python.exe
>>>>
>>>> It breaks the same way if a PYTHONHOME that doesn't go through .py is set.
>>>>
>>>> I think it breaks in dlls/kernelbase PathAllocCanonicalize(). Testing.
>>>>
>>>> Regards
>>>> Damjan
>>>>
>>>>
>>>> On Sun, Sep 15, 2019 at 10:25 AM Sebastian M. Ernst <ernst at pleiszenburg.de>
>>>> wrote:
>>>>
>>>>> Hi all,
>>>>>
>>>>> I am working with CPython on top of Wine 4.14. I know for certain that
>>>>> some path manipulation logic in Wine is broken but I am having trouble
>>>>> tracking down the root cause.
>>>>>
>>>>> I am launching Python like this:
>>>>>
>>>>> user at comp:/path/other> WINEDEBUG=-all PYTHONHOME="z:\\path\\.to\\target"
>>>>> wine /path/.to/target/python.exe
>>>>>
>>>>> Notice that the environment variable PYTHONHOME contains a folder with a
>>>>> leading dot (`.to`). Python should pick it up and expose it as the path
>>>>> to its executable - like this:
>>>>>
>>>>> user at comp:/path/other> WINEDEBUG=-all PYTHONHOME="z:\\path\\.to\\target"
>>>>> wine /path/.to/target/python.exe -c "import sys; print(sys.executable)"
>>>>> Z:\pathto\target\python.exe
>>>>>
>>>>> Notice that `\.` is missing from the path that is being printed.
>>>>>
>>>>> It gets even weirder if the folder name starts with two leading dots,
>>>>> i.e. `..to` - like this:
>>>>>
>>>>> user at comp:/path/other> WINEDEBUG=-all
>>>>> PYTHONHOME="z:\\path\\..to\\target" wine /path/..to/target/python.exe -c
>>>>> "import sys; print(sys.executable)"
>>>>> Z:\to\target\python.exe
>>>>>
>>>>> Now an even longer segment is missing: `path\..`
>>>>>
>>>>> I confirmed that the above steps work correctly on Windows 10, so it is
>>>>> very likely a bug in Wine. I collected all necessary steps required for
>>>>> reproducing this bug below this email.
>>>>>
>>>>> The startup code of CPython is somewhat convoluted, which is why I am
>>>>> having trouble identifying the Wine/Windows API in question. Any help is
>>>>> greatly appreciated.
>>>>>
>>>>> Best regards,
>>>>> Sebastian
>>>>>
>>>>>
>>>>> ---
>>>>>
>>>>>
>>>>> user at comp:/path/.to/target> wget
>>>>> https://www.python.org/ftp/python/3.7.4/python-3.7.4-embed-win32.zip 2>
>>>>> /dev/null
>>>>> user at comp:/path/.to/target> unzip python-3.7.4-embed-win32.zip >
>>>>> /dev/null; rm python-3.7.4-embed-win32.zip
>>>>> user at comp:/path/.to/target> rm python37._pth
>>>>> user at comp:/path/.to/target> mkdir Lib; mv python37.zip Lib/; cd Lib;
>>>>> unzip python37.zip > /dev/null; rm python37.zip; cd ..
>>>>> user at comp:/path/.to/target> cd /path/other/
>>>>> user at comp:/path/other> WINEDEBUG=-all PYTHONHOME="z:\\path\\.to\\target"
>>>>> wine /path/.to/target/python.exe
>>>>> Python 3.7.4 (tags/v3.7.4:e09359112e, Jul  8 2019, 19:29:22) [MSC v.1916
>>>>> 32 bit (Intel)] on win32
>>>>> Type "help", "copyright", "credits" or "license" for more information.
>>>>> Traceback (most recent call last):
>>>>>   File "/etc/pythonstart", line 7, in <module>
>>>>>     import readline
>>>>> ModuleNotFoundError: No module named 'readline'
>>>>>>>> exit()
>>>>> user at comp:/path/other> WINEDEBUG=-all PYTHONHOME="z:\\path\\.to\\target"
>>>>> wine /path/.to/target/python.exe -c "import sys; print(sys.executable)"
>>>>> Z:\pathto\target\python.exe
>>>>>
>>>>>