[Bug 29886] New: Microsoft Visual Studio 2005: "attach to process" crashes IDE (marshalling/unmarshalling of GUID struct -> VT_CARRAY type)

wine-bugs at winehq.org wine-bugs at winehq.org
Mon Feb 13 16:59:25 CST 2012 

http://bugs.winehq.org/show_bug.cgi?id=29886

             Bug #: 29886
           Summary: Microsoft Visual Studio 2005: "attach to process"
                    crashes IDE (marshalling/unmarshalling of GUID struct
                    -> VT_CARRAY type)
           Product: Wine
           Version: 1.4-rc3
          Platform: x86
        OS/Version: Linux
            Status: NEW
          Severity: normal
          Priority: P2
         Component: oleaut32
        AssignedTo: wine-bugs at winehq.org
        ReportedBy: focht at gmx.net
    Classification: Unclassified


Hello,

while verifying some Visual Studio 2005 bugs I found another issue.

Clicking menu item "Tools" -> "Attach to Process" crashes the IDE.

--- snip ---
$ pwd
/home/focht/.wine/drive_c/Program Files/Microsoft Visual Studio 8/Common7/IDE
...
$ WINEDEBUG=+tid,+seh,+loaddll,+variant,+ole,+olerelay,+relay wine ./devenv.exe
--- snip ---

--- snip ---
...
0097:trace:olerelay:xCall strModule=(tdesc.vt VT_BSTR)
0097:trace:olerelay:serialize_param C:\\Program Files\\Microsoft Visual Studio
8\\SmartDevices\\Debugger\\bin\\eps.dll(100002,0,0x32e86c) => 0x12aafbc
0097:trace:ole:BSTR_UserSize string=L"C:\\Program Files\\Microsoft Visual
Studio 8\\SmartDevices\\Debugger\\bin\\eps.dll"
0097:trace:ole:BSTR_UserSize returning 166
0097:Call ntdll.RtlAllocateHeap(00110000,00000008,000000a6) ret=7e72cb31
0097:Ret  ntdll.RtlAllocateHeap() retval=012ad748 ret=7e72cb31
0097:trace:ole:BSTR_UserMarshal (100002,0x12ad748,0x32e86c) => 0x12aafbc
0097:trace:ole:BSTR_UserMarshal string=L"C:\\Program Files\\Microsoft Visual
Studio 8\\SmartDevices\\Debugger\\bin\\eps.dll"
0097:trace:olerelay:xCall ,rclsid=(tdesc.vt VT_PTR) 
...
0097:trace:ole:serialize_param (tdesc.vt VT_USERDEFINED)
0097:trace:ole:ITypeInfo_fnGetRefTypeInfo typeinfo in imported typelib that is
already loaded
0097:trace:ole:ITypeLib2_fnAddRef (0x153c90)->ref was 3
0097:trace:ole:ITypeLib2_fnGetTypeInfo 0x153c90 0 0x32e3d4
0097:trace:ole:ITypeInfo_fnAddRef (0x1540d0)->ref is 1
0097:trace:ole:ITypeLib2_fnAddRef (0x153c90)->ref was 4
0097:trace:ole:ITypeLib2_fnRelease (0x153c90)->(4)
0097:trace:ole:ITypeInfo_fnGetRefTypeInfo (0x12ab998) hreftype 0x000d loaded
SUCCESS (0x1540d0)
0097:trace:ole:ITypeInfo_fnGetTypeAttr (0x1540d0)
0097:Call ntdll.RtlAllocateHeap(00110000,00000000,0000004c) ret=7e739575
0097:Ret  ntdll.RtlAllocateHeap() retval=012abf90 ret=7e739575
0097:trace:olerelay:serialize_param {(0x1540d0) index 0
0097:Call ntdll.RtlAllocateHeap(00110000,00000000,0000002a) ret=7e71a1de
0097:Ret  ntdll.RtlAllocateHeap() retval=012acae8 ret=7e71a1de
0097:trace:ole:serialize_param (tdesc.vt VT_UI4)
0097:trace:olerelay:serialize_param 2d32aa54
0097:trace:ole:ITypeInfo_fnReleaseVarDesc (0x1540d0)->(0x12acaec)
0097:Call ntdll.RtlFreeHeap(00110000,00000000,012acae8) ret=7e719fbd
0097:Ret  ntdll.RtlFreeHeap() retval=00000001 ret=7e719fbd
0097:trace:olerelay:serialize_param ,(0x1540d0) index 1
0097:Call ntdll.RtlAllocateHeap(00110000,00000000,0000002a) ret=7e71a1de
0097:Ret  ntdll.RtlAllocateHeap() retval=012acae8 ret=7e71a1de
0097:trace:ole:serialize_param (tdesc.vt VT_UI2)
0097:trace:olerelay:serialize_param 1f84
0097:trace:ole:ITypeInfo_fnReleaseVarDesc (0x1540d0)->(0x12acaec)
0097:Call ntdll.RtlFreeHeap(00110000,00000000,012acae8) ret=7e719fbd
0097:Ret  ntdll.RtlFreeHeap() retval=00000001 ret=7e719fbd
0097:trace:olerelay:serialize_param ,(0x1540d0) index 2
0097:Call ntdll.RtlAllocateHeap(00110000,00000000,0000002a) ret=7e71a1de
0097:Ret  ntdll.RtlAllocateHeap() retval=012acae8 ret=7e71a1de
0097:trace:ole:serialize_param (tdesc.vt VT_UI2)
0097:trace:olerelay:serialize_param 4964
0097:trace:ole:ITypeInfo_fnReleaseVarDesc (0x1540d0)->(0x12acaec)
0097:Call ntdll.RtlFreeHeap(00110000,00000000,012acae8) ret=7e719fbd
0097:Ret  ntdll.RtlFreeHeap() retval=00000001 ret=7e719fbd
0097:trace:olerelay:serialize_param ,(0x1540d0) index 3
0097:Call ntdll.RtlAllocateHeap(00110000,00000000,0000003e) ret=7e71a1de
0097:Ret  ntdll.RtlAllocateHeap() retval=012abb20 ret=7e71a1de
0097:trace:ole:serialize_param (tdesc.vt VT_CARRAY)
0097:trace:olerelay:serialize_param carr[8](vt VT_UI1)[(tdesc.vt VT_UI1)
0097:trace:seh:raise_exception code=c0000005 flags=0 addr=0x7e72eb06
ip=7e72eb06 tid=0097
0097:trace:seh:raise_exception  info[0]=00000000
0097:trace:seh:raise_exception  info[1]=b8ec13bc
0097:trace:seh:raise_exception  eax=b8ec13bc ebx=7e816d7c ecx=00000000
edx=7e72ead5 esi=7e74438d edi=0032e714
0097:trace:seh:raise_exception  ebp=0032e208 esp=0032e080 cs=0023 ds=002b
es=002b fs=0063 gs=006b flags=00010202
0097:trace:seh:call_vectored_handlers calling handler at 0x406b98 code=c0000005
flags=0 
--- snip ---

It seems there is a GUID struct being marshalled/serialized.

VT_UI4, VT_UI2, VT_UI2, VT_CARRAY (8 x VT_UI1)

Indeed, going back with some olerelay:serialize_param values one can find:

--- snip ---
0097:Call advapi32.RegOpenKeyExA(80000002,012aad28
"Software\\Microsoft\\VisualStudio\\8.0\\CLSID\\{2D32AA54-1F84-4964-BC13-ECB871943797}",00000000,00020019,0032e868)
ret=54bbc0f4 
--- snip ---

Code:
http://source.winehq.org/git/wine.git/blob/74a3d9ee5eff36b6fa4283cbc29b9cd13d4cb09a:/dlls/oleaut32/tmarshal.c#l883

--- snip ---
 883     case VT_CARRAY: {
 884         ARRAYDESC *adesc = tdesc->u.lpadesc;
 885         int i, arrsize = 1;
 886 
 887         if (debugout) TRACE_(olerelay)("carr");
 888         for (i=0;i<adesc->cDims;i++) {
 889             if (debugout)
TRACE_(olerelay)("[%d]",adesc->rgbounds[i].cElements);
 890             arrsize *= adesc->rgbounds[i].cElements;
 891         }
 892         if (debugout) TRACE_(olerelay)("(vt
%s)",debugstr_vt(adesc->tdescElem.vt));
 893         if (debugout) TRACE_(olerelay)("[");
 894         for (i=0;i<arrsize;i++) {
 895             hres = serialize_param(tinfo, writeit, debugout, dealloc,
&adesc->tdescElem, (DWORD*)((LPBYTE)(*arg)+i*_xsize(&adesc->tdescElem, tinfo)),
buf);
 896             if (hres)
 897                 return hres;
 898             if (debugout && (i<arrsize-1)) TRACE_(olerelay)(",");
 899         }
 900         if (debugout) TRACE_(olerelay)("]");
 901         if (dealloc)
 902             HeapFree(GetProcessHeap(), 0, *(void **)arg);
 903         return S_OK;
 904     }
--- snip ---

Line 895: serialize_param(tinfo, writeit, debugout, dealloc, &adesc->tdescElem,
(DWORD*)((LPBYTE)(*arg)+i*_xsize(&adesc->tdescElem, tinfo)),

"arg" is already the address of 8-byte buffer here (GUID -xxxxxxxx part), hence
dereferencing causes harm.

With that part fixed, the GUID "{2D32AA54-1F84-4964-BC13-ECB871943797}" is
properly serialized:

--- snip ---
...
0039:trace:olerelay:serialize_param {(0x153f80) index 0
0039:Call ntdll.RtlAllocateHeap(00110000,00000000,0000002a) ret=7e71a1de
0039:Ret  ntdll.RtlAllocateHeap() retval=012abad0 ret=7e71a1de
0039:trace:ole:serialize_param (tdesc.vt VT_UI4)
0039:trace:olerelay:serialize_param 2d32aa54
0039:trace:ole:ITypeInfo_fnReleaseVarDesc (0x153f80)->(0x12abad4)
0039:Call ntdll.RtlFreeHeap(00110000,00000000,012abad0) ret=7e719fbd
0039:Ret  ntdll.RtlFreeHeap() retval=00000001 ret=7e719fbd
0039:trace:olerelay:serialize_param ,(0x153f80) index 1
0039:Call ntdll.RtlAllocateHeap(00110000,00000000,0000002a) ret=7e71a1de
0039:Ret  ntdll.RtlAllocateHeap() retval=012abad0 ret=7e71a1de
0039:trace:ole:serialize_param (tdesc.vt VT_UI2)
0039:trace:olerelay:serialize_param 1f84
0039:trace:ole:ITypeInfo_fnReleaseVarDesc (0x153f80)->(0x12abad4)
0039:Call ntdll.RtlFreeHeap(00110000,00000000,012abad0) ret=7e719fbd
0039:Ret  ntdll.RtlFreeHeap() retval=00000001 ret=7e719fbd
0039:trace:olerelay:serialize_param ,(0x153f80) index 2
0039:Call ntdll.RtlAllocateHeap(00110000,00000000,0000002a) ret=7e71a1de
0039:Ret  ntdll.RtlAllocateHeap() retval=012abad0 ret=7e71a1de
0039:trace:ole:serialize_param (tdesc.vt VT_UI2)
0039:trace:olerelay:serialize_param 4964
0039:trace:ole:ITypeInfo_fnReleaseVarDesc (0x153f80)->(0x12abad4)
0039:Call ntdll.RtlFreeHeap(00110000,00000000,012abad0) ret=7e719fbd
0039:Ret  ntdll.RtlFreeHeap() retval=00000001 ret=7e719fbd
0039:trace:olerelay:serialize_param ,(0x153f80) index 3
0039:Call ntdll.RtlAllocateHeap(00110000,00000000,0000003e) ret=7e71a1de
0039:Ret  ntdll.RtlAllocateHeap() retval=012ac200 ret=7e71a1de
0039:trace:ole:serialize_param (tdesc.vt VT_CARRAY)
0039:trace:olerelay:serialize_param carr[8](vt VT_UI1)[(tdesc.vt VT_UI1)
0039:trace:olerelay:serialize_param bc
0039:trace:olerelay:serialize_param ,(tdesc.vt VT_UI1)
0039:trace:olerelay:serialize_param 13
0039:trace:olerelay:serialize_param ,(tdesc.vt VT_UI1)
0039:trace:olerelay:serialize_param ec
0039:trace:olerelay:serialize_param ,(tdesc.vt VT_UI1)
0039:trace:olerelay:serialize_param b8
0039:trace:olerelay:serialize_param ,(tdesc.vt VT_UI1)
0039:trace:olerelay:serialize_param 71
0039:trace:olerelay:serialize_param ,(tdesc.vt VT_UI1)
0039:trace:olerelay:serialize_param 94
0039:trace:olerelay:serialize_param ,(tdesc.vt VT_UI1)
0039:trace:olerelay:serialize_param 37
0039:trace:olerelay:serialize_param ,(tdesc.vt VT_UI1)
0039:trace:olerelay:serialize_param 97
0039:trace:olerelay:serialize_param ](0x153f80)->(0x12ac204)
0039:Call ntdll.RtlFreeHeap(00110000,00000000,012ac200) ret=7e719fbd
0039:Ret  ntdll.RtlFreeHeap() retval=00000001 ret=7e719fbd
0039:trace:olerelay:serialize_param }(0x153f80)->(0x12ab710) 
...
--- snip ---

Unmarshalling in TMStubImpl_Invoke/proxy (had to turn on "debugout" flag
manually):

--- snip ---
...
0020:trace:ole:BSTR_UserUnmarshal string=L"C:\\Program Files\\Microsoft Visual
Studio 8\\SmartDevices\\Debugger\\bin\\eps.dll"
0020:trace:olerelay:deserialize_param L"C:\\Program Files\\Microsoft Visual
Studio 8\\SmartDevices\\Debugger\\bin\\eps.dll"vt VT_PTR at 0x129b690 
...
0020:trace:ole:deserialize_param vt VT_USERDEFINED at 0x129b750
0020:trace:ole:ITypeInfo_fnGetRefTypeInfo typeinfo in imported typelib that is
already loaded
0020:trace:ole:ITypeLib2_fnAddRef (0x153b40)->ref was 3
0020:trace:ole:ITypeLib2_fnGetTypeInfo 0x153b40 0 0xddbe1e0
0020:trace:ole:ITypeInfo_fnAddRef (0x153f80)->ref is 1
0020:trace:ole:ITypeLib2_fnAddRef (0x153b40)->ref was 4
0020:trace:ole:ITypeLib2_fnRelease (0x153b40)->(4)
0020:trace:ole:ITypeInfo_fnGetRefTypeInfo (0x129b590) hreftype 0x000d loaded
SUCCESS (0x153f80)
0020:trace:ole:ITypeInfo_fnGetTypeAttr (0x153f80)
0020:Call ntdll.RtlAllocateHeap(00110000,00000000,0000004c) ret=7e739571
0020:Ret  ntdll.RtlAllocateHeap() retval=0129b768 ret=7e739571
0020:trace:olerelay:deserialize_param {(0x153f80) index 0
0020:Call ntdll.RtlAllocateHeap(00110000,00000000,0000002a) ret=7e71a1de
0020:Ret  ntdll.RtlAllocateHeap() retval=0129b7c0 ret=7e71a1de
0020:trace:ole:deserialize_param vt VT_UI4 at 0x129b750
0020:trace:olerelay:deserialize_param 2d32aa54(0x153f80)->(0x129b7c4)
0020:Call ntdll.RtlFreeHeap(00110000,00000000,0129b7c0) ret=7e719fbd
0020:Ret  ntdll.RtlFreeHeap() retval=00000001 ret=7e719fbd
0020:trace:olerelay:deserialize_param ,(0x153f80) index 1
0020:Call ntdll.RtlAllocateHeap(00110000,00000000,0000002a) ret=7e71a1de
0020:Ret  ntdll.RtlAllocateHeap() retval=0129b7c0 ret=7e71a1de
0020:trace:ole:deserialize_param vt VT_UI2 at 0x129b754
0020:trace:olerelay:deserialize_param 1f84(0x153f80)->(0x129b7c4)
0020:Call ntdll.RtlFreeHeap(00110000,00000000,0129b7c0) ret=7e719fbd
0020:Ret  ntdll.RtlFreeHeap() retval=00000001 ret=7e719fbd
0020:trace:olerelay:deserialize_param ,(0x153f80) index 2
0020:Call ntdll.RtlAllocateHeap(00110000,00000000,0000002a) ret=7e71a1de
0020:Ret  ntdll.RtlAllocateHeap() retval=0129b7c0 ret=7e71a1de
0020:trace:ole:deserialize_param vt VT_UI2 at 0x129b756
0020:trace:olerelay:deserialize_param 4964(0x153f80)->(0x129b7c4)
0020:Call ntdll.RtlFreeHeap(00110000,00000000,0129b7c0) ret=7e719fbd
0020:Ret  ntdll.RtlFreeHeap() retval=00000001 ret=7e719fbd
0020:trace:olerelay:deserialize_param ,(0x153f80) index 3
0020:Call ntdll.RtlAllocateHeap(00110000,00000000,0000003e) ret=7e71a1de
0020:Ret  ntdll.RtlAllocateHeap() retval=0129b7c0 ret=7e71a1de
0020:trace:ole:deserialize_param vt VT_CARRAY at 0x129b758
0020:Call ntdll.RtlAllocateHeap(00110000,00000008,00000008) ret=7e7312f0
0020:Ret  ntdll.RtlAllocateHeap() retval=0129b808 ret=7e7312f0
0020:trace:ole:deserialize_param vt VT_UI1 at 0x129b808
0020:trace:olerelay:deserialize_param bcvt VT_UI1 at 0x129b809
0020:trace:olerelay:deserialize_param 13vt VT_UI1 at 0x129b80a
0020:trace:olerelay:deserialize_param ecvt VT_UI1 at 0x129b80b
0020:trace:olerelay:deserialize_param b8vt VT_UI1 at 0x129b80c
0020:trace:olerelay:deserialize_param 71vt VT_UI1 at 0x129b80d
0020:trace:olerelay:deserialize_param 94vt VT_UI1 at 0x129b80e
0020:trace:olerelay:deserialize_param 37vt VT_UI1 at 0x129b80f
0020:trace:olerelay:deserialize_param 97(0x153f80)->(0x129b7c4)
0020:Call ntdll.RtlFreeHeap(00110000,00000000,0129b7c0) ret=7e719fbd
0020:Ret  ntdll.RtlFreeHeap() retval=00000001 ret=7e719fbd
0020:trace:olerelay:deserialize_param }(0x153f80)->(0x129b768)
...
--- snip ---

There is another crash after _invoke in TMStubImpl_Invoke/proxy, when
marshalling back the parameters.
Serialize_param() now encodes a pointer value in first 4 bytes of 8-byte array
of last GUID part.
In this case deserialize_param() is the root cause.

http://source.winehq.org/git/wine.git/blob/74a3d9ee5eff36b6fa4283cbc29b9cd13d4cb09a:/dlls/oleaut32/tmarshal.c#l1173

--- snip ---
1173         case VT_CARRAY: {
1174             /* arg is pointing to the start of the array. */
1175             ARRAYDESC *adesc = tdesc->u.lpadesc;
1176             int             arrsize,i;
1177             arrsize = 1;
1178             if (adesc->cDims > 1) FIXME("cDims > 1 in VT_CARRAY. Does it
work?\n");
1179             for (i=0;i<adesc->cDims;i++)
1180                 arrsize *= adesc->rgbounds[i].cElements;
1181            
*arg=(DWORD)HeapAlloc(GetProcessHeap(),HEAP_ZERO_MEMORY,_xsize(tdesc->u.lptdesc,
tinfo) * arrsize);
1182             for (i=0;i<arrsize;i++)
1183                 deserialize_param(
1184                     tinfo,
1185                     readit,
1186                     debugout,
1187                     alloc,
1188                     &adesc->tdescElem,
1189                     (DWORD*)((LPBYTE)(*arg)+i*_xsize(&adesc->tdescElem,
tinfo)),
1190                     buf
1191                 );
1192             return S_OK;
1193         }
--- snip ---

Looking at the history it seems this is an old regression.

http://source.winehq.org/git/wine.git/commitdiff/b8d7088e88d7c077c0c4ad1b2c4d7f3503e2806a

--- snip ---
commit b8d7088e88d7c077c0c4ad1b2c4d7f3503e2806a
Author: Jeremy White <jwhite at codeweavers.com>
Date:   Sat Oct 24 17:29:02 2009 -0500

    oleaut32: Implement the ability to marshall VT_CARRAY's of user defined
types.
--- snip ---

Reverting the first part of commit ("dlls/oleaut32/tmarshal.c") prevents the
crash and lets the IDE show the "attach to process" dialog with choice of
various remote debugger backends.

$ wine --version
wine-1.4-rc3

Regards

-- 
Configure bugmail: http://bugs.winehq.org/userprefs.cgi?tab=email
Do not reply to this email, post in Bugzilla using the
above URL to reply.
------- You are receiving this mail because: -------
You are watching all bug changes.

More information about the wine-bugs mailing list