dbghelp detached from wine into standalone windows dll...

Sat Jun 1 10:24:59 CDT 2019

Hi !
I was looking for here and there for portable (windows & linux) call stack determination functionality - but could not find any.
I have some experience with call stack determination, but only on windows - made memory leak detection utility for windows once upon a time.
But all API's with which I have played around are windows based. Now wanted to go deeper with call stack determination and moreover make functionality portable - to start from windows & linux, later on expanding to android. 
Brief analysis showed a lot of complexity without any clear / good / portable library.
Analysis came upon dbghelp.dll which is implemented in Wine - I have noticed that besides windows PE file format it contains also support for loading debug information for DWARF / ELF file format. I've got curious, and started to prototype.
I've made windows project where I have dragged some parts of wine functionality. (LGPL licensed)
Something similar was done once in here: https://github.com/vtjnash/dbghelp2
but he have used cross compilation toolchain, I wanted to use Visual studio + microsoft compiler.
My own repository is located here:https://github.com/tapika/dbghelp2

My prime target is 64-bit windows, 32-bit is nice-to-have.
Well - dll itself is nice, but I needed test application. Quick analysis came upon https://github.com/KjellKod/g3log that logger was using dbghelp - official microsoft distribution.
After some fighting with compilation and linking errors I've managed to detach dbghelp from wine into separate .dll, I've called it dbghelp2.dll.
But at run-time library refused to work. 
I'll try to list here problems upon which I've came accross:
32-bit compilation:
A lot of unsupported traces in base functionality:
976c:fixme:dbghelp_msc:c:\prototyping\dbghelp2\dlls\dbghelp\msc.c 725: Unsupported type 1404 in ENUM field list976c:fixme:dbghelp_msc:c:\prototyping\dbghelp2\dlls\dbghelp\msc.c 2038: Unsupported symbol id 1124976c:fixme:dbghelp_msc:c:\prototyping\dbghelp2\dlls\dbghelp\msc.c 123: 00000000: 06 00 24 11 73 74 64 00                          ..$.std.        976c:fixme:dbghelp_msc:c:\prototyping\dbghelp2\dlls\dbghelp\msc.c 2038: Unsupported symbol id 1124976c:fixme:dbghelp_msc:c:\prototyping\dbghelp2\dlls\dbghelp\msc.c 123: 00000000: 1a 00 24 11 5f 48 61 73 5f 41 44 4c 5f 73 77 61  ..$._Has_ADL_swa976c:fixme:dbghelp_msc:c:\prototyping\dbghelp2\dlls\dbghelp\msc.c 123: 00000010: 70 5f 64 65 74 61 69 6c 00 00 00 00              p_detail....    976c:fixme:dbghelp_msc:c:\prototyping\dbghelp2\dlls\dbghelp\msc.c 2038: Unsupported symbol id 1124976c:fixme:dbghelp_msc:c:\prototyping\dbghelp2\dlls\dbghelp\msc.c 123: 00000000: 0a 00 24 11 72 65 6c 5f 6f 70 73 00              ..$.rel_ops.    ...
same problems occurs in 64-bit compilation.
Finally call stack determination refused to work:976c:fixme:dbghelp:c:\prototyping\dbghelp2\dlls\dbghelp\stack.c 59: Failed to linearize address 771f:7000 (mode 0)
64-bit compilation:
I've noticed that addressing uses 32-bit unsigned long, while 64-bit addressing mode requires uint64_t. Besides this g3log also was using incorrect API's - I've ended up replacing SymGetSymFromAddr64 with SymFromAddr.
call stack determination seems to work, also function resolving, but not source code position or line information, what Microsoft dbghelp.dll can do out of box.
So now dbghelp2.dll can determine stack like this:
******* STACKDUMP *******stack using dbghelp2.dll
stack dump [0]  std::basic_string<char,std::char_traits<char>,std::allocator<char> >::appendstack dump [1]  std::basic_string<char,std::char_traits<char>,std::allocator<char> >::appendstack dump [2]  example_fatal::tryToKillWithAccessingIllegalPointerstack dump [3]  mainstack dump [4]  invoke_mainstack dump [5]  __scrt_common_main_sehstack dump [6]  __scrt_common_mainstack dump [7]  mainCRTStartupstack dump [8] stack dump [9] 
while Microsoft's implementation determines it like this:stack dump [0] c:\program files (x86)\microsoft visual studio\2017\enterprise\vc\tools\msvc\14.16.27023\include\xstring L: 2571 std::basic_string<char,std::char_traits<char>,std::allocator<char> >::appendstack dump [1] c:\program files (x86)\microsoft visual studio\2017\enterprise\vc\tools\msvc\14.16.27023\include\xstring L: 2593 std::basic_string<char,std::char_traits<char>,std::allocator<char> >::appendstack dump [2] c:\prototyping\dbghelp2\g3log\example\main_sigsegv.cpp L: 52 example_fatal::tryToKillWithAccessingIllegalPointerstack dump [3] c:\prototyping\dbghelp2\g3log\example\main_sigsegv.cpp L: 115 mainstack dump [4] d:\agent\_work\1\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl L: 79 invoke_mainstack dump [5] d:\agent\_work\1\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl L: 288 __scrt_common_main_sehstack dump [6] d:\agent\_work\1\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl L: 331 __scrt_common_mainstack dump [7] d:\agent\_work\1\s\src\vctools\crt\vcstartup\src\startup\exe_main.cpp L: 17 mainCRTStartupstack dump [8]  BaseThreadInitThunkstack dump [9]  RtlUserThreadStart
Noticed stack frames [8] & [9] contains additional functions, not available in wine port.
Here is 64-bit enabling commit:
https://github.com/tapika/dbghelp2/commit/3a3e9daacae14c4f73585aeff4ef53a207a9f55d

Basically my intention was to prototype that it's possible to determine call stack using wine dbghelp parts. 
- Does it makes any sense to try to make my dbghelp prototype be more or less official / or create such version which would be suitable for wine and for standalone compilation ?
- I would be interested in further development of dbghelp - as standalone dll / so independently from wine.
At the moment dbghelp serves like read / parse of existing file formats (PE & ELF), has anyone considered of expanding support towards generation of same file formats ?
See also CV2PDB: https://github.com/rainers/cv2pdbclang pdb generation support: http://blog.llvm.org/2017/08/llvm-on-windows-now-supports-pdb-debug.html

-- Have a nice day!     Tarmo.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.winehq.org/pipermail/wine-devel/attachments/20190601/41387c94/attachment.html>