[PATCH v2] mscoree: Implement VTable fixup for x86_64 architecture.

[Paul Gofman]

On 01/22/2016 10:46 PM, Vincent Povirk wrote:
> So, I'm trying to understand which registers we need to preserve, and
> I found some pages on MSDN.
>
> register usage: https://msdn.microsoft.com/en-us/library/9z1stfyw.aspx
> parameter passing: https://msdn.microsoft.com/en-us/library/zthk2dkh.aspx
I used the same for reference
>
> I think we need to preserve all parameter registers and can freely use
> any scratch registers that are not parameter registers, and that we
> only need to worry about the 64-bit Windows ABI.
That's the same what I had in mind.
>
> MSDN seems to say that only XMM0 through XMM3 are used for parameter
> passing, and that the extra bits in YMM0 through YMM3 are scratch
> registers. I think that means we can ignore XMM4, XMM5, and AVX
> extensions.
That's not quite true for __vectorcall convention. From your first link
for XMM4,YMM4 for instance:  "Must be preserved as needed by caller;
*fifth vector-type argument when __vectorcall is used*". So to support
all the possible cases we need to preserve all of them I suppose. That's
why I added this mess with AVX detection and 2nd thunk version.
>
> The requirements may be different for the non-Windows ABI, but I hope
> we can trust the compiler to take care of that as long as we're not
> calling such a function directly.
GCC must save non-volatile regs. Just as paranoid check I observed that
in disassembly of my build. It does that, including XMM non-volatile
regs. I suppose a lot of things would be broken in Wine if ms-abi in gcc
was buggy this way.
>
> I'm not sure if we can use __attribute__((force_align_arg_pointer))
> directly in this way, but I will defer to others on that question.
>
>
What problem do you see with it? I know just one which I pointed in my
comment:it may not work in the older compiler (it works in my gcc 5.3
from Fedora 23 though). I just found related discussion for it in Wine:
https://bugs.winehq.org/show_bug.cgi?id=27680. Please note that if this
attribute does not work for older compiler, we will still get the crash
only in case of buggy Win app call (just the same way some non .Net apps
already crashing due to this bug in older compilers). The attribute does
work for me with gcc 5.3.1 20151207. When I remove the 8 bytes from my
esp math (sub $0xE0,esp instead of sub $0xE8) for testing purpose, and
also remove force_align_arg_pointer, I do get the crash on my machine on
first movaps in ReallyFixupVTable. When I leave thunk stack without
padding, but add force_align_arg_pointer, there is no crash, and I see
forced stack pointer alignment in ReallyFixupVTable prolog. I also
checked of course that removing the attribute does not cause the crash
in my and your test case (both provide esp properly aligned).

Or do you mean that we must use some define instead? I searched the use
of force_align_arg_pointer through all the code, it seems like it is not
defined in any constant for x86_64 for now.