ptrace single-stepping change breaks Wine

[Linus Torvalds]

On Mon, 22 Nov 2004, Eric Pouech wrote:
> 
> For the linux folks, here a small comparison of what happens in the working 
> (old) case and in the non-working (new) case:
> 
> In both case
> 
> - Wine gets a first SIGTRAP (in it's sig_trap handler)
> 	+ Wine converts it into a Windows exception (w-exception in short).
> 	  This includes creating a context for the generic CPU registers
> 	+ This w-exception is sent to the w-exception handler the program
> 	  installed (this one can modifiy the all registers)
> 		o this handler touches one of the i386 debug registers
> 	+ as we need to update the debug registers values (and we don't do in
> 	  the signal handler return), this task is delegated to the Wine server
> 	  (our central process, which is in charge of synchronisation...)
> 		> the Wine server ptrace-attach:es to the process which handled
> 		  the SIGTRAP.
> 		> Wine server wait4:s on the SIGSTOP (after ptrace:attach)
> 		> modify (with ptrace) the debug registers
> 		> and resumes excution (ptrace: cont)
> 	+ wine terminates the sig trap handler and resumes the execution with
> 	  the modified basic registers (from the saved context), and the
> 	  modified debug registers (from the Wine server round trip)
> - a second sig trap is generated
> 	> since the wine server is still ptrace:attached, it gets the signal.	
> 
> What differs then in both execution:
> - in the working case, the sig trap handler is called on the client side, 
> whereas it's never called in the non-working case. We do have a couple of 
> protection (to avoid some misbehaving apps), but none of them get triggered. So 
> it seems like the trap handler is not called (ugh).

This actually implies that the current -bk tree with my latest patch may 
actually fix it.

One of the things that 2.6.9 did wrong was exactly that it cleared TF too
much in the ptrace interface. The current development tree is a lot more
careful about that, and it fixes the horrid test-case that I used to debug
it. The test-case (when run under gdb) actually does something similar to
what Wine appears to do.

> - in Windows trap handling, the TF is explictly reset before calling the windows 
> exception handler (which is what Wine does, before calling the window exception 
> handler). Of course the handler can set it back if it wants to continue single 
> stepping.

TF will be still set in Linux when ptrace gets access, but the ptracer can
choose to clear it with PTRACE_PEEKUSR/PTRACE_POKEUSR (or with
PTRACE_GETREGS/SETREGS). I assume you already do that, since I think that
has been true forever (although maybe you don't: PTRACE_CONTINUE used to 
unconditionally clear TF, so it may be that Wine may need some minor 
modification to not do that - but the good news is that mod should be 
backwards-compatible, so it should be pretty easy).

I actually broke down and am downloading the latest source tree of wine,
let's see if I can find the place you do this.

		Linus