World Wine News

Beginning of this week, a colleague of mine walked up to me and said "Wine does not work.". Since this happens sometimes, I just walked to his office and checked it for myself. And yes, it did not work. It reported:

wineserver: lstat /tmp/.wine-coolo : No such file or directory

Some straces later the problem became apparent:

He is using a glibc 2.3 headbranch snapshot (the one we will ship in SuSE 8.2). Looking at the output of "nm", then looking at the sourcecode confirmed first suspicions.

__errno_location and __h_errno_location are no longer weak symbols and so cannot be overwritten any longer. The internal glibc systemcall wrappers no longer call the functions by reference, but directly.

Investigations and several talks to one of our glibc gurus (Andreas Schwab) gave following results:

• This is intended behaviour and it will not be changed back.
• They are not meant to be overwritten.
• The functions are provided by the pthread implementation with which glibc was compiled. (either linuxthreads, ngpt, nptl or none). These are optimized for speed and do not support hooking.

I spend 2 days studying glibc and thinking on how to best solve this problem, to no avail:

• Hooking into those functions is not possible, any approach will most likely not be accepted by the glibc hackers.
• Using dietlibc or uClibc is not possible, since dietlibc does not support shared libraries yet and for both C libraries libX11 and others would need to be compiled with them too.

The afore mentioned glibc guru did not have any ideas either.

There however is a happy but messy end, which appeared to me yesterday: Overwrite those 2 functions with a jump to our implementations.

So I did. You can now mark me up for the dark side.

Excuse me, but somehow I think this is p*ss poor. (and yes, I'm now marking you up for the dark side ;-)

I mean, both Wine and glibc are successful (?) OSS projects, so they should be able to come up with something much better than this terribly embarrassing solution (after all everybody knew that OSS development was a "superiour" approach, didn't they ? ;-).

I for one would feel much better if we simply rejected that particular "broken" glibc version and supported a *new* glibc method of properly interfacing errno things in a newer glibc version... (maybe have some "advanced" setting in glibc that enables all sorts of funky interfacing capabilities in case a program needs it)

After all if Wine needs this errno support, then there's probably a need for it, so it's glibc's bloody obligation to make sure there's proper support IMHO.

BTW: why did they even choose to abandon a public errno_location ?

Anyway, thanks for tackling this severe issue !

P.S.: Do you really think that Alexandre would commit it like that ? ;-)

glibc has switched to using thread-local storage for errno (i.e. it is declared as 'extern __thread int errno') when the tool chain supports the __thread keyword.

This means that C source code compiled against the new headers will result in assembler code that *directly* accesses a thread-local variable as defined by the TLS ABI. In the case of errno, this will involve accessing the %gs segment using an offset from the GOT, without any function call being performed. (errno is defined to use the initial-exec TLS storage model.)

The __errno_location routine is provided only for backwards compatibility reasons, it is no longer guaranteed that every access to errno calls it. Thus, if you overwrite the implementation of __errno_location, you will only catch *some* errno accesses, not all of them ...

Lets have a small example:

wine does:
#include <errno.h>
...
ret = mkdir("/blub/");
if (ret == -1) {
fprintf(stderr,"mkdir failed with errno %d\n",errno);
exit(1);

}

In 1980 this was rather nice and worked all the time with the nice global 'errno' integer variable. However, someone had to invent multi threading.

After this, errno could not stay a global integer variable, since you could get into race conditions writing/accessing it. Sinc millions of lines of code could not be changed, the <errno.h> header was changed to defined it as:

extern int *__errno_location();
#define errno *(__errno_location())

With __errno_location (or similar) a function that returns a pointer to an integer variable within the thread local storage area.

(The same goes for __h_errno_location and other internal functions.)

The glibc implementation basically does:

... convert registers ...
int 0x80
jae error
lret...

error:

...
lcall __errno_location
mov errorcode , ($eax)
...
lret

glibc follows the pthread style of threading, at the time WINE needed threading implemented by SIGALRM timers within a single process (clone(2) was not invented yet).

As WINE started Win32 threading the clone(2) handler was available for us and we implemented our own kind of threading, Windows style. glibc however does not know a single thing about that and still assumes there is no threading and had __errno_location return a single pointer.

So we had to overwrite __errno_location(), which was easy possible, since libpthread also did so and it was exported as weak symbol meant to be overwritten.

However with glibc 2.3 the internal thread representation changed, most pthread libraries now use clone(2) too, and use a new way of Thread Local Storage, using a segment register. Since WINE was using %fs, they chose to use %gs.

Now a system call looks like:

... convert registers ...
int 0x80
jae error
lret...

error:

...
mov errorcode , %gs:(offset)
...
lret

So we no longer can overwrite __errno_location to have our own errno storage, so we need to cooperate with the libc threading.

We have a winethreads-on-pthread implementation that we did for some of our non-x86 work a while back. It's pretty simple, and we haven't really tested it on an x86 machine recently - when we did so last, it definitely had some problems. It is pretty similar to work that Andrew Lewycky did at Corel to support cprof and multithreaded profiling, and probably has some of the same caveats that were discussed on wine-devel in March of 2000.

The code (excluding some configure checks) is here - some updates will probably be required to get it working with a more recent tree. WineHQ is welcome to use it. It would be nice if any improvements made to it were donated to ReWind as well:

http://www.geocrawler.com/archives/3/9376/2001/12/50/7309863/

There is a new page in my little Wine WWW -- the UI status page:

http://www.winehq.org/?page=status_ui

This is a page that I've been meaning to create for many months now. Please check it out, and let me know if you notice any inaccuracies.

This one is sure to give Greg something to work with... all of this was implemented in a bit of a hurry, but since it's based on my research, it should be a good starting point in understanding how Microsoft's NDR engine works. It doesn't properly implement marshalling alignment or memory sizing, may not handle a number of fringe cases, does not conform to the DCE RPC wire protocol (mostly because I don't have a description of it... where did you find it, Greg?), and probably needs some cleanup, but it seems to do the job for me.

Log:

Ove Kaaven
Implemented marshalling of pointers, simple and complex structures, conformant and complex arrays, and user-marshalled types. Improved marshalling of conformant strings and interface pointers a bit.

I think, I had to sign away my firstborn to OpenGroup for it. If you feel like spending money /and/ signing away your firstborn, this looks like the definitive OpenGroup DCE package:

http://www.opengroup.org/products/publications/catalog/t151x.htm

and here is some free-as-in-beer RPC 1.1 stuff:

http://www.opengroup.org/products/publications/catalog/c706.htm

Of course, I was kidding about signing away your firstborn, although you do at least sign away any redistribution rights by downloading the free stuff. AFAIK OpenGroup/DCE controls the RPC spec -- expecting consumers to sign an NDA or something would kind of defeat their stated purpose of enhancing interoperability.

There is one really significant open question left by their documentation: what is the gap between "MSRPC" and DCE RPC? I still can't decide whether this difference is just an API-level difference, or a full-blown wire-protocol incompatibility.... Luckily, we probably don't care too much -- ultimately, if we can make something that interoperates with Windows, we win, regardless of how well MS complied with the spec.

Is there a reason that:

1. The Wine open/save dialog boxes don't show or follow symlinks and
2. They show dotfiles?

I seem to recall a discussion on wine-devel way back on the topic of dotfiles, but a quick archive search didn't turn up much of use. At the very least I think it should be a pref, wading through lots of dotfiles in your home dir makes it much harder to open files with wine, and obviously non-technical users will wonder what is going on with all these stranges folders that I never made......

Check out the config file:

[wine]
;"ShowDirSymlinks" = "1"
;"ShowDotFiles" = "1"

I am interested in taking on the "build a database of windows APIs and dependencies" task. I have started work on scripts to scan a windows system and list import/export information. So far I can gather a list of dlls and run dumpbin on them, then parse imports (mostly, I think). I've started with simply printing the parsed output in three columns. The first is the DLL being scanned, the second is the DLL being imported, and the third is API imported. I am unsure of one thing. There is a 1 - 3 character hexadecimal number preceding almost every API. What is it, and should it be included in the output? Here is an example of what I am referring to: The hexadecimal number, in the third column.

Also I could use some direction on what the goals of this task are. Do we want text output? That makes a very large text file when listing all imported/exported APIs. Do we actually want a database for this?