WineHQ

World Wine News

All the news that fits, we print.

05/14/2007
by Brian Vincent
Issue: 331

XML source
More Issues...

This is the 331th issue of the Wine Weekly News publication. Its main goal is to moonlight. It also serves to inform you of what's going on around Wine. Wine is an open source implementation of the Windows API on top of X and Unix. Think of it as a Windows compatibility layer. Wine does not require Microsoft Windows, as it is a completely alternative implementation consisting of 100% Microsoft-free code, but it can optionally use native system DLLs if they are available. You can find more info at www.winehq.org


This week, 340 posts consumed 672 K. There were 93 different contributors. 52 (55%) posted more than once. 53 (56%) posted last week too.

The top 5 posters of the week were:

  1. 33 posts in 36K by julliard at winehq.org (Alexandre Julliard)
  2. 25 posts in 39K by speeddymon at gmail.com (Tom Spear)
  3. 19 posts in 78K by stefan at codeweavers.com (Stefan Dösinger)
  4. 19 posts in 120K by mounir.idrassi at idrix.fr (Mounir IDRASSI)
  5. 18 posts in 18K by dmitry at codeweavers.com (Dmitry Timoshkov)

News: Wine 0.9.37, No Wine on Dells, Kidney Stones Suck Archive
News

Last Friday Wine 0.9.27 came out. New improvements you'll find are:

  • MSI automation with JScript/VBScript support.
  • Various MSHTML improvements.
  • The usual assortment of Direct3D fixes.
  • Support for a few more exe protection schemes.

Unless you were living in a cave a few weeks ago, or at least ignoring every Linux news website, you probably heard Dell is preloading Linux now on desktop machines. I'm still trying to figure out why this is big news since pre-loaded Linux has been around for a while. Anyway, other people think it's big news so we'll play along. Apparently it was big enough news that eWeek wondered if Wine would be shipped on those machines. There's a decent article, Ubuntu Founder: No Emulation Software for Dell Systems , that explained Wine wouldn't be part of the package. There's also a nice quote by Mark Shuttleworth:

"While Linux is an alternative to Windows, it is not cheap Windows. Linux has its own strengths, and users should want it because of those strengths and not because it's a cheap copy of Windows."

To make this non-newsworthy event even an even bigger sensation, Slashdot decided to run with it as well. If for some reason you have a Debian based system and you can't figure out how to install Wine, check out our Debian download page.

In a hilarious press release (and for the record, hilarious press release is an oxymoron) Jon Parshall of CodeWeavers postulates that developing Wine may lead to kidney stones :

Forty-four year old Jon Parshall, chief operating officer for CodeWeavers (www.codeweavers.com) - a leading developer of Wine and other Linux programming solutions, which allows users to run Windows applications without purchasing a license from Microsoft - recently suffered two weeks of mind-numbing agony and extraordinary urethral discomfort as a result of at least one or possibly more kidney stones that suddenly appeared within his urinary system.

"It was like I was descending into the eighth level of hell," Parshall explained. "My days were filled with mouth-drying, white-hot shards of torment that stretched from my lower back across to my abdomen and beyond. Sweat-soaked nights were spent rolling in bed in agony. I pled for a second of respite in the form of sleep that never came."

Oh, and a new version of CrossOver Mac is expected soon that will support more apps.


Direct3D Milestone Archive
DirectX

Stefan Dösinger wanted to let everyone know Direct3D as of Wine 0.9.37 had reached a milestone:

With Wine 0.9.37 we've achieved something that I allow myself to call a small milestone - All Direct3D7 Immediate mode SDK demos successfully perform their intended rendering. I have some screenshots on my university junk server:

Two major problems are left though, namely windowed opengl rendering (see the junk where a menu bar should be) and GetDC (there should be a little bit of text rendered, I disabled render target locking to get proper performance).

Some demos have a few smaller problems too. The bump earth and bump waves need vendor specific extensions (GL_NV_texture_shader2 / GL_ATI_envmap_bumpmap) and the bend demo needs GL_ARB_vertex_blend which nvidia does not support. So I took that screenshot on my mac running linux (ati radeon X1600). The mipmap demo renders the mipmaped texture garbled, although this works on nvidia cards. The Z buffer demo says w buffers aren't supported, but the normal z buffer does what the w buffer is supposed to do.

What does that mean for gamers? Nothing fancy really, the features used by games were implemented a long time ago, the ones we used to miss (fixed function bump mapping, vertex blending) aren't really important for games. Other DirectDraw features like Overlays aren't implemented either, but we are Direct3D 7 feature complete.

Where to go from here? I am currently fixing the DirectDraw rewrite regressions I can get hold of, and I am trying to make D3D thread safe finally. From the application point of view my focus will stay on fixing older apps first, which somewhat includes getting thread safety, render target locking and GetDC working properly. My hope is also to fix the other related DX7 sdk demos, like the DDraw demos (except overlays, they are highly tricky with the current wined3d-x11drv integration) and getting native d3drm.dll working.

I also want to say thanks to all the small and big helpers who help with technical advice, debugging and regression testing. Special thanks goes to Henri without whom I'd be totally lost and who has to be credited for a huge majority of the shader work in the last half year, and a lot of other things too.

Thanks and continue the good work :-)

PS: As for the screenshots, my space on that server is limited, so they will be gone sooner or later. I didn't want to send huge screenshots as attachments, not even as jpgs.

Henri Verbeet then suggested D3D8 be the next target, although a lot of it is implemented. Stefan explained a little more about future DirectX work:

Yeah, that's the other thing of course :-) But as I said even d3d7 and ddraw had a few biggies I want to fix. Of course I may grab some low hanging fruits like kicking out the fvf code from d3d8 and wined3d :-) Or get the sdk demos going, they should all work now by looking at their names.

The plan is still to sort as many biggies out before the 1.0 freeze. The sole difference between .36 and .37 from that regard are a few little fixes specific to the d3d7 sdk and bumpenv mapping implemented. Nothing that gets us as far as working multithreaded d3d.

After that, several people put in requests for what they'd like to see working with regard to Direct3D.


Activation Context Archive
Architecture

A largish patch arrived this week and before we get into it, we'll step back and give a little introduction.

Some new features added to Windows in the past few years require learning some new terminolgy. Some of this functionality is used by .NET, though it's not strictly a .NET technology. So let's say you've got an assembly , a collection of PE files like DLLs, and you want to describe how they all interelate and what version of DLL you want to use. What you'd do is create a manifest composed of an XML structure that defines how the different classes are bound together, dependencies, versioning info, etc. To get fancier, you can have side-by-side assemblies (SxS) that allow different versions of the same DLL to exist on the system. Furthermore, those libraries can be installed either in private directories, such as the application directory, or in c:\windows\winsxs. You can also do things like enable different features of the DLL based on the assembly.

(Now, if you're thinking that Microsoft has once again tried to engineer itself out of a DLL-hell, you'd be correct. No longer does a library need to worry about backwards compatibility because you can simply ship a slightly tweaked version of the same library. The different manifests will take care of the versioning.)

Okay, so how much of that infrastructure exists in Wine.. um.. lesse.. not a whole hell of a lot. What is it needed for? Well, pretty much any app requiring version 8 of the Microsoft C runtime libraries (msvcr80.dll). These changes require modifications of Wine's loader mechanism, something that's been considered stable for quite a few years. Fortunately, the changes shouldn't be too invasive and don't require rearchitecting such a core component.

Eric Pouech and Jacek Caban posted the initial patch for some of this stuff. It consists of 2800+ lines of code spread across 27 different patches. By way of introduction, Eric explained what's been implemented so far:

The following series provides a first real implementation of activation contexts. It has been written by Jacek Caban and myself. Even if individual patches show only one of the two names (for ease of packaging), reality is different. In many cases, ideas, implementation, bug fix(es), improvements have been done by both of us.

Basically, this first series of patches implements:

  • searching and building the hierarchy of manifests needed by an application
  • parsing of such manifests
  • implementation of the Query function on this hierarchy of manifests
  • core activation context manipulation APIs (add/release ref, stacking...)

This doesn't allow yet to let an application with manifest being successfully run in Wine. This will require:

  • implementing the Find functions
  • patching the DLL loader (for both DLL lookup at first step; name prefixing will come later on)
  • enhancing the process loader a bit

The core idea of this set would be to let an app with a manifest and linked to msvcrt80 be successfully run under Wine.

Jacek and I have preliminary work on this, but this is not fully ready, so let's start with committing this first set.

What will still have to be done afterwards includes:

  • moving code from kernel32 to ntdll (no doc exists on the Web currently for those)
  • all the COM, window class, name prefixing that's allowed

Alexandre had some technical comments about the patch series and ended it with:

I don't think this patch series is quite ready to go in; this is the third resend and all patches still have obvious bugs. You need to spend more time on it, this is kernel stuff, it needs to be correct.


USB Support in Wine Archive
Architecture

This topic hasn't come up for a while, so it's probably useful to go over it again. Jon Burgess asked a question about USB support in Wine:

I have found some talk of implementing USB device support in wine in this list some time ago (2005), but as far as I know, nothing ever came of it.

I would perhaps be interested in getting this going again, as I have an application (Serato Scratch Live: http://www.rane.com/scratch.html) for which the software appears to run ok under wine (not that I am able to test much of its functionality on the other hand), but is utterly useless without support for its associated USB hardware device.

Any ideas, or anyone else who would be interested in helping me?

Damjan Jovanovic replied with an excellent description of the problem at hand:

There is no "standard" driver for your USB device (like for example there is just 1 driver for all USB flash disks, because they're all in the same device class). You have the use the vendor-supplied .SYS file.

Will it work on wine right now? Not even remotely. Will it ever? Read on...

There is no standard user-mode interface for accessing USB hardware - there is no equivalent of Linux's libusb on Windows (there is apparently some user-space USB stuff in mingw's headers, but I couldn't find any official docs on it, and it's not enough to write a user-space driver because there is no bulk/interrupt/isochronous pipe support, so I doubt anybody actually uses it). There is only kernel-mode interfaces for talking to USB - and different kernel mode drivers will export different user-mode interfaces (if any).

So user-space software uses a kernel-mode driver. If that driver is documented, like USBSCAN.SYS and USBPRINT.SYS, its interface can be done in wine, and wine can be connected to USB in a number of ways (which I'll discuss later). If that driver is undocumented, either you have to document it by reverse-engineering (very hard) and continue with the first way, or (the second way:) make wine's not-yet-existing NTOSKRNL.EXE load that driver, *and* provide NTOSKRNL.EXE with all the USB interfaces that Windows has.

CONNECTING WINE TO USB: A HOW-TO GUIDE

There are 4 ways to do it:

  1. Make a kernel module in your OS (eg. a Linux kernel module) that exports the same interface that the software expects and works the same way as the Windows driver. Using USBSCAN.SYS as an example, reading does a USB bulk read, writing does a USB bulk write, and 13 or so i/o control codes do various other things, among them USB control and interrupt transfers. Change kernel32's CreateFileW() to open the /dev device node used by the kernel module and send it to the wine server using wine_server_fd_to_handle(), then reading and writing will go to your kernel module, where you can implement them by doing USB reads and writes. Change ntdll's file.c's NtDeviceIoControlFile to capture i/o control codes used by your device, call wine_server_handle_to_fd(), and do an ioctl() on that fd to send that i/o control call to the kernel module, which then reacts appropriately. This way has been tested by me, it works and it's fast, but it's non-portable (eg. Linux-only, and Linux's USB interface keeps changing so an out-of-tree module will only work on some versions, the usual problems...), difficult (kernel-mode code is hard to write), and generally a royal PITA.
  2. Like (1) but use a framework like FUSD (the xiph.org version works on 2.6 kernels) to write the driver in user-space and then do USB I/O using libusb. I'm currently testing this approach, I suspect it's slow (how significantly remains to be seen), but at least it's more portable between Linux versions and easier to write and maintain.
  3. Integrate your device into wine directly the way eg. serial ports have been done. This is hard, and requires introducing a new FD_TYPE, changing ntdll's file.c's NtReadFile, NtWriteFile, and NtDeviceIoControlFile (and asynchronous versions of those) to use special behavior for that FD_TYPE, and managing global device state (eg. i/o timeouts) in wineserver (look at server/serial.c). With a lot of time, and some changes to both wine and libusb, you could get it to work properly. This should IMO only be done for generally useful drivers, not drivers for just 1 type of device.
  4. Integrate NTOSKRNL.EXE into wine, add USB infrastructure to NTOSKRNL.EXE so that kernel-mode drivers can access USB (probably through libusb), and modify ntdll to forward the appropriate reads, writes, and i/o control requests to NTOSKRNL.EXE so that the .SYS file can handle them. This is the only way that works with .SYS files out-of-the-box, the others all require a rewrite of the .SYS driver's functionality. Architecturally, this is the best way, you wouldn't need to change any code in wine to add a new driver.

Way 4 is probably the best and I hope it works at some stage, but it's still a long way off seeing as how NTOSKRNL.EXE itself still isn't in wine.

A dedicated person could probably pull this off over a few months. One of the tricky things will be to get it accepted into Wine. An ntoskrnl.exe was written a couple of years ago to load drivers, specifically the Safedisc Windows kernel driver, but it still hasn't been accepted. Apparently there's some low-level internal Wine issues and Alexandre wants those fixed before accepting the patch.

A few people pointed out that different methods on Windows do exist that allow device drivers to be written to a standard framework.


Mandriva RPMs Archive
Packaging

Marco Meijer packaged updated Mandriva RPMs and announced them:

As I have still no way to upload the mandriva wine rpm to the official winehq download server, I put it on my own website.

So Mandriva users you can download the latest wine binaries here:


Improving Debugging Performance Archive
Debugging

Markus Amsler felt the time it took to load debug symbols was way too long. For example, in World of Warcraft it's 100 seconds or so. He came up with a patch to use Wine's Heap memory functions directly and it reduced the load time to 18 seconds. There was some back and forth discussion with Eric Pouech, the resident dbghelp.dll guru, and some new patches were generated. Markus benchmarked all the patches and concluded:

I've played around with dbghelp performance. My test case was breaking at an unknown symbol (break gaga) while WoW was loaded in the debugger (wine winedbg WoW.exe).

Conclusions:

  • current git wins with small debug files (<2M or so), pool_heap wins with bigger files. insert_first, process_heap are out.
  • small pools have less memory overhead than small heaps
  • big pools have more memory overhead than big heaps.
  • big pools are a lot slower than big heaps.

IMO the best results would be removing the pools (like in process_heap) and freeing unused memory manually, the other way round it was allocated. But at a first glance it looks like quite a bit of work, which I'm not sure is worth the result. I think the best approach would be to code some destroy functions in storage.c which would free the allocated vector, sparse_array and hash_table memory. And then gradually replace pool_alloc calls with HeapAlloc/HeapFree pairs.

After more discussion, Markus sent four patches and explained how well they performed:

I tried several approaches to avoid the memory leak still present after this patch. But all of them used more memory, were more complex and slower. I tried HeapCreate for pools, separate HeapAlloc, HeapReAlloc for buckets, using HeapAlloc for pool_alloc with storing the pointer in the pool. I even implemented a simple radix tree for the bucket storage.

The initial bucket size has no measurable effect, so I left it at 1 to save some bytes.

Here are the stats for this series:

x Memory after startup(kB) Memory normal symbol(kB) Time to load big debug file(s) Memory after big debug file loaded(kB)
orig 10920 55036 115 146388
dbghelp[1] 10664 52372 48 92448
dbghelp[2] 10668 52604 16 92632
dbghelp[3] 10632 52604 7 92632
dbghelp[4] 7444 50256 7 90452


All Kernel Cousin issues and summaries are copyright their original authors, and distributed under the terms of the
GNU General Public License, version 2.0.