World Wine News

WHAT'S NEW with Wine-20050111: (see ChangeLog for details)

• Many OLE bug fixes and improvements.
• A lot more work on the MSI dll.
• Update regions now handled in the Wine server.
• Beginnings of typelib generation in the IDL compiler.
• Many janitorial cleanups.
• Lots of bug fixes.

This clears out the next blocking part of the d3d9 / wined3d support and has been a real pain to implement. I think I've got it right now, and it adds support for render targets but more importantly the beginnings of a front and back buffer which a lot of the code relied upon.

Unfortunately I couldn't get this next step any smaller than the patch attached....

Major areas still to go - textures and pixel / vertex shader support. I'm moving onto the textures next...

Changelog

Add render target support, and ensure there is a front and back buffer created during device creation

I've just done more-or-less exactly the same this weekend with a couple of differences.

I've merged all the creation surface code into a single createSurface function that is called with different parameters/flags to create a backbuffer, PlainOffscreenSurface or rendertarget etc...

This should mean that you only require one call-back function for CreateDevice in WineD3D.

I'll merge your patch with what I've got and see how it goes.

As soon as I'm happy with how wine's and directX work I can stop duplicating what you've done and start moving onto a more co-operative development, or moving DirectX8 over to using WineD3d.

So far I can run Pirates! and a few of the demos from http://www.codesampler.com/dx9src.htm , but without textures.

Pirates uses DirectX 9c but doesn't do anything too fancy and runs without crashing so it seems good for testing.

Many of the demos at http://www.codesampler.com/dx9src.htm have OpenGL versions and since I don't know OpenGL or DirectX (but can hack a good Dos demo) it seemed a good place to start.

I've spent the last couple of months putting together a directx 8 level support into wined3d and calling it from d3d9 such that I can also change d3d8 easily to use the new code. This is mostly complete other than the texture support and the shader support plus moving some functions / stubs into the common code.

If you have any directx 8/9 questions feel free to email me directly, put questions on wine-devel or perhaps even the wined3d (very, very quiet - http://www.bbrox.org/mailman/listinfo/wine-d3d) mailing list (Lionel's). You really are welcome, and I'll try to answer any questions I can.

I wouldn't spent too much time trying to move existing functionality between d3d8 to wined3d unless you really want to as I am trying to get this move completed asap. I really don't want to discourage any d3d assistance, it's just I know where I am and what I have left to do.

I have generally been doing a patch and getting it committed both due to lack to time, but also to avoid potential rework of multiple patches. I really hope to get most of the non-shader support done quite quickly once the 20 patch goes in (Christmas and FarCry got in the way, and I had some teething problems with the render target support which delayed it!). Next will be the texture support and that should be quick as it is relatively trivial to move across.

One thing wine has suffered from is a lack of people doing the directx support! Note that Raphael (copied on this note) has some outstanding patches he has put off while I do the move to support hardware vertex buffers so the drawstridedslow case is driven less frequently, but that's the only outstanding work I know about.

I have a version locally where I have changed d3d8 up to about patch 13. I can send it to you if it helps - Unfortunately I think the swap to wined3d will have to be one big patch, I don't think there is a way to move d3d8 over slowly, as you have to move the state block code in one go, the resource8 subclasses in one go etc...

The problem I had with the latest patch was purely being able to add the support for getfrontbuffer etc which needed to return d3d8 or d3d9 objects, but the initial creates being done in the wined3d layer. A single callback sounds useful, as I know I'll have to do something similar for stencildepth surfaces (nothing else is created purely from the wined3d layer, so I don't think others need the same).

FWIW I do the same, ie find demos on the web with source code, and then try to get them working. That's how most of the d3d8 support came about as it's a lot easier making a demo work than finding why there's a small amount of corruption for a particular game. I wouldn't claim to be a directx nor opengl expert - it's all been a learning experience as I go! Maybe we should collect some useful test program links!

If there's any d3d8/9 programmers out there, there's definitely a fun project to do to produce test programs for wine. They can't be automated (I think!) but it would be useful to be able to visually see the effect differences of all the render states, the texture states, prove the shader support works in both hw and sw vertex and pixel code etc etc.

Note that Con indicated several times that Wine is a very "special" piece of software, it has wildly differing gaming behaviour (threading behaviour) than native Linux games.

For those of you who don't know what priority inversion is (I'll try to explain it, but I'm sure my explanation sucks ;): one (high priority) thread is waiting on a lock, and in order to keep this high priority, another thread owning that lock gets more CPU to get that lock released. Problem is that that thread should have a lower priority, but since the high priority thread always has to wait on the lock, the low priority thread effectively has a much higher priority.

These could just be the usual known slow wineserver communication effects at work here, but I wanted to make sure you know about these issues on the CK list.

I suspect that the issue is much deeper than just the wineserver effects; Windows scheduling requirements are fundamentally different than the scheduling that the Linux kernel tries to supply, and the differences can be hard to troubleshoot.

Photoshop, for example, relies on the ability of a thread to Sleep for precisely 5 ms, and then wake up in a certain priority state. The Linux kernel inverts that (I forget the exact details, but we dug into this in great detail this past fall), and so the Photoshop thread has a great deal of difficulty. AFAIR, we had issues with the thread sleeping for 115 ms instead of 5 ms, and we alternately had problems with the thread starving other threads in the system, so that nothing else was being served.

On my todo list is an item to start a conversation with the Kernel guys about these scheduling differences, and to see if we can ask for a few tuning hooks to help us better match Windows. But I keep trying to goad Alexandre into doing it, and he keeps reminding me that I'm not supposed to bother him with anything until after the Window Manager rewrite is done... <grin>.

I spent a great deal of time this past fall exploring Wine scheduling issues, and I thought I would post what I learned.

The most critical point is Wine was pretty badly broken; it simply had a number of flaws in how scheduling was done.

First, the winmm timer, used by a lot of programs, was pretty awful. That has now been fixed, and I believe that it now works correctly.

Second, we did not synchronize Wine message times with X11 message times properly. Fixing that resolved some issues with Photoshop (and given that Ove reworked that patch into WineX the same day I submitted it, I'd guess it fixed some games too <grin>).

Third, Wine did not yield the processor properly at points where Windows does, including Sleep(0) and certain event conditions. That's now been (mostly) fixed, I think.

Fourth, the Wine server had an issue with the way it managed it request queues. To be honest, I don't understand this bug or the fix; all I know is that Mike McCormack figured this one out and posted a magic fix. This was, I believe, the primary reason that the wineserver would chew 100% cpu with iTunes.

Beyond that, there are a number of ways that Wine 'gets lucky' and Just Works (TM) works on the Linux scheduler, in ways that we couldn't have asked for.

First, because many Wine calls resolve into a wineserver call, and a wineserver call involves a write to a pipe, we potentially risk yielding the processor far more often in Wine that a Windows program would 'naturally' do so on Windows; opening us to horrendous timing bugs. However, in practice, because the Wine server often handles requests within a single quanta, and because waiting on a pipe with data gets a priority boost in Linux, the net effect is what Wine wants. (Thread A makes a request, yields to the Wine server who gets in ahead of all other threads, does the requests, sends the response back, and Thread A regains control before any other Wine thread because it got a priority boost).

Second, the new 2.6 average time quanta of 100 ms is comparable to the observed Windows quanta of 115 ms (I may have that wrong, but I recall they were quite close), although the old, much smaller time quanta masked a number of Wine scheduling problems. But, in theory, having this longer time quanta should let us better replicate Windows scheduling behavior.

However, we still have some issues. First, the problem with thread priorities still remains an issue. A non zero number of important applications rely on Windows priorities to change timing behavior to make their applications run properly. ([editors note: imho, these are poorly written apps, but we have to support them as they are; wishing that MS would rewrite PowerPoint to my spec is amongst the most futile things I can imagine]).

There is a patch Mike worked up which makes a theoretical improvement to the Wine server (pull requests from its queue in thread priority order), but we have no evidence that change makes any difference, and Mike backed away from it when he found the afore mentioned 'correct' fix. Nonetheless, I think it is conceptually 'right', so we probably should bug Mike into redoing that patch.

I've played with a number of artificial ways to try to replicate this within Wine, and have come up with a few compromises, but nothing works perfectly (or, more importantly, worked up nothing that Alexandre would apply <grin*gt;). If you do try to exactly mirror Windows priorities, for example (running as root, say), you run into issues where the wineserver gets starved and Wine stops functioning.

Candidly, that's where I'm stuck. I'm going to revisit a few of my 'problem' apps hopefully sometime this year and see if I can cobble together a hack that works best. Ove, I haven't tried the signalling approach; that seems like an interesting idea.

Sadly, short of asking the Linux guys to rewrite the Linux kernel scheduler to exactly mimic Windows behavior, I'm not really sure if there is a lot we can do to get this exactly right.

At this point, I think the only glaring hole we have is the issue of thread priorities. That seems to be a capability we simply can't accurately replicate in Wine. My hope is that if we can somehow mock that up (or address the few dodgy cases where it's a problem), we won't face any other timing issues.

From there, I think the honest truth is that I have more homework to do; we tend to take the pragmatic approach that a theoretical problem is uninteresting until we have real apps with real unsolvable problems. So I need to go find some more problem cases and understand them.

In the interest of making this thread complete, I wanted to add a concern or two I forgot, and to belabor a point or two (that's your clue to tune out now, folks <grin>).

Specifically, there are other areas of interest when you think about timing and scheduling. First is the Windows very clear cut 'foreground window' boost. I have no idea how this maps into the Linux interactivity strategy. They seem to be roughly equivalent, and I never found an application that relied on that timing.

Second, the exact sequence of when a thread yields or not on Windows is not something I'm sure I fully learned; I did my best, but my gut tells me that I must have missed some cases. The main trick as we make sys calls here is to make sure that we don't accidentally yield at times when Windows wouldn't.

That connects to the theoretical flaw in Wine (the whole use of the Wine server) that should make us vulnerable to all sorts of unexpected context switches, creating timing patterns unfamiliar to Windows applications, and causing problems. And yet, from observation, I never see any such unexpected context switches. The Linux kernel priority boost for pipes seems to protect us here; we should make sure we understand why and let the kernel guys know that we depend on it, so they don't knock it out from under us some day.

Fourth, based on your reply to my earlier email, I suspect I do not understand how time quanta are assigned. In fact, I had come to some conclusions at one point last fall, that I can no longer convince myself of (I was persuaded that a thread that constantly yielded had it's time quanta dranstically reduced; I can no longer reproduce that). Further, time quanta differences can be crippling to Wine. Photoshop, for example, relies on reliable 5ms timing, and if it doesn't get it, it doesn't work. Similarly, I could swear I had a problem case in IE where two threads were interdependent, and the one thread never got a large enough quanta to keep the second thread happy (and this then caused a spiral that led to the small quanta for thread A). Sadly, I didn't take good notes, so I have to go back and relearn that case again. And, knowing me, I probably completely misunderstood it anyways :-/.

Sorry to ramble on; it's just tough around here, because everyone is tired of hearing me claim that every bug is a timing problem...

Your typical Photoshop user relies on the fact that Photoshop is the only application of any priority on the machine; Photoshop has a thread that runs that samples the mouse position every 5 ms; if it doesn't get it's samples in the right amount of time, you don't draw smooth curves. (It may be a timer event callback routine, or a thread that is woken up by a timer event, I can't recall exactly atm).

I suspect that if you start a heavy compile on a Windows box, Photoshop won't paint right, just as it wouldn't if you burdened the CPU on Linux.

What we need to protect is the apples to apples case - when Photoshop is the only app using the CPU with Wine, it needs to draw smoothly, and for that, it needs to know that the thread it has marked as super duper high priority will get run every 5 ms as it has requested.

(My gut reaction to this was that it was horrid programming on Adobe's part; but they are trying to provide extremely fine resolution on drawing, and their approach makes sense in that context. I am now persuaded of their competence <g>).

This is not at all unusual. If there is no locking between threads, and you simply hope that one will complete in time for the other, you will get various degrees of priority inversion. Native linux applications do it too, and 2.6 has forced them to improve their coding principles (see the thread on 'blender' in the linux kernel mailing list during 2.5 development. There are various algorithms that have been tried to detect when this is happening and inherit appropriate priorities, but they have their own overhead and aren't as effective as proper blocking in the first place.

Generous use of blocking where appropriate is the answer to being friendly to any scheduler. Hoping dependent threads will complete in time is futile as the speed of execution of each thread will change on different schedulers, hardware etc.

The biggest problem is that there is no way to say to the kernel that one thread is more important than another without permanently renicing all other threads. A potential kernel solution to the problem would be to implement process scoping in the kernel, i.e.

pthread_attr_setscope(attr, PTHREAD_SCOPE_PROCESS)

and then allow threads scoped this way to be set to high priority, since with a process scope, these threads should only preempt other threads in the same process (i.e. Wine), not threads run by other Linux apps, and thus the security concerns of elevated priority threads are irrelevant.

Allowing a process-scoped thread to set the scheduling policy to SCHED_RR in order to inhibit the kernel's interactivity priority boost system would also help.

Recently, I helped a student at Harvey Mudd college write and submit a conformance test for the lzexpand dll: http://www.winehq.com/hypermail/wine-cvs/2004/11/0164.html I picked lzexpand.dll since (lacking a GUI) it seemed simple to test.

Now I'm putting together a reusable lab project for other colleges, at http://kegel.com/wine/sweng , and am trying to pick a few good DLLs for the students to write tests for. (I plan ask the students to send me their patches so I can review them before they post them on wine-patches.)

Clearly, I shouldn't ask anyone to write a conformance test for a huge DLL in a term, so I guess I'll focus on on the DLLs with under 100 functions. And I'd like to skip DLLs that have GUIs, so I guess I'll omit any where the word HWND occurs.

That's a small enough list that I can start guessing at which ones are worth writing tests for. Let's see, how 'bout the following 12:

Are any of those obviously poor choices for writing a conformance test? Or are there others that are obviously good choices, but that I missed?

Well, there's still plenty of test cases that can be written for each DLL. Ideally we start from the bottom (ntdll, kernel, gdi, etc) with our test cases so that we can fix code in the dlls and programs that depend on those dlls.

Following that line of thought, imagehlp (used by winedbg), cabinet (used by msi and setupapi), wsock32 and crypt32 seem like good choices.

since most of the symbol (& debug information) part of imagehlp has been moved to dbghelp, I'd say dbghelp might be a better candidate than imagehlp (and winedbg uses directly dbghelp, not imagehlp)

I want to write a linux driver for my USB scanner which is currently not yet supported by sane. So I wonder where to start; where do I find detailed information about the Windows driver interface, esp. for USB scanners. I want to know which interface a driver has to offer to Windows and which functionality the API has to offer.

Well I think the right solution would be to add support for your scanner to SANE. Then Wine should be able to use it using the twain dll (see dlls/twain) which acts as a bridge to SANE.

Interfacing directly with the scanner might work using Wine's brand new and very incomplete USB interface (I believe part of it is in setupapi) but that would probably be more code and you would still end up writing what is essentially a driver. In other words in either case you will probably need the hardware specifications.

I have a Tungsten E PDA and use K-pilot/J-pilot to backup/restore/install .pdb/.prc files with my linux box. No problem for that.

However, I would like to install a .pdf viewer on my PDA and the files provided for this installation are not .pdb/.prc files but a Windows executable file (AdobeReader305-PalmOS.exe). To install this file, I need to install the whole Palm Desktop CD under windows.

Consequently, I thought to use Wine for that: I compiled Wine_20041201 on my linux box (RedHat 9.0), configured it with 'winesetuptk' and run 'wine /mnt/cdrom/autorun.exe' to install the whole Palm Desktop CD for window.

Is it possible to do install the 'Palm OS Desktop' CD with Wine and/or any idea for my problem ?

A small step-for-step howto:

• Install Palm Desktop.
• install the Adobe Reader for PalmOS converter. This means running adobereader-palmos.exe with wine
• Launch hotsync.exe from the directory where Palm.exe is installed. You'll get the hotsync icon in your Desktop's systray
• Double-click it with the right mouse button. You'll always have to double click the hotsync icon to open the menu, I don't know why
• Select the "Setup..." menu item or whatever it is called. I have the German version and it's labeled "Einrichten..."
• Open the network tab, press "TCP/IP Settings". Check if the ip is correct and enter a correct subnet mask, e.g. 255.255.255.0
• Press OK to close the windows
• terminate any Linux app which might access the Palm. Connect the Palm to your PC with the USB cable
• Launch the Hotsync program on your PalmOS device
• Open the menu and select "LANSync-Setup" or what it's called. The german version labels it "LANSync-Einst..."
• Select LANSync and tab OK to close the dialog
• Start the Hotsync over the usb cable.
• Run pi-nredir -p /dev/pilot on your computer 3 or 4 seconds later. The Hotsync dialog should pop up and the Device should perform a hotsync with the Win32 Palm Desktop. (replace /dev/pilot with the hotsync port visor.c provides to you. Usually /dev/ttyUSBX or /dev/usb/tts/X )

To upload a pdf launch the pdf converter from it's install directory, and select the pdf file to upload. Then perform a hotsync.

I hope this is correct, I wrote it down from memory.

Your best resource may be:

http://www.codeweavers.com/support/docs/crossover-pro/troubleshooting#ITUNES-IPOD

I think it explains the steps pretty clearly, and would be pretty useful even to non CrossOver Wine users (don't worry about not having cxipod, it doesn't do anything that can't be done manually, and I'm pretty sure it's all explained).