Rearchitecting the d3dx DLLs

[Matteo Bruni]

I think we're past the point where we need to figure out what we want
to do with the d3dx* DLLs, particularly regarding how to avoid
duplication of common functionality between d3dx9, d3dx10 and d3dx11.
This is something that's been on my mind for a very long time but it's
generally been a topic where people either care strongly or not at
all, so I kept pushing back the discussion over and over again. Now
I'm finally asking the former category to comment. The latter can stop
reading now :P
TL;DR: jump to the second-to-last paragraph.

Let's start with a quick summary of what those DLLs are. They are
basically a loose collection of utility APIs that might often come in
handy when writing 3d applications. To give a broad idea, these APIs
can be roughly categorized into math, mesh, shader, texture functions.
A d3dxn is somewhat related to the corresponding d3dn version: for
example d3dx9's D3DXCreateTextureFromFileA() creates a
IDirect3DTexture9 object and fills it with data from an on-disk image
file. Many other APIs (e.g. the math functions) are abstract enough to
be independent from any d3d version.
A particularly important detail with these DLLs is what MS decided to
do with versioning: each new release of the DirectX SDK (when those
were still a thing) usually had a new version of the d3dx DLLs as
entirely separate files. There might have been additions or changes
compared to a previous version but applications wouldn't see any of
that unless they intentionally switched to linking with the new DLL.
For example, the last released DirectX SDK (SDK version 43) contains
d3dx9_43.dll, d3dx10_43.dll, d3dx11_43.dll, in addition to ALL the
DLLs from the previous SDKs (which for d3dx9 means going back up to
d3dx9_24.dll).
There are interesting side effects of this choice. It allowed MS to
make incompatible changes (e.g. see the ID3DXEffect vtbl). It also
means that applications can end up depending on quirks specific to a
particular d3dxn_xx.dll version. It's also not uncommon to see
applications that load and possibly use multiple d3dx DLLs inside the
same process. In general, it's lots of fun...

Now let's go through the current state of those DLLs in Wine. The
d3dx9 DLLs are in a pretty good state as far as API coverage goes
(with a few notable exceptions, e.g. shader compiler). The
implementation is in d3dx9_36/ and all the other _xx variants use the
PARENTSRC mechanism to build entirely separate DLLs, mimicking native.
Originally we forwarded everything to d3dx9_36 but that broke some
applications, particularly from the demo scene.
d3dx10 still forwards most of the math functions to d3dx9_36. The rest
is largely unimplemented. d3dx11 is all but unimplemented.

Fortunately, MS dropped most of the APIs along the way, especially
when moving from d3dx10 to d3dx11. Unfortunately, that doesn't help a
ton. For example, there is enough stuff in the texture API alone that
could (and should) be shared and the API itself is different enough
that it can't be done by simply forwarding some public function from
d3dxn to d3dx(n-1) or such. Having >= 3 similar but separate
implementations of image filtering over various d3dx DLLs that all do
the same thing seems not that attractive. And then let's not forget
the other "axis" i.e. the _xx suffixed versioning which can make
things extra convoluted.

If you're still here, congratulations! Now for the proposal i.e. how
to move forward. There are a few options I can think of:
1 - Allow to have multiple paths in PARENTSRC, rework the d3dx9 code
to make it possible to reuse helpers in d3dx10 and d3dx11
2 - Introduce a new PE DLL where the shared code would reside, keep
the PARENTSRC mechanism to handle the _xx versioning
3 - Introduce a new unix lib for the shared code, keep the PARENTSRC
mechanism for the _xx versioning part
4 - Just duplicate everything, durr

Reasons to favor one option over the rest? Other ideas I have missed?
Personally I prefer option 2. The only downside that I see is that it
would introduce a Wine-specific DLL, which is something that we
normally want to avoid. There are many advantages though: compared to
1 (and the status quo) we'd reduce Wine compilation time and the
overall size of the d3dx DLLs; we'd also avoid loading extra d3dx*
DLLs as required by the forwards. In respect to option 3, we wouldn't
introduce unnecessary (sys)call overhead, which we really don't want
to have for stuff like D3DXVec4Transform().
Looking forward to any type of comment.