HLSL Compiler and d3dcompiler_xx.dll

[Matijn Woudt]

On Wed, Apr 14, 2010 at 5:59 PM, Matteo Bruni <matteo.mystral at gmail.com> wrote:
> 2010/4/14 Matijn Woudt <tijnema at gmail.com>:
>> On Wed, Apr 14, 2010 at 4:27 PM, Stefan Dösinger <stefandoesinger at gmx.at> wrote:
>>>
>>> Am 14.04.2010 um 15:44 schrieb Henri Verbeet:
>>>
>>>> On 14 April 2010 15:07, Stefan Dösinger <stefandoesinger at gmx.at> wrote:
>>>>>> 3) Implement the compiler in d3dcompiler_xx.
>>>>> I wrote a basic HLSL compiler as university project in 2008, this is where part of the assembler code came from. Do you have the sources, do you need them?
>>>>>
>>>> Quite frankly, I also believe that's where some of the issues in the
>>>> early versions of those assembler patches came from. I don't know if
>>>> that compiler has seen any work since 2008, but I'd be careful with
>>>> taking it as too much of an example.
>>> Yep, I recommend using Mattheo's code if possible. He has rebased my old code and fixed a bunch of issues. Beyond that, my compiler is fairly minimalistic. I think it supports most features except arrays(needs loop unrolling support). However, the optimizer is fairly simplistic. For better optimizations you may want to investigate SSA transformations and related stuff. I guess perfect optimization isn't a requirement for a first version though.
>>
>> First thing would probably be cleaning up and getting something minimal past AJ.
>>
>>>
>>> Just in case my git tree is here: http://84.112.174.163/~stefan/wine/ . It's not online all the time, and I am not guaranted a fixed IP(although it hasn't changed in years), so check it out as long as you can if you need it.
>>>
>>
>> Am I supposed to find your compiler code there? It seems like a git
>> tree of ages ago (where d3dx9_36 only contains math and font code).
>>
>>
>>
>
> Yeah, Stefan's project is based on wine 1.1.0 and you can find the
> relevant code into dlls/wineshader. One of the things I did was to
> port the assembler to current wine and move it into d3dx9_36.dll. In
> the process I changed quite some things all around and I stripped from
> the bytecode writer what wasn't needed by the assembler. I didn't
> update the compiler code, and as Henri said, at this point probably
> it's better to just continue what you are doing and look at Stefan's
> implementation only as a "source for inspiration".

dlls/wineshader doesn't exist in that tree?

> As d3dcompiler_xx.dll now seems to contain also the assembler, I'm ok
> to move the assembler there (once completed) to share the bytecode
> writer with the compiler.
>
> Regarding the optimization, I'd suggest to keep it simple in the
> beginning. I liked the idea in Stefan's compiler to have a parser
> which produces an abstract intermediate representation, which is then
> optimized to generate the bwriter_shader. The optimizer could then be
> implemented from scratch, or using something third-party as LLVM or
> such, while as a first step the optimizer could (ideally) simply spit
> out the code unchanged. I believe that some language features cannot
> be supported without some "optimizations", so this problem can crop up
> sooner than expected.

That's the direction I wanted to take indeed. The optimizer should at
some point optimize according to the flag setting. At first, I would
only create test cases with D3DXSHADER_SKIPOPTIMIZATION.

>
> Lastly, a bit on testing the compiler. I'm not sure trying to get
> exactly the same bytecode as native is a feasible objective: while for
> an assembler program there is only one correct translation, this is
> not true for a compiler. Moreover I expect each iteration of the
> Microsoft compiler could produce a different output with the same
> shader program, so there isn't a "right" implementation to compare
> with. I agree, this makes testing the compiler much harder...
>

I think that with a lot of testing, the logic behind can be figured
out. Then the compiler can be built around this logic. This could
possibly even work for a lot of optimizations.

On Wed, Apr 14, 2010 at 5:44 PM, Roderick Colenbrander
<thunderbird2k at gmail.com> wrote:
> My main worry is that in the end if we use a compiler framework
> (remember we don't have a lot of manpower for this) I think it would
> reduce in more efficient shaders and it prevents some reinventing of
> the wheel.

I'm still not convinced it's worth the effort. Also, I think that a
parser/compiler written in lex/bison is much easier for other
developers to understand and maintain once it is in the tree (And
probably also a lot easier to review patches).