Benchmarking D3D?

[Stefan Dösinger]

Am 18.04.2010 um 04:57 schrieb Dan Kegel:
> At the moment, my plan is to simply dedicate a machine
> and script a daily run of a couple benchmarks with +fps,
> and upload the resulting framerates to a web page
> so we can notice if somebody checks in something that
> slows them down.
> - Dan
I still have my cxtest scripts and php hackerish that runs a bunch of benchmarks, and I'm still running them. Unfortunately this stuff is a pain to maintain, for various reasons.

I'm testing 3dmark2k, 3dmark2k1, ut2004, halflife2 and team fortress 2. I am also running the Wine tests. The big requirement for the benchmark games is that I have a way to automatically read the results from the games. Running with +fps and collecting the output is imprecise IMO.

The main problems are:

*) Maintainance. The tf2 timedemo format changes every now and then, and similar issues
*) The way I wrote the code. Very hacky.
*) The test machines. I could use a server farm, but I don't have any. The boxes are standing in my living room.
*) System updates(ubuntu) need to be installed and can affect performance.
*) Unreliable apps. The steam login fails every now and then, requiring manual intervention.