Performance comparison initiative

[yellonet]

Something that should be in the interest for a lot of people would be a comparison of performance between different hardware configurations when running the game with the same settings.

This would be beneficial for:
People looking to invest in new hardware for this game
People looking to maximize performance on their existing hardware
People who want to know if that new patch gave better or poorer performance

What I propose is that we agree (yes I know :lol:) on three different graphical settings that are used, low, middle, max, and use them as we run a specific mission as viewer only, then we only need some software - unless it can be done in-game - that can create some statistics for the FPS, min, max and average.
The mission could be created, but for simplicity it might be easier to use one of the stock missions.

I don't know how to best go about the FPS measurement, anyone have a suggestion?


The result could look something like this:

----------
1c forum CloD test result

i7-860 @ 2.8 GHz
4 GB RAM
ATI 4870 1 GB (Catalyst 11.7)
W7 64-bit

CloD version: 1.02.14821

Low settings
Min: 24 fps
Max: 148 fps
Avg: 66 fps

Medium settings
Min: 18 fps
Max: 92 fps
Avg: 48 fps

High settings
Min: 9 fps
Max: 44
Avg: 16 fps
----------

The critical bit will be to agree to which settings to use for each "performance level" and to get reliable statistics.
It would be best if everyone used the the same graphics driver but that won't happen so we'll have to make due with just specifying which driver was used in the test.

I know that the game is perhaps not ready in terms of performance tweaking, but any performance improvements should be of a general nature, and relative numbers between hardware might still be valid.

[FFCW_Urizen]

We could use the "Black Death" track in conjunction with fraps. that should be sufficient.
a definite must have:
-CPU (i.e. Intel Core 2 Quad Q6600 @ 2.4 GHz)
-RAM (i.e. Corsair Dominator DDR2-8500 @ 1100 MHz 5-5-5-15)
-GPU (i.e. ATi/AMD Radeon HD5870 @ 900/1300 MHz running with Catalyst 11.5)
-OS (i.e. Windows 7 Home Premium 64Bit with Service Pack 1)
-PCIe Version (1.0, 2.0 or 3.0 @ 105 MHz and yes that can have an impact ;) )

Test may be run (or not) with forced AF at least at 8x, 4xMSAA (if supported, if not no MSAA/MLAA/CFAA/SSAA)

any further suggestions?

So Long
Uri

[yellonet]

We could use the "Black Death" track in conjunction with fraps. that should be sufficient.
a definite must have:
-CPU (i.e. Intel Core 2 Quad Q6600 @ 2.4 GHz)
-RAM (i.e. Corsair Dominator DDR2-8500 @ 1100 MHz 5-5-5-15)
-GPU (i.e. ATi/AMD Radeon HD5870 @ 900/1300 MHz running with Catalyst 11.5)
-OS (i.e. Windows 7 Home Premium 64Bit with Service Pack 1)
-PCIe Version (1.0, 2.0 or 3.0 @ 105 MHz and yes that can have an impact ;) )

Test may be run (or not) with forced AF at least at 8x, 4xMSAA (if supported, if not no MSAA/MLAA/CFAA/SSAA)

any further suggestions?

So Long
Uri

Perhaps we should agree on a level of hardware information that is easily attainable through CPU-Z, because everyone will not know exactly what memory timings they have and so on.

CPU: Model + frequency
Main board: Brand + model
Memory: Size + frequency + timings
GPU: Model + VRAM size + driver version
OS: Name + 32/64-bit + build number
Version of CloD

Obviously one could give more info, but this could be the minimum.

[Pudfark]

This is a good idea that would benefit the community.
However, I think?
It should be implemented after the release of the next patch.
I would be willing to contribute to this positive endeavor.

[yellonet]

This is a good idea that would benefit the community.
However, I think?
It should be implemented after the release of the next patch.
I would be willing to contribute to this positive endeavor.

It doesn't really matter when we start as long as we write down which version of the game we used in the tests.
The sooner the better I'd say.

[Pudfark]

Ok...agreed.

[FFCW_Urizen]

@yellonet: ram timings can be easily obtained using CPU-Z, u just have to look under the Memorytab ;) . And for the mobo, i´d say chipset is enough, as the differences are marginal at best, thats why i didn´t include it.

[lothar29]

I think it would be interesting to say about stuttering, as to my going with some more or less decent FPS but I have a horrible stuttering in Earth...impossible to fly...

CPU- I7 950 @3.6GHz
Mother- ASUS P6T Deluxe V2
RAM- Corsair Vengance3x2Gb/1600MHz/9-9-9-24
GPU- 2xGTX 470 @608/GDDR5 1280Mb/275.33---- The second GPU this would give me very bad performance with the CoD
OS- Windows 7 Home premium x64
PSU-Nox Apex 800W modular

http://gpuz.techpowerup.com/11/08/02/gwy.png

High settings
Min:15 fps
Max:60 fps
Avg:29 fps


The configuration, commissioning the maximum less effects in half and trees is low, and ORIGINAL...

Then I would put a picture of the NVIDIA control panel

[yellonet]

@yellonet: ram timings can be easily obtained using CPU-Z, u just have to look under the Memorytab ;) . And for the mobo, i´d say chipset is enough, as the differences are marginal at best, thats why i didn´t include it.I know, I have it in the list in my previous response to you.

Black Death seems to be a good track to use too.

Suggestions for settings are as follow.

Low (In-game "Medium" setting @ 1280x1024)

http://img.photobucket.com/albums/v332/yellonet/CloD_low.jpg

Med (In-game "High" setting @ 1680x1050)

http://img.photobucket.com/albums/v332/yellonet/CloD_med-1.jpg

High (In-game "Very High" setting @ 1920x1200)

http://img.photobucket.com/albums/v332/yellonet/CloD_high.jpg


I realize that not everyone has monitors that supports these resolutions, so if someone runs tests at different res they must clearly report this in the results.

[yellonet]

I think it would be interesting to say about stuttering, as to my going with some more or less decent FPS but I have a horrible stuttering in Earth...impossible to fly...

Yes of course it's encourouged that people testing give their impressions, micro stutters are hard to measure so that would have to be written as example "sometimes a few micro stutters" or "heavy microstutters at all times". The more information one can give about the performance, the better.

But what we have to start with is to agree on the formal measurable tests so that everyone tests at the same three settings.

[FFCW_Urizen]

for res, i would suggest a resolution every monitor is able to display, my tft only goes up to 1680x1050 for example, so i´d suggest a max resolution of 1440x900 or even just 1280x1024 (which every tft should be able to handle).

For Comparison reasons we should use the presets shipping with CloD, VSync off, AEF off, SSAO on/off.

[yellonet]

for res, i would suggest a resolution every monitor is able to display, my tft only goes up to 1680x1050 for example, so i´d suggest a max resolution of 1440x900 or even just 1280x1024 (which every tft should be able to handle).

For Comparison reasons we should use the presets shipping with CloD, VSync off, AEF off, SSAO on/off.

It would be ideal to use a low resolution that everyone could set, but then we wouldn't get very meaningful results as people tend to want to play especially simulations at high resolution. As you drop the resolution, the CPU becomes more and more important and the GPU less important for high FPS.
Perhaps if it is not too much to ask people with high resolution monitors could run the tests at lower resolution too.
I have a 24" screen with 1920x1200, I could make the tests in that resolution and several lower resolutions as well.


Edit: Anyway, we need more opinions so that we can have as wide an agreement as possible.

[FFCW_Urizen]

then we should go with your first suggestion of resolution and related detailsettings.

So Long
Uri

[yellonet]

Fraps results... what am I doing wrong?

Frames, Time (ms), Min, Max, Avg
15, 60000, 0, 14, 0.250

[yellonet]

I take it that no one is interested in this?