Mass testing/benchmarking

[Aruffell]

Got about 5 graphics cards here. 4 AMD and 1 nVidia and i need to test them all on the same motherboard with all the same components.

The only difference will be the graphics card.

OS will be Vista 64-bit and im wondering what the best way of testing would be?

Obviously i will be using the same benchmark tools and games but don't want to have to install Vista 5 times.

Best way being to install Vista, don't load drivers, install benchmarks and games and then make an image and then install drivers, then reload image and rinse, lather, repeat?

Andy

 

[Aruffell]

Anyone? Need to know asap.

Andy

 

[ViRuS2k]

No one has replyed becuse its a pritty noobie thing to ask lol
obviously you install one card install the drivers then run the benchmarks

uninstall drivers shutdown system, install next card go into windows install drivers run test rince and repeate its as simple as that.

cant be to difficult lol

 

[Aruffell]

Surely there will still be traces of the driver though, deeming an unfair test. Surely?

Andy

 

[Lanz]

Nah, it should be fine.

If you were doing the test as a professional job for a mag or website, then maybe you'd do all the imaging stuff, but for standard testing, an uninstall of drivers should be fine.

I've been through 4 cards (2 Nvidia and 2 ATI) on this current build of Vista I have, and never had a problem.

 

[HangTime]

"traces of the driver" is a bit of an old wives tale anyway, in practice it's very rare that they will have any effect at all (usually in cases where settings have been changed from default and are retained somewhere).

Windows is intelligent enough to handle multiple hardware/driver installations, heck, that's why you can even run with graphics cards from different manufacturers installed at the same time (not that I'm suggesting you do so for benchmarking!).

I'd be more concerned about what you actual benchmarking process is, i.e. you need to make each benchmark a consistent, repeatable process (as opposed to say a run-through with fraps in the background) and do multiple runs for each benchmark, discarding any anomolies.

 

[Aruffell]

Nah, it should be fine.

If you were doing the test as a professional job for a mag or website, then maybe you'd do all the imaging stuff, but for standard testing, an uninstall of drivers should be fine.

I've been through 4 cards (2 Nvidia and 2 ATI) on this current build of Vista I have, and never had a problem.

Guess what, i'm doing the test as a professional job for a website

"traces of the driver" is a bit of an old wives tale anyway, in practice it's very rare that they will have any effect at all (usually in cases where settings have been changed from default and are retained somewhere).

Windows is intelligent enough to handle multiple hardware/driver installations, heck, that's why you can even run with graphics cards from different manufacturers installed at the same time (not that I'm suggesting you do so for benchmarking!).

I'd be more concerned about what you actual benchmarking process is, i.e. you need to make each benchmark a consistent, repeatable process (as opposed to say a run-through with fraps in the background) and do multiple runs for each benchmark, discarding any anomolies.

I'll be running each benchmark 3 times, maintaining the same settings for each at 2 resolutions. One being 1280x1024 and one being 1920x1080.

Just trying to figure out which tests to run.

Got:

3DMark06
3DVantage
Crysis Warhead
FarCry 2
Was told to do something lower end like company of heroes.

Any other suggestions?

Andy

 

[Alexrose1uk]

From what you recommend as doing it for a semi-pro site, I'd say get some imaging software, get the computer completely setup WITHOUT any graphics drivers installed (windows defaults at worst) so basically fresh install of windows new drivers, a few games/any relevant updates etc. Make an image from this, it won't be huge, so shouldn't take TOO long.

Keep the latest drivers on a USB stick, then literally flash a machine with the 'raw' install. Install your card and it's drivers, run the tests x times to get the averages and figures you need, remove the card, re-image the machine again, insert new card, install it's drivers...

Rinse and repeat until done and you have all the figures you need. Bit of an arse to do, but using an image does mean you'll be going off exactly the same base each time, bar graphics drivers and card.

The windows section should be able to point you to the best imaging software at the moment, it used to be Acronis TruImage and Symantec Ghost, but this may have changed since I last looked!

Games wise I'd suggest testing:
<Bar the usual 3dmark stuffs>
Crysis Warhead
Crysis Vanilla
Farcry2
Arma2
STALKER
Unreal Tournament3/Mass Effect
Half-Life 2: Ep 2
Doom 3

Will give you wide range of tests, and utilises most of the graphics engines in major use

 

[Ducky Spud]

For lower end tests ive seen quite a lot of people use GRID and COD4 simply because theyre pretty damn popular and what people want to see fram rates for.

 

[[timko]]

Something like this would be good...

You might want to consider squeezing in 1680 resolution benching as well since a lot of users are still on 1680 x 1050 resolution monitors.

 

[gt_junkie]

Yep, don't worry about drivers.

It just takes a lot of time going through each game/bench, each resolution (i tend to use 1280x1024, 1680x1050 and 1920x1200) and each card.

Only testing 3 cards over 3 resolutions with 9 games and a few benches you easily reach over 100 tests.

Pretty hard going.

The layout timko posted is a good guide.

gt

 

[HazardO]

Make sure your game selection doesn't unfairly favor Nvidia or ATI (or visa versa). Crysis for instance loves Nvidia.

 

[AMG]

yeah but they still use it for reviews and benhics and you don't get too bad results from it ethoir.

thats why you test multiple games