Mantle vS 780Ti in BF4

Gregster · 6 Feb 2014 at 06:28

So now the dust has settled, review sites have had a chance to take in Mantle and TheTechReport tested the XFX 290X against a reference 780Ti and used a couple of processors from a 4770K to a A10-7850K. It was only when the 7850K was paired with the 290X, did it outshine the 780Ti and shows that Mantle is good for relieving the CPU bottleneck. It seems even with the reported 8% performance gains from using Mantle, it still wasn't enough to beat the 780Ti in their findings on a 4770K CPU.

http://techreport.com/review/25995/first-look-amd-mantle-cpu-performance-in-battlefield-4/2

Battlefield 4
We tested Mantle performance versus Direct3D in Windows 8.1 using a couple of different processors, a Kaveri-based AMD A10-7850K APU and an Intel Haswell-based Core i7-4770K. The idea was to test in a CPU-constrained performance scenario using two processors with different levels of performance. In fact, I had hoped to show a lower level of CPU performance by including another AMD APU, a 65W Richland-based A10-6700. However, its performance turned out to be almost identical to that of the 95W Kaveri 7850K, so I held it out of our final results in order to keep things simple.

The main video card we used was a Radeon R9 290X card from XFX. This 290X comes with a custom cooler and sustains its peak Turbo clock almost constantly, even under the heaviest of loads. It essentially eliminates the clock speed and thermal variance issues we've seen with stock-cooled 290X cards. (I'll be writing more about this card soon.) To ensure the GPU wasn't the performance constraint, we tested BF4 at 1920x1080 on the "high" image quality presets, which is fairly easy work for a video cards of this power. We also tested at these same settings using a GeForce GTX 780 Ti, in order to see how Nvidia's Direct3D driver fares compared to AMD's D3D and Mantle implementations.

The known issue with occasional stuttering rears its head in one plot, for the 4770K with Mantle. You can't see the full size of the frame time spike on the plot, but it's 295 milliseconds—nearly a third of a second. We didn't see this sort of hiccup all that often, but it did happen during some test runs, including the one we plotted for the 4770K.

AMD has made some big claims for performance improvements from Direct3D to Mantle, and the numbers from the A10-7850K appear to back them up. The leap from an average of 69 FPS to 110 FPS is considerable by any standard, particularly for an API change that apparently produces the same visuals. Even better, our latency-focused metric, the 99th percentile frame time, tends to agree that Mantle is substantially faster than D3D in this case. Mantle also outperforms Direct3D in combination with the Core i7-4770K, but the differences aren't quite as dramatic.

One thing we didn't expect to see was Nvidia's Direct3D driver performing so much better than AMD's. We don't often test different GPU brands in CPU-constrained scenarios, but perhaps we should. Looks like Nvidia has done quite a bit of work polishing its D3D driver for low CPU overhead.

Of course, Nvidia has known for months, like the rest of us, that a Mantle-enabled version of BF4 was on the way. You can imagine that this game became a pretty important target of optimization for them during that span. Looks like their work has paid off handsomely. Heck, on the 4770K, the GTX 780 Ti with D3D outperforms the R9 290X with Mantle. (For what it's worth, although frame times are very low generally for the 4770K/780 Ti setup, the BF4 data says it's still mainly CPU-limited.)

The "time spent beyond X" graphs are our indicator of "badness," of how long frame production times exceed several key thresholds. Those intermittent stuttering episodes with the early Mantle driver show up in the beyond-50-ms results for the A10-7850K, even though we didn't see a hiccup of this size in every run. Since we're showing the median result from three runs, the spike we plotted for the 4770K doesn't show up at all here. (There were no such spikes in the other two test sessions.)

The big takeaway here comes from the "time spent beyond 16.7 ms" plot. You need to produce a frame every 16.7 milliseconds to achieve a smooth 60-FPS rate of animation. Mantle moves the A10-7850K much, much closer to that goal, even with that one big latency spike in the picture. If AMD can eliminate those hiccups, then slower CPUs like the 7850K should be capable of delivering a much smoother gaming experience than they can with Direct3D.

Conclusions
These are still early days for Mantle, but we can already see its ability to reduce CPU overhead rather dramatically compared to Direct3D. That's exactly the sort of innovation folks have wanted to see in PC gaming, and AMD and DICE are already delivering. One would hope this demonstration of a more modern approach to graphics programming would spur others (ahem, Redmond) to innovate in a way that can benefit the entire PC ecosystem.

There's lots of work yet to be done on Mantle. AMD needs to refine its drivers, add some key features, and improve performance scaling for its older GCN-based graphics chips. Meanwhile, in order for Mantle to really gain traction, EA and DICE will have to follow through on their promise to bring the Mantle rendering path to a host of other games based on the Frostbite 2 engine.

Based on these first results, the big beneficiaries of Mantle's proliferation will probably be folks who, for one reason or another, have a PC that isn't built to perform especially well in many of today's games. PCs with slower processors stand to gain the most.

That said, there are already some well-worn paths to very good gaming experiences on the PC today. The Haswell-based Core i7-4770K is faster than the A10-7850K regardless of the graphics API. Switching from AMD's Direct3D driver to Nvidia's will get you more than halfway to Mantle's performance on an A10-7850K, too. AMD would do well to work on improving its Direct3D drivers and CPUs, as well as pursuing Mantle development—but I'm sure they already know that. I'm happy to see AMD pushing innovation in graphics APIs at the same time.

We'll surely test Mantle's performance on a broader range of CPUs as it matures. I'm curious to play around with different core counts and to see whether low-power chips like Kabini can provide good gaming experiences with Mantle. Our next task, though, will be to see what performance benefits Mantle can deliver in GPU-limited scenarios. Stay tuned for that.

andybird123 · 6 Feb 2014 at 08:01

Doesnt exactly look like the 780ti is being ridiculed

JediFragger · 6 Feb 2014 at 08:06

andybird123 said:
Doesnt exactly look like the 780ti is being ridiculed

Ti is a monster

Waaaaaaant!!

On another note, just HOW poor is that A10!!! Jeez

Marine-RX179 · 6 Feb 2014 at 08:07

It's rather impressive how much gain there is for the lowly A-10 7850K (on CPU performance term).

To be honest, I have always thought that the biggest benefit for Mantle would be if RTS and MMOs makers were willing to take advantage of it. Getting really sick of getting 30fps with even a highly overclocked i5, and upgrading to the latest i7 won't help much.

Back to BF4...considering the GTX780Ti and 290x are getting over 120fps, I'm guessing they were testing using singleplayer for testing and is not really CPU limited? I'm not sure if that's really a good representation of the benefit of Mantle...I mean if the same setting was used for multiplayer 64 players map, the difference would may be that the i7 4770K with GTX780Ti will get dips down to 70-100fps occassionally (due to CPU bottleneck), whereas as the i7 4770K with 290x+Mantle could hold 100fps+?

Also I'm not sure too sure about how accurate is their results for the frame rate as it is not really a fix benchmark, considering the part which they said "We captured performance info while playing through a two-minute-long section of BF4 three times on each config." Wouldn't the average frame rate have been easily affected depending of where they look on each runs?

Gregster · 6 Feb 2014 at 08:07

andybird123 said:
Doesnt exactly look like the 780ti is being ridiculed

Well, it was actually the Titan and 780 they were talking about.

“[We] will definitely compete with the GTX 780 and Titan” ... “with Battlefield 4 running with Mantle (AMD’s new graphics API), the card will be able to ‘ridicule’ the Titan in terms of performance.”

http://www.cinemablend.com/games/AMD-Says-R9-290X-Ridicule-Nvidia-Titan-Performance-59418.html

It would be good if a few more ran the test that Matt has set up to see if the Titan is being ridiculed, as from what I have seen, my Titans are taking names and kicking ass

andybird123 · 6 Feb 2014 at 08:13

andybird123 · 6 Feb 2014 at 08:17

Marine-RX179 said:
It's rather impressive how much gain there is for the lowly A-10 7850K (on CPU performance term).

To be honest, I have always thought that the biggest benefit for Mantle would be if RTS and MMOs makers were willing to take advantage of it. Getting really sick of getting 30fps with even a highly overclocked i5, and upgrading to the latest i7 won't help much.

Back to BF4...considering the GTX780Ti and 290x are getting over 120fps, I'm guessing they were testing using singleplayer for testing and is not really CPU limited? I'm not sure if that's really a good representation of the benefit of Mantle...I mean if the same setting was used for multiplayer 64 players map, the difference would may be that the i7 4770K with GTX780Ti will get dips down to 70-100fps occassionally (due to CPU bottleneck), whereas as the i7 4770K with 290x+Mantle could hold 100fps+?

Also I'm not sure too sure about how accurate is their results for the frame rate as it is not really a fix benchmark, considering the part which they said "We captured performance info while playing through a two-minute-long section of BF4 three times on each config." Wouldn't the average frame rate have been easily affected depending of where they look on each runs?

Only if they spent a significant amount of time on each run doing something drasticaly different like staring at the floor, even over 2 minutes it should be relatively easy to get a representative sample, particularly as they did 3 runs each

LtMatt · 6 Feb 2014 at 08:19

Marine-RX179 said:
It's rather impressive how much gain there is for the lowly A-10 7850K (on CPU performance term).

To be honest, I have always thought that the biggest benefit for Mantle would be if RTS and MMOs makers were willing to take advantage of it. Getting really sick of getting 30fps with even a highly overclocked i5, and upgrading to the latest i7 won't help much.

Back to BF4...considering the GTX780Ti and 290x are getting over 120fps, I'm guessing they were testing using singleplayer for testing and is not really CPU limited? I'm not sure if that's really a good representation of the benefit of Mantle...I mean if the same setting was used for multiplayer 64 players map, the difference would may be that the i7 4770K with GTX780Ti will get dips down to 70-100fps occassionally (due to CPU bottleneck), whereas as the i7 4770K with 290x+Mantle could hold 100fps+?

Also I'm not sure too sure about how accurate is their results for the frame rate as it is not really a fix benchmark, considering the part which they said "We captured performance info while playing through a two-minute-long section of BF4 three times on each config." Wouldn't the average frame rate have been easily affected depending of where they look on each runs?

They used Medium/High settings.

The question you ask yourself is who spends £550 on a high end gpu to play at lower details? I don't think many people buy a 780TI/290X to play at 1080P Medium/High settings. I think Nvidia's D3D driver is better than AMD's in this regard with multi threaded rendering, it helps at low res/low details. Its been the case for a while to be fair. Nvidia typically faster at 1080P or lower, but AMD faster at 1440P or higher. Its not a complete science, but its been that way for a while.

Always worth remembering what happens when you play at Higher details/Higher resolution/Ultra Preset though. The tables turn dramatically. I know which one id prefer, given the choice.

I will forward those results onto AMD though and ask for their opinion on the cpu over head of their D3D driver vs Nvidia's.

K1LaW8tt5 · 6 Feb 2014 at 08:22

Benchmarked at high settings with no AA.

Gregster · 6 Feb 2014 at 08:33

weldon855 said:
Benchmarked at high settings with no AA.

That seems to be a theme with a lot of the review sites and Mantle for some reason. I picked up on it yesterday and said it was a daft way of testing. We need to see more people joining in the Mantle bench thread or more review sites doing ultra testing with top end cards.

Besty · 6 Feb 2014 at 08:33

If they had run with higher settings Mantle would not have demonstrated a lower increase. We already saw this done at anandtech, in gpu limited situations Mantle in bf4 gives single digit increases.

The techreport are the best, most thorough and impartial site that is out there.

Nobody does this stuff as good as them, I would imagine that this is just the first article of a bunch that they will do on mantle.

It's a glass half full moment, I would avoid concentrating on the 290x numbers, ignore Roys previous comments and just look at how much of a boost Amd CPUs are getting. This is really great news in some respects for Amd as the battle is more and more I the living room with smaller boxes and not people like us.

uksoldierboy · 6 Feb 2014 at 08:34

I didn't think it was possible to see such a huge amount of BS/misleading information in one set of graphs..

triss · 6 Feb 2014 at 08:36

always niggles me most benchmarks (not having a go at you LT) show a non reference against a reference ,most of these reviews have tested aftermarket cards from both sides and i would have thought it would be easy to include them

Gregster · 6 Feb 2014 at 08:36

uksoldierboy said:
I didn't think it was possible to see such a huge amount of BS/misleading information in one set of graphs..

But even Matts graphs only use 2xMSAA for some reason. Why don't they set Ultra and go?

Matt is demonstrating what he is lambasting.

LtMatt · 6 Feb 2014 at 08:40

Gregster said:
That seems to be a theme with a lot of the review sites and Mantle for some reason. I picked up on it yesterday and said it was a daft way of testing. We need to see more people joining in the Mantle bench thread or more review sites doing ultra testing with top end cards.

It makes sense for Low-Mid Range cards for sure. High end cards? It makes no sense at all.

Gregster said:
But even Matts graphs only use 2xMSAA for some reason. Why don't they set Ultra and go?

Matt is demonstrating what he is lambasting.

The difference in image quality between 2x and 4x at 1440P is not very big. 2x MSAA at 1440P is definitely playable on some cards, 4x would not be playable on any. In this situation, it makes perfect sense.

Diagro · 6 Feb 2014 at 08:41

Why make this thread when you know the test was BS

Besty · 6 Feb 2014 at 08:44

Setting everything to ultra would have capped the mantle increases at around 8%, dropping the slider allows for mantle to stretch its lets a little.

I wanted then to test with 200% oversample but that does not mean I try and debunk their findings.
The two best sites (Anandtech and tech report) have both published their findings and have concluded similarly.

Ian Evey · 6 Feb 2014 at 08:44

Diagro said:
Why make this thread when you know the test was BS

Probably because its the same graphs as every other BS graphs out there since mantle was released, We cannot have some BS graphs without other BS graphs now can we, that would show bias.

Gregster · 6 Feb 2014 at 08:45

Diagro said:
Why make this thread when you know the test was BS

I am reporting news from a well respected tech site and as I also like nVidia, I thought it was some good news to share with nVidia users. Sorry if you didn't like it but there is plenty of pro AMD threads

Tonester0011 · 6 Feb 2014 at 08:46

Don't know why there is outrage because they used lowered settings, mantle helps to alleviate a CPU bottleneck, ultra would minimize the potential gains...

also worth noting I dont care that much either way lol.