AMD accuses BAPCo and Intel of cheating with Sysmark benchmarks

[Gregster]

AMD threw out a bombshell and accused its rival Intel and BAPCo, the benchmarking consortium, of cheating.

In a video posted Thursday on Youtube, John Hampton, director of AMD’s client computing products, went so far as to refer obliquely to the recent Volkswagen scandal, where the German car manufacturer was accused of cheating on diesel emissions tests. “The recent debacle over a major auto maker provide the perfect illustration as to why the information provided by even the most established organizations can be misleading,” Hampton said.

Intel declined to comment on AMD’s accusation, but when asked BAPCo officials said its customers trusted it.

"The reason thousands of customers trust BAPCo benchmarks is because we are an industry consortium that focuses on the performance of applications that people use on a daily basis," a spokesman for the consortium said.

Why this matters:
Performance still matters to consumers and organizations. Third-party benchmarks hold heavy sway over purchasing decisions even if few understand what they measure. AMD asks reasonable questions, but the answers remain murky—even from AMD.

http://www.pcworld.com/article/3023...ntel-of-cheating-with-sysmark-benchmarks.html

AMD are up against it again it seems. The good guys always come last

 

[Jono8]

Article needs to be more specific really. Which i5 and which FX cpu did they use?

 

[Gregster]

Article needs to be more specific really. Which i5 and which FX cpu did they use?

It is all quite murky after reading the article and watching the video. You are quite correct as well and they do need to be more specific but generally, they are saying that one bench gives a 50% delta against their CPU and another 2 benches only give a 6/7% delta swing.

 

[mmj_uk]

I've never used Sysmark personally, it's pretty obvious in most other general applications and benchmarks that AMD processors lack performance when compared to Intel though so even if this claim is true what does it change?

 

[drunkenmaster]

They've always been biased, way way back when in a new version of the benchmark they removed literally every single one that ran faster on AMD than Intel and instead ran the Intel favouring ones more often to pump up the score.

Benchmarking is a joke, Intel's compilers actively turning off optimisations when detecting AMD hardware but more specifically, paying companies to use the Intel compiler.

Linux in a large portion of software for instance shows a noticeably smaller gap in performance between AMD and Intel.

I can't even remember which benchmark it is that was basically owned by Intel but pretended to be independent for years. This is besides the extremely obvious and literally proven cases of Intel paying companies to not buy AMD and spending(or discounting) 100's or millions if not billions to achieve it.

 

[h4rm0ny]

Well, Intel have been caught cheating before when they limited compiler optimizations to their own chips for (and this is the critical part) no reason.

The thing with benchmarking, is that the more complex an area is, the easier it is to make statements that are factually accurate, sound good, yet are misleading. And CPUs are very complex. It's as much about what you don't measure, as what you do.

 

[lsg1r]

As much as I'd like to see amd get a little more support this isn't going to help them with anyone tech savvy...
They go on about sysmark being cpu only.... where they get beaten and we all know it's true that amd fx is slower than intel clock per clock. An i5 with 4 cores will almost match an fx with 8 cores in most multi threaded tests and it's considerably faster in single thread too. Does sysmark even consider multithreading?

So they want to use pcmark which just happens to include the (I'll assume built in) gpu where we know amd are generally stronger. Pretty sure pcmark makes more use of multithreading too so that would bring them closer to equal terms.

Which is fine if that's where it actually matters and I'm sure AMD's in house custom coded office task script was designed to use multi threading as much as possible.

Having said that I'd rather see AMD actually release a 'new' product which can compete rather than be moaning about the accuracy of a benchmark.

 

[gpuerrilla]

Interesting post, but thanks Greg for finding this beyond the Graphics forum!

Whilst everone knows intel are the superior chip when it matters, educated folk will know deep down the big guys never play fair. I dont generally purchase anything soley on benchmarks though, people with deep pockets with a hint of ingnorance will.

 

[CAT-THE-FIFTH]

I remember reading something about a senior Intel chap(head of performance testing?) being on the board of directors of BAPco/another benchmark or something along those lines. Not sure if that was the case now.

 

[Dave2150]

I've never used Sysmark personally, it's pretty obvious in most other general applications and benchmarks that AMD processors lack performance when compared to Intel though so even if this claim is true what does it change?

Yep, fully agree, this doesn't change anything at all.

Intel Skylake/Haswell-E simply outperform AMD CPU's in every area, by a much larger than "7%" margin.

 

[CAT-THE-FIFTH]

Yep, fully agree, this doesn't change anything at all.

Intel Skylake/Haswell-E simply outperform AMD CPU's in every area, by a much larger than "7%" margin.

Yes,but it might be more significant with things like Zen,since it will only take a few percent either way for people to declare victory.

 

[Dave2150]

Yes,but it might be more significant with things like Zen,since it will only take a few percent either way for people to declare victory.

Sure, though remember Zen is a 2017 product. It hasn't even taped out yet.

Kabylake launches this year, as well as Broadwell-E. Then in 2017 we'll have 10nm Cannonlake - these will be Zen's competition.

Hopefully Zen will be a monster, but after the Bulldozer fiasco, I remain highly sceptical.

 

[CAT-THE-FIFTH]

But I would never trust Sysmark as loads of companies left it - not only AMD,but also Nvidia and VIA too. There was a whole lot of stuff on Anandtech forums about it I think and I believe webxprt(I think that was the name of it) also had some dubious links when you dug deeper.

I need to go and find the post/posts which I read a few years ago,but yeah Sysmark is not really worth even considering after that.

 

[AlamoX]

either tempring with the bench itself, or with api inserted into software, to advantage X chip over the competition, is really bad, it doesnt only decieve the user ability to objectivly qualify a chip to make a choice, but it also overstep on the morality of competitive market, these actions are becoming more and more common, and laws need to adapt, instead of leaving it in a grey area.

 

[faceman123]

if only AMD invested as much time in to their short comings....

 

[d_brennen]

X company cheats in benchmark shocker. Intel has been at this kind of thing for at least a decade.

The mobile space is even more lurid with cheating. It's said some SoCs detect when certain code is executed then it's ok to exceed power envelopes...

 

[CAT-THE-FIFTH]

if only AMD invested as much time in to their short comings....

Nvidia also left BAPCo like AMD and VIA did.

 

[Martini1991]

Bit rich coming from the vendor who released a promo video which lied about the competitors CPU's.

 

[CAT-THE-FIFTH]

Bit rich coming from the vendor who released a promo video which lied about the competitors CPU's.

I really need to find those posts which I read a few years ago. It was basically "the head of performance testing at Intel" or someone who used to work in that role who was on the board of directors of at least one of these benchmark consortiums. Its why it stuck in my head since it was truely WTF!!!!!!!

But considering that both Nvidia and VIA left there really is somewhat not going right with it. Intel is the only chip member still left on it.

Also,Sysmark is used in US and UK Government tenders.

 

[Martini1991]

I really need to find those posts which I read a few years ago. It was basically "the head of performance testing at Intel" or someone who used to work in that role who was on the board of directors of at least one of these benchmark consortiums. Its why it stuck in my head since it was truely WTF!!!!!!!

But considering that both Nvidia and VIA left there really is somewhat not going right with it. Intel is the only chip member still left on it.

Also,Sysmark is used in US and UK Government tenders.

I'm pretty sure there was an article which portrayed AMD in the exact same situation, and pushed X benchmark when they were top, etc etc.

Intel have no reason to lie about performance, what I'd be skeptical of is their power usage statistics etc, as that's what people will care about more (Big buyers, no one cares about enthusiasts)

Don't think there's a single AMD Desktop in use at the College, everything I deal with is either Core 2 or some I series.