After watching the video the incompetence of Principled Technologies is staggering. I'm not sure that there were any shenanigans listening to Bill, just poor testing methodology. Some of the issues Steve had were debatable really with regards to is a gamer who wants to get playing games asap going to worry about, but some issues were just terrible decisions. Hopefully they take this as a lesson going forward.
What I do agree with Bill about is the reaction. There should have been a reaction, but it seems as though there were people that were waiting for the release specifically wanting something to be wrong so they can kick off (as they probably wait for any company they have some beef with so they can go loco). On this forum for example the level of outrage gives the impression that there is a lot more to it for those people than just poor methodology. That's the whole rabid fan stuff I guess. Even with 30 odd years of PC enthusiasm it's still just a CPU. Also people like Steve thrive off this hysteria for views which is also an issue as I think it clouds things for them.
Hopefully PT accept the issues and get better in the future.
I seen it differently.
Things came to light in this video its clear that GN (and probably other reviewers) went to great length to maximise AMD's results in their reviews, something which I previously complained about. This interview is as much as about maintaining their own credibility as anything else.
One example is that he revealed in GN testing on ryzen2 chips he pushed fan's to 100% instead of AUTO, that I do not remember been disclosed in his reviews, and the average user is not going to do that, especially when noise is important. He openly admitted when explaining his reason in that it has impact on bench testing results.
Another thing I moaned about, is most of the reviewers were using intel stock configuration, against ryzen2 scores, which to me was wrong, as they were putting in effort to maximise ryzen2 performance but decided to not manually OC the K processors on comparison scores as well as disable MCE.
a lot of people have a bias towards AMD as they feel they deserve it as an underdog with better priced chips, but to me reviews should be done in a fair manner and GN has clearly not been carrying out tests in this way in my opinion.
He raised the point about not using the same exact GPU for testing to remove variability, but I think unless PT deliberately gave the AMD rigs the worst binned GPUs I have no complaint, He mentioned that in the most extreme cases there is a 5% swing in performance, but if the test is not GPU bottlenecked for most of the test, then the impact on the test itself shouldnt be anywhere near 5%. I accept his point but I dont see it as a major one. The GPU silicon lottery e.g. would not cause the impact we seen on the graphs only probably 1-2% of it.
I am still watching the video so I may edit this post as I learn more. Edits will be at bottom.
On the cooler, his reply was rationale, he is imitating an average gamer not a high end gamer, and the AMD chips come with cooler so they get used, intel does not so an after market cooler "has" to be used. I accept that answer personally.
ON the ram,xmp issue. Steve had a point when asking why ram was XMP enabled but then downclocked but I also feel the answer returned was reasonable, the bit on the secondary and tertiary timings been board, chipset and bios dependent for reliability I feel steve wasnt been realistic, the vast majority of gamers wont be manually tuning timings. This video is really interesting as it shows how unrealistic reviewers can get. Info on this video that isnt disclosed in reviews. Also there is no rule stating that all reviewers should be like robots testing in the exact same way. e.g. lets say you got 10 AMD systems, they all run at 2933 ram speed, but 6 run at 3200, and 2 run at 3600. A reviewer like GN I expect would use the binned parts and post a test result at 3600 to show it in its best light, that to me is flawed, a better result is what ALL chips can do in that respect.
On the 64gig ram, yeah that was a silly config, as thats not a typical ram setup, I am definitely happy 4 dimms were tested as reviewers time and time again only test 2 dimms, but I agree with steves point pretty much 100%, given them ram was not at very high frequencies anyway, for both platforms I dont think this amount of ram gimped AMD tho, it was just an unusual config.
Game mode, the big one. His answer to me was logical. Threadripper which steve accepted gaming mode on is the only real choice. The 2700x, apparently works better with it on in some games and others with it off, so more variable, GN would probably toggle it to get best results for every game (imo unrealistic), more likely a user would leave it off all the time or on all the time, its off by default. I feel on all the time or off all the time are both valid ways to test. I feel toggling it for each specific game is "not" realistic. GN seem to be trying to hide deficiencies in their reviews, whilst I feel a review should highlight them instead.
FF15 may be a coding mess, but it remains valid, as there is people who play that game, only testing optimised games again in my view is unrealistic.
I know my post will get blasted but you guys know I havent been happy with the review scene for a while, and I think this set of benchmarks was done using a differet testing viewpoint that is closer to my way of thinking and not what has been accepted as the industry standard and of course with AMD not looking as good a lot of people have thrown their hands in the air.
One reviewer hardware unboxed actually already validated the results as been accurate for the way they was tested, just it was a different testing methodology used.
