Jan 11, 2019

Some Smartphone Manufacturers Caught Cheating On Benchmarks Last Year

Gaining the extra edge has been the theme for the smartphone market as even the smallest edges have been known to convince consumers on what device to buy. In an effort to gain an edge over the rest, some smartphone makers have resorted to tricky tactics.

2018 was a great year for the smartphone market as a number of newer and faster smartphones have been released by smartphone manufacturers. With the better performance of these new smartphones also comes better price points that make new devices more affordable for everybody.

With the latest breakthroughs also comes a bit of shade though, as some smartphone manufacturers have been caught cheating on benchmark results just so they can get that extra edge against their competitors by putting up a front of invincibility.

Experts say that OEMs usually resort to "benchmark optimization" or tweaking their smartphones' hardware and software to get the highest result possible whenever a benchmarking software is being run. What we end up seeing are what go beyond realistic daily usage patterns, giving the consumers a not-so-realistic picture of the smartphone's actual performance.

To get a better idea of how bad the situation is, some journalists worked together with the folks behind benchmarking software Geekbench, with the software company even coming up with a clocked version of the software undetectable by smartphones just so it can detect any potential anomaly.

At least six different smartphone models have been caught cheating on their numbers during this investigation, though it is important to note that not all phones have resorted to dirty tactics. For perspective, the smartphone brands OEMs caught fudging the numbers were Xiaomi, HTC, OPPO, HUAWEI, and Honor. The biggest anomaly discovered during this exercise were devices from HUAWEI and Honor, the P20 Pro and the Honor Play, with both devices showing a performance gap of 21% between optimized performance and ordinary performance.

The snapshot below may give you a better idea of the extent of the cheating. The yellow bars in the single-core performance benchmarks represent scores achieved in optimized mode, while the blue bars represent performance in stock mode.

The multi-core test results show almost the same trend:

While it's great that smartphones are now able to reach high levels of performance like never before, the consumer deserves to get a performance that is consistent across the board and not just gets the promised amount of computing power only when running a benchmark.

Thankfully, not everyone is into cheating. Samsung, Sony, and Vivo smartphones were observed to have consistent performances all throughout. Also worth noting is that OnePlus, who was initially caught cheating in the past, has abandoned this notorious practice and was also consistent during this process.

When choosing a new smartphone, it is recommended to not only base the choice on benchmark scores but also on actual usage and reviews by experts. We have an extensive database of smartphone reviews from actual users here at Priceprice.com, so be sure to check those out if you're in the market for a new smartphone.