Amazon made news earlier this month when CEO Jeff Bezos was asked to appear before the U.S. Senate to explain Amazon’s Choice badge and the methodology for awarding products this coveted title.
Our research shows that having an Amazon’s Choice badge increases product sales by 25%, on average. So, when a subsequent report by Digiday revealed that some brands had early access to bid and pay for securing their products among Amazon’s chosen elite, the negative response came as no surprise.
But while questioning Amazon’s business practices is certainty in vogue, I don’t believe closed bidding for favorable treatment is intrinsically evil. Between the retail quest for frictionless commerce and the present limitations of applied machine learning, it’s possible that Amazon’s move to drive profitability by soliciting brand spend does not disadvantage consumers. Here are my thoughts:
- Providing product recommendations improves the online shopping experience by reducing purchase friction.
- The size and success of Amazon’s marketplace in the U.S. is solely made possible through deployment of algorithms that rely on a constrained number of inputs.
- Featuring profitable products does not signal foul play; instead think of brand spend as a proxy for brand commitment to eCommerce success.
Paradox of choice
The negative reaction may be driven by the idealization of eCommerce as the great equalizer, where every product wins top visibility by merit alone. But having unlimited options is not idyllic. Providing too many options is worse than providing none.
This is known as the paradox of choice, and it’s one of the reasons why Amazon invests in product recommendations. Just how today’s Top 40 songs are getting more airtime despite a record number of independent artists, the top hits of search results get a disproportional share of clicks. Amazon runs 2,000+ optimization experiments per day because reducing purchase friction means shoppers buy more in less time. It’s quantifiably profitable: each 100 millisecond delay costs millions, and eliminating two clicks is worth billions.
It’s possible to employ seasoned experts to model shopper experience for priority categories (Amazon vendor mangers are typically experienced category managers), but it’s not possible to have an expert for every search term. In order to be The Everything Store, Amazon has to automate shopper insights by algorithmizing the customer experience.
Algorithmizing the customer experience
From our experiments at Profitero, we know that Amazon models customer behavior uniquely for each platform and category. It’s valuable to Amazon to understand which digital metrics best measure customer behaviors. This is why Amazon probably uses some form of decision trees to determine which KPIs matter the most for driving conversion.
Decision trees are a family of machine learning algorithms that leverage computational power to evaluate permutations of parameter-weights and choose the best-performing combination. Despite Amazon’s celestial processing force, these models favor simplicity as much as they favor complexity. There is a trade-off: if the model is too simple, it won’t be accurate; if it’s too complex, it won’t be scalable. Either way, too simple or too complex = not useful.
So, when Amazon evaluates which products are winners for a given search query or category module, the decisions are made from a rightfully limited set of metrics. This means that many products can have the same score as far as the algorithm is concerned.
Here’s an example. Let’s say there is a formula (shown below) for measuring a product’s quality score. The product with the highest score wins the coveted Amazon’s Choice badge.
Sample formula: Quality Score = orders in last 90 days + count of reviews above 3 stars – (return rate x 100) – (out-of-stock rate x 100)
But see how these two products end up having the same score.
|
KPI |
Product A |
Product B |
|
Orders for last 90 days |
10,000 |
9,000 |
|
Count of reviews above 3 stars |
1,000 |
2,000 |
|
Return rate |
5% |
10% |
|
Out-of-stock rate |
5% |
0% |
|
Quality Score |
10,000 |
9,000 |
So, how does Amazon choose a winner from a basket of virtually indistinguishable products?
- Option A: Amazon can expand the formula to include additional KPIs, such as delivery time or the count of lifestyle images below the fold. But as previously mentioned, having too many explanatory variables reduces the accuracy of the model by making every variable appear statistically significant. The model would become useless for measuring what it’s intended to measure.
- Option B: Amazon can manually perform shopper surveys, outsource product scoring to expert reviewers, and/or build in other qualitative metrics. Amazon actually started experimenting with some of these ideas in early 2018. But sourcing reliably unbiased reviews for every product is not feasible, so this option seems best reserved for select categories.
- Option C: Amazon can add just one extra variable in the scoring formula: profitability. Amazon already tracks Net Pure Product Margin (which includes contra-COGS/co-op spend) for its vendors and the algorithm includes this metric in its optimizations of assortment and availability. We see evidence of this when products are delisted (CRaP) or moved to a more profitable program (Add-on/Exclusively for Prime) when below a certain profitability threshold (often 30%).
The best option happens to be a combination of all the above. (A) We can assume Amazon’s formula includes more than four variables, but probably not a lot more. (B) We’ve seen Amazon deploy qualitative metrics to convert more shoppers. (C) And now it appears that Amazon does include profitability when awarding its Choice badge.
Brand investment, not brand bribery
Keep in mind that brand spend is not just a profitability play. Brands that invest in their Amazon relationship also invest in rich product content, run Vine campaigns to generate quality reviews, and engage in agile product development to meet customer needs. In many ways, brand activity improves the online customer experience.
Don’t despair if you are the Amazon Account Manager at a brand that was excluded from bidding (or got outbid) for Amazon’s Choice. Like best-seller ranks and Amazon Advertising bids, the badge is determined in near real-time. You don’t need to wait for Jeff Bezos to reveal the secret parameters because you can act today by optimizing for customer satisfaction.
- Use a tool like Profitero’s AMZ Maximizer™ to identify opportunity areas in availability, traffic, and conversion
- Campaign for additional funding to invest in your brand presence on Amazon
- Research and solve consumer needs by launching new products
Whatever you do — make sure it elevates the shopper experience. The path forward is not to reverse-engineer, but to innovate.