01The quick read
A faster way to answer the question every agency hears: “Which one should we use?”
Same-day direction
A conversation at 10. A test at 11. A preferred image by noon.
Outside the bubble
Feedback from targeted shoppers instead of another internal company poll.
A reason behind the result
Verbatim responses help the agency and client understand the choice.
Cartograph manages the moving parts that make Amazon work for its clients: advertising, photography, inventory, content, and strategy. That means its team often sits between two valid points of view. The brand knows its identity. Cartograph knows the Amazon shelf. Both have opinions about what shoppers will respond to.
ProductPinion brings the shopper into the conversation. The loop Cartograph runs looks like this: a creative question, shopper validation, evidence, then a recommendation the agency can stand behind in the client conversation.
02A real result
This is what arrives before the recommendation gets made.
In one test for a beverage client, Cartograph put four hero-image directions in front of 87 targeted shoppers. One direction led with 45 of 87 votes—51.72%, against 36.78% for the nearest alternative—and ProductPinion reported 82.89% certainty in the result.
Which direction won
One hero-image direction took 45 of 87 votes.
How strong the signal is
82.89% certainty reported behind the split.
Why shoppers chose it
Verbatim responses explain the preference in shoppers' own words.
What it informs
Evidence the agency brings into the client conversation.
03Inside the agency
The people making the call do not all enter the room with the same context.
Justus Mai is a Brand Manager at Cartograph. He runs Amazon end to end for several brands, meets with clients weekly, and helps decide the strategy that will show up on the search results page.
The work comes with a built-in tension. A client may favor the look that fits its direct-to-consumer world. Cartograph may favor what experience says will work on Amazon. Both opinions come from real experience; they just come from different sides of the table.
“The insights that we get from ProductPinion help us check assumptions and get on the same page.”
Justus was explicit that the test is not about proving the agency right; sometimes the client's idea wins. The point is landing on the strongest solution, not winning the argument.
04The decision problem
The old options were slow, biased, or missing the most useful part.
Amazon’s own A/B testing gives Cartograph real marketplace behavior. It also requires a live listing with sales, can take roughly a month to a month and a half, and does not explain what a shopper saw in the image.
Internal poll
Fast, but familiar
Ask 10 or 15 coworkers which image they like. The answer arrives quickly, along with the same shared context and bias.
Amazon A/B test
Behavior, but later
Wait for a live test to finish. Learn which version converted, without hearing why shoppers made the choice.
“Either gut, or use slow tools.”
While a test runs, the agency is still on the clock. Clients want updates. Launch dates keep moving closer. Production work can sit behind a decision that has not arrived yet.
05The same-day workflow
A content question at 10 a.m. can become a tested direction by noon.
Justus laid it out plainly: discuss the content at 10, set up the test at 11, and know which image the team wants to use by noon. That result can guide more than one SKU.
- 10:00
Frame the decision
The agency and brand identify the competing creative directions and what they need to learn.
- 11:00
Put it in front of shoppers
Cartograph targets a relevant audience and launches the comparison.
- 12:00
Choose with context
The team sees the preference and reads the reasons behind it before production expands.
The speed matters because of how agency work compounds. When you're making creative decisions across multiple Amazon brands, waiting weeks for directional feedback doesn't scale—and it pushes testing too late in the process to change anything.
This also creates a path to testing before launch—when Amazon's native experiment cannot run because there are no sales yet.
06The catalog-defining test
Four hero-image directions. One clear lead. A style decision that could travel across the catalog.
The beverage test shown earlier did more than settle a single image. Because the preference was measured before production scaled, Cartograph could use what it learned to guide the hero-image style across the client's catalog instead of producing every asset first and discovering the preference later.
That is the order of operations change: the style decision gets evidence while it is still a direction on screen, at the point where changing course is cheap.
07The client conversation
Better evidence for the client conversation.
“You need the why. It’s easy to dismiss just who won.”
When the team reads the responses, the debate usually ends. A recurring mention can surface a detail the creative team had stopped noticing: an organic claim, the wrapper itself, the way a package looks, or a benefit shoppers expected to see.
Those reasons can move into bullet points, supporting images, packaging, keyword strategy, and the next round of tests. In some client work, Cartograph has used this kind of feedback to influence brand colors and visual guidelines beyond Amazon, including websites and email.
The deeper agency benefit
The client can inspect the evidence, too.
Cartograph can share an external test link so the client reads what shoppers said directly. The recommendation changes shape in that conversation: from “we think this direction is stronger” to “here’s the direction shoppers preferred—and what they told us about why.”
ProductPinion strengthens the judgment an experienced Amazon team already brings. It adds an evidence layer under the recommendation, which is what makes the recommendation easier to defend.
08The agency playbook
Test the direction before multiplying the production work.
Cartograph does not wait until a catalog is finished to ask whether the direction works. Testing is built into the order of operations.
- 1
Find the consequential choice
Start with the hero image or creative direction that will shape clicks, the listing, or a larger asset set.
- 2
Test before the catalog is built
Compare a focused set of directions before spending weeks producing the same style across many SKUs.
- 3
Target people who could buy
Narrow the audience with relevant shopping habits and category characteristics.
- 4
Read the reasons, not only the rank
Look for repeated language that explains what shoppers valued, missed, or misunderstood.
- 5
Carry the learning forward
Use the result in the listing, then apply useful themes to the catalog, packaging, website, email, or a follow-up test.
“If we’re going to produce a whole bunch of stuff in the same style, let’s test it before we spend weeks making all of these images.”
09The takeaway
A good test settles more than the image. It settles the discussion.
Justus summed up what changed: confidence, alignment, time, and decisions. ProductPinion helps Cartograph move faster, show clients what “data-driven” looks like in practice, and hear what people see before a creative direction becomes expensive to unwind.
He doesn’t oversell it, though. Amazon’s live A/B test remains valuable because it measures actual shoppers in the marketplace. ProductPinion gives the team a faster learning loop—especially before launch—and the qualitative context Amazon’s result does not provide. In Justus’s own tests, the later Amazon results have aligned with the ProductPinion direction.
Your next creative decision can start now
Poll 20 shoppers free. Get feedback in seconds.
Bring the two images your team keeps debating. Leave with a preference—and the reasons behind it.
Questions agency teams ask
Does this replace Amazon A/B testing?+
No. Cartograph values Amazon’s behavioral data. ProductPinion adds speed, pre-launch testing, targeted feedback, and the reasons behind a choice.
Is the goal to prove the agency right?+
No. Justus emphasized that the client’s direction sometimes tests better. The goal is to help both sides reach the strongest answer.
What should an agency test first?+
Cartograph puts particular weight on hero images because they shape the first click from the search results page.
Where can the learning go after the test?+
Into hero images, listing copy, supporting creative, packaging, brand colors, websites, email, and the next experiment—depending on what shoppers say.