Dancing with Uncertainty
AI can generate an infinite number of product ideas, lines of code, and experiments in mere minutes. This means execution is cheaper and faster than ever.
But when we focus too much on speed, we can easily forget one of the fundamental questions of product: Should this even be built in the first place?
Surbhi Marwah was one of the keynote speakers who—along with Thomas Brouwer and Pippa Topp—reflected on what is staying the same in product, despite the explosion of AI tools and capabilities.
Sharing a real-life case study from her time at Zalando, Surbhi explored where AI excels and where human skills remain critical. Watch Surbhi’s keynote below or read on for the highlights!
While earlier sessions focused on the excitement and potential of AI, Surbhi began by saying that her talk might contradict some of what we’d heard earlier in the day.
And with that warning out of the way, Surbhi called out an important distinction: “The fundamental question in product was never, ‘Can we build it?’ It was always, ‘Should we build it?’”
Yes, AI is accelerating everything. Execution is becoming cheaper and faster. But Surbhi estimates that these changes will only account for about 10 to 20% of product development.
The question Surbhi invited us to reflect on was: How do you decide what to build? And to illustrate one example of how she’s faced this question herself, Surbhi shared a case study from her time at e-commerce company Zalando.
Unraveling a tangled process at Zalando
Surbhi prefaced her story by saying that it might look like a data problem, but it was actually a discovery and leadership problem.
As an e-commerce platform, Zalando aims to help customers discover different brands, figure out if something appeals to them, and decide whether or not to purchase it.
And the first steps in that journey—discovery and consideration—fell under Surbhi’s team’s remit.
The fundamental question Surbhi’s team was trying to answer was, “How do you move a customer from a deeply held preference to trying something they never asked for? How do you inspire them?” If someone only wants to wear black, how do you get them to consider a green or pink item, for example?
Digging into the data
Initially, Surbhi said, this seemed like a data problem. If they measured engagement and retention, they could then optimize a matrix.
But it wasn’t that simple.
When they measured weekly active users/monthly active users, the numbers looked great. People were returning to the platform, but the matrix did not tell them what was bringing them back. And as they tried to substantiate it with the “time spent” KPI, they realized that some customers had high time spent because they were engaging with new brands or style categories while others had high time spent because they couldn’t find what they were looking for and were frustrated.
One big lesson from this exercise: The same metrics of weekly active users and time spent could reflect different realities.
To get a better idea of what was happening, they built the engagement value system. They gave a point value to every action that the customer would take in the experience so they could trace a customer from impression to flow to purchase, saves, etc. and show correlation to the final revenue figure.
And then they had to make a decision—what should they actually optimize for?
Bringing in different perspectives
Again, the question of what to optimize for might seem to be simple on the surface, but the reality was much more complicated. There were four teams—Global Marketing, Brands and Partners, Content, and Local Markets—who were involved in this decision, and they each had their own agenda and perspective.
For example, Global Marketing looked at the data and said, “Let’s drive reach. More eyeballs,” while Content saw the data and wanted to focus on storytelling, inspiring the customer, and getting the best influencers.
Each team was pulling in different directions of the customer experience.
In a situation like this, Surbhi explained, “You can’t just delegate this analysis—the data doesn’t just make the decision for you in this case. The data creates options. It makes the uncertainty bigger; not smaller.”
This was one of Surbhi’s key takeaways: Data gives you options, but you still have to pick the option that works best for you, your company, and your customers.
Finding the balance between AI capabilities and areas where humans excel
At this point, Surbhi began to reframe the way she was looking at the problem. Instead of asking, “What does the data say?” she started asking “What are these people actually worried about?”
This led Surbhi to an important realization: Data was never the problem. It was a lack of shared vocabulary.
Here’s how Surbhi approached this challenge:
She used Claude Code to plug into every team’s repository. This exercise allowed her to identify use cases, dependencies, and which features were used most often.
Next, she sat down with stakeholders to ask them to define what success looked like to them.
Surbhi explained why this approach was so powerful: “The machine showed us the fragmentation but the interviews told us what mattered to different people and why these siloes had started to emerge.”
A few key learnings and takeaways
Going through this exercise taught Surbhi some critical lessons: AI is extraordinary at reading complex systems and pattern recognition at scale. It only took a week to create a map of multiple tools and figure out where the dependencies were and which features were being used. AI translated the architecture, which was something that they wouldn’t have been able to do earlier.
But the humans translated the intentions of why it was the way it was. They helped make sense of the chaos, which gave Surbhi the information she needed to make a decision.
And now Surbhi took what she’d discovered back to leadership. “I didn’t lead with the problem. I led with what it meant: If we fix this, here’s what becomes possible.”
Through this process, Surbhi had learned that they were hemorrhaging millions of euros every year, not to competition, but to themselves—there was friction in the form of tools that didn’t talk to each other, bugs that were breaking the content journey for customers, and campaigns that kept running after they should have ended.
So instead of building new features or AI recommendations, Surbhi chose to fix the foundation. “In the age of AI, if you don’t fix the foundation, you’re just building more garbage on top of the garbage you already have,” she explained.
To close out, Surbhi reminded us of a few of her key lessons from this experience:
Just because AI makes it possible to code, that doesn’t mean you should do your engineer’s job. Your job is to bring judgment and intention to what gets built.
Don’t forget that you sit in a unique position that straddles different teams and disciplines. One of your other key roles is to ensure there’s a shared language between these different groups.
Remember that sometimes exercising your judgment means not deciding to build anything new. Sometimes the best choice is simply to fix your foundation.