Enterprise Communities and Unbiased Metrics | Episode #18 with Jason Yi

Jason Yi did not grow up immersed in technology. His interest emerged in college, where he came to see technology as a practical kind of modern magic: people could create tools that removed problems and made daily life more convenient. Conversations with friends also showed him that building technology involved more than software engineering.

Product management appealed to Jason because it combines logic and creativity. It asks someone to understand a customer's viewpoint, form hypotheses about a problem, and work with specialists to develop and validate a solution. At Microsoft, he has practiced that work in two different settings: the enterprise product Azure Stack HCI and the consumer-focused Edge Shopping experience.

What you'll learn

• Why enterprise PMs can rely on direct relationships with a smaller set of high-value customers

• How consumer teams use experiments and behavioral data when direct access to millions of users is limited

• What Jason learned by adding a feedback capability to Edge Shopping

• Why metric definitions and selection require care to avoid biased interpretation

• How a PM can respect expert opinions without being carried toward every proposed solution

Enterprise depth versus consumer scale

Jason describes enterprise product management as community-oriented. With a smaller number of customers and significant value attached to each relationship, PMs can arrange direct calls, learn how customers use the product, and investigate specific issues. Internal and external events provide another feedback channel. Teams can explain new capabilities, reserve time for questions, and use surveys to collect structured responses.

This work also taught Jason about product evangelism: making a complex enterprise offering easier for customers to understand. The relationship is not only a source of requests. It is a continuing exchange in which the team explains the product, listens to its community, and learns how the offering fits real environments.

Consumer products change the evidence mix. A feature may affect millions of people, making one-to-one access harder and requiring much more behavioral data. On Edge Shopping, whose purpose is to help people shop conveniently and save money, Jason's team uses metrics and experiments to test whether hypotheses improve the experience. Scale provides many observations, but it also increases the responsibility to define what those observations mean.

Building feedback into the product

Edge Shopping initially emphasized releasing an MVP and getting core features into customers' hands. Once the product reached that point, the team needed better evidence about satisfaction and quality. Jason worked on a feedback feature that joined quantitative measures, such as clicks, with comments about specific failures, including coupons that did not work.

Even the feedback mechanism required validation. Jason ran an A/B test to check that adding it did not create performance regressions. This is a useful product lesson: an instrument for learning is still part of the customer experience. It should be measured for both the information it produces and the costs it might introduce.

Jason sees the most satisfying moment as connecting a vision to a solved customer pain point and then confirming the improvement. In an enterprise setting, that confirmation can arrive in a direct conversation. In a consumer setting, it may appear as a measurable increase in positive sentiment or another carefully defined outcome. The channel changes, but the standard remains evidence that a person's experience became better.

Making decisions without distorting the evidence

Data does not remove judgment. Jason says one of the hardest parts of his daily work is ensuring that a team does not select or interpret data in a way that makes a result appear artificially positive or negative. People may disagree about definitions, segments, and priorities, so the team has to examine how a metric is constructed and whether it represents the question being asked.

He applies similar discipline to stakeholder ideas. Early in his PM career, brainstorms with domain experts could pull him in many directions because each specialist made a valid point. He has turned that vulnerability into an ability to understand multiple perspectives. Instead of reacting immediately, he collects his thoughts, includes his own view in the same evaluation, and weighs the trade-offs.

The PM's contribution is not to dismiss expertise or accept every recommendation. It is to synthesize expert knowledge, customer evidence, and product goals into a decision that serves the customer. That requires enough patience to separate the strength of an argument from the authority of the person presenting it.

Listen to the full episode for Jason's comparison of enterprise relationships, consumer experimentation, feedback design, and balanced stakeholder judgment.