Testing Context-Aware Alexa Experiences with Parth Acharya | Episode #11

Parth Acharya did not begin university knowing that product management was a profession. His interests spanned technology, filmmaking, storytelling, and data science. A mentor introduced him to the discipline in 2013, and he recognized a role that could combine technical depth, storytelling, and customer-focused innovation.

At Amazon, Acharya owned the systems behind core Alexa smart-home interactions, including requests to turn on a light and the ability to group devices by rooms such as a kitchen. That scope offers a useful look at product management for an interface where there is no screen to clarify what the user meant. The experience has to interpret language, apply context, act on the correct devices, and avoid surprising outcomes.

What you'll learn

• How Acharya's mix of technology and storytelling led to product management

• How Alexa features move from guided testing to limited customer rollout

• Why voice teams study many ways of expressing the same intent

• How writing and anticipation support decisions across complex organizations

Test with people before expanding exposure

Acharya describes multiple layers of experimentation. For some use cases, the team brings in internal or external customers and watches them interact with Alexa and its app. The team learns what works, what causes difficulty, and which interface is most meaningful for the task. These sessions offer direct evidence before a broader release.

The team can then run an unsupervised experiment by enabling a new interaction for a portion of customers, such as 10 percent. After several weeks, it measures whether the change introduced new friction and whether Alexa resolved the intended requests correctly. Only after that evidence does the team consider a full rollout and globalization.

This sequence gives PMs a practical model for managing exposure. Start with observation, move to a limited real-world cohort, measure both benefit and regression, and expand only when the evidence supports it. Each stage answers a different question. Guided use reveals confusion, while a partial rollout shows what happens without the team present to interpret the experience for the user.

Design for variations in language and context

A voice request that sounds simple can represent a large product problem. Acharya explains that the team identifies a set of core, or golden, utterances for a use case, perhaps 15 or 100 ways people could ask for the same action. Annotators and machine-learning systems then help determine variations beyond that starting set. The objective is for a request such as turning on a light to work reliably despite differences in phrasing.

Context also determines whether an action is sensible. Alexa should understand which lights belong to the kitchen and should not interpret a broad command in a way that activates every possible device. Acharya notes that many expected outcomes feel like common sense to a person, but the product must make that context operational.

Because consumer products invite ideas from many directions, Acharya recommends listening widely. His organization had a dedicated distribution list where people could propose things Alexa might do. He regularly reviewed those suggestions for ideas relevant to his product area. The principle is to keep discovery open while still testing whether a proposal creates the intended customer outcome.

Write decisions and practice anticipation

A major challenge in Acharya's role is learning Amazon's document-centered culture. Depending on the size of the investment, documents such as PRFAQs or six-page narratives can take from a month to six months. He estimates that roughly 60 percent of his time goes into writing product, roadmap, decision, technical-design, or conflict-resolution documents. Writing is therefore not just reporting. It is a mechanism for clarifying an idea and aligning people around a consequential decision.

For early testing, Acharya advises building a quick version, having team members live with it, inviting people across the organization to try it, and then gathering feedback from a more demographically diverse group. Nearby colleagues are a useful first laboratory, but they are not a substitute for broader representation.

He also reframed a tendency to think extensively before implementation as a capacity for anticipation. When several projects are moving at once, he tries to predict downstream effects, possible escalations, and experience failures before the team commits significant engineering time. Combined with staged experimentation, anticipation becomes a way to identify what needs evidence rather than a reason to avoid acting.

Listen to the full conversation: Episode #11 audio.

Acharya's approach joins careful writing with graduated testing: make the idea understandable, expose it to increasingly realistic use, watch for friction, and anticipate what could break as the product reaches more people.