Designing voice first, without the answers

I usually write about things after I’ve figured them out. There’s a problem, some wrong turns, and eventually a lesson that holds up well enough to share. This post is different. I’m in the middle of designing a voice-first product right now, and I have more questions than answers. So instead of waiting until it’s all resolved, I want to write down the questions while they’re still open.

I can’t say much about the product itself yet. What I can say is that voice is the primary way you interact with it, and that this has turned out to be a much deeper design problem than I expected.

The parity problem

Here’s the core question I haven’t solved. The product is voice first, but most of its functionality also needs to be reachable through touch. Sometimes you’re in a meeting. Sometimes you’re on a train. Sometimes you’d rather tap than talk, and the product should respect that.

The moment you commit to that, though, you’ve created a trap. Because once every feature exists as a touch interaction, voice starts to feel like an alternative way of doing things rather than the way. And that’s exactly what you see in almost every app that adds voice. There’s a microphone icon somewhere, usually in a corner, and using it feels like a shortcut for people who don’t want to type. Voice becomes an add-on, no matter what the pitch deck says.

The best voice-first product I’ve used is Sesame. Talking to it feels genuinely natural, and it never makes you feel like you should be tapping instead. But Sesame gets to cheat a little. The functionality is the conversation. There’s not much else to mirror through touch, so the parity problem never really comes up. The interesting version of the problem starts when the product actually does things, and those things need to work both ways. I don’t have a good answer for how you design that hierarchy yet. I’m not sure anyone does.

What do you show while someone is talking?

The second question is one I’m further along on, though not done with. When someone speaks to the product, how much of that do you show back to them?

The default answer in most apps is a live transcription. Words appear on screen as you say them. It feels like feedback, and feedback is usually good design. But I’ve come to believe it’s the wrong call here. When people watch their own words appear in real time, they start editing themselves. They pause, rephrase, go back. The transcript turns speaking into a form of typing, and you lose the thing that made voice worth building for in the first place.

It also misunderstands where the intelligence sits. What AI models are genuinely good at is taking messy, half-formed speech, with all its false starts and “wait, what I mean is”, and finding the intended meaning. The raw transcript is an intermediate format. Showing it to the user is like a restaurant serving you the mise en place.

What you want is an experience where talking feels as natural as talking. Which probably means showing less, not more, while someone speaks. How much less is the part I’m still working out.

Transcripts as a confession

The third question grew out of the second. I’ve started to think a live transcript isn’t only a UX problem. It’s a signal. Showing users every word as it’s captured says, in effect, we’re not sure we heard you right, so please check our work.

That might have been the honest position five years ago. It isn’t anymore. Transcription is good now, and interpretation on top of it is better. If you’re building voice first, I think you have to design from the assumption that the model will get it right the large majority of the time, and handle the rest through graceful correction rather than constant verification. A product that asks you to proofread everything you say doesn’t trust itself, and users can feel that. Trust in the model isn’t a technical detail. It’s a design stance, and it changes what you put on the screen.

I don’t have a neat way to close this, which is fitting for a post about open questions. If you’ve worked on voice interfaces and wrestled with any of this, especially the parity problem, I’d genuinely like to hear how you approached it. Hit reply or send me an email.