Doing More Customer Interviews Won't Make You Better At Them

Founders and product managers fail at customer development, even when they're consistently talking to users. But this is less from conducting too few interviews and more from a missing feedback loop. The art of posing the right question is counterintuitive and cognitively taxing, so founders can’t analyze their own interviewing performance in real time. A transcript-based AI coach that flags hard-fail interviewing errors restores that loop—raising ground truth per hour faster and cheaper than brute-forcing more interviews.

Customer development is commonly phrased as advice to simply "talk to users," but that undersells the difficulty of doing it right. It's actually the ethnography of demand, with many unintuitive pitfalls, and it requires deliberate practice to do well. Avoiding these pitfalls is difficult to do in the heat of an interview.

But AI coaches can bridge that gap between theory and practice by providing specific actionable feedback to your specific customer interviews. This feedback is the critically missing piece to improve the skill of inferring customer demand by improving the number of ground truths per hour.

What makes customer development so hard? It's hard in two ways: 1) People usually can't provide a clear articulation of their problem or a feasible solution. 2) The questions you ask can unintentionally incentivize people to lie to you.

This is unintuitive. Founders think customers would be properly incentivized to be truthful about their problems and their solutions. Why would you lie about your needs? Reality is, most people are too busy trying to achieve some other goal to take a step back to introspect and articulate the problem. Hence, they may not identify the core problem, nor the most important problem. They're also not aware of all the technological solutions they can bring to bear on a problem. But that doesn't stop them from articulating their problems as solutions, which can often be misleading.

First-time founders can be naive to the Heisenberg-like effects of asking for customer opinions like, "would you use this?" and "what do you think of my idea?" Social norms of politeness dictate you don't trash something the other person is clearly emotionally invested in. So unless a founder's asking for money, it's often easier to say "Sure, I'd use it!", "I think lots of people would find it useful," and "It sounds like a great idea!" than to deal with the social awkwardness of telling someone's idea doesn't resonate with you. It's akin to telling someone their baby is ugly.

As a result, you cannot just build the solutions people ask for. People are experts in their problems, not their solutions. You cannot ask people about their problems directly. They can misjudge the value of a problem or how pressing a problem is. And you definitely cannot tell them your idea and ask them what they think about it and whether they'd buy. They'd sooner lie to you than to tell you your idea is bad or that they don't need your product.

With those constraints, what can you even ask a potential customer in an interview? The only reliable topics you can ask potential customers are recent recollections of their concrete actions when they came across a problem and the context in which they found themselves when the problem arose. Everything else from why their need exists to the product solution must be inferred.

Hence, customer development is unintuitive and requires deliberate practice to overcome these common pitfalls when interviewing potential customers. It requires just a bit of post-interview introspection and analysis on what went right, and what could be improved, in the same way professional basketball players review game footage.

Unlike professional sports, the customer development feedback loop is missing. But why do we need this feedback loop in the first place?

You can't run meta-analysis while you're interviewing. When you're trying to infer demand, your working memory is already overloaded. The right questions are counterintuitive, so you're juggling two hard tasks at once: finding the next question that gets to ground truth while listening closely enough to follow the thread of what's being said. It's predictable that you'd miss the meta-level entirely, because all your attention is consumed by the conversation itself.

By the time the interview ends, the details have already faded. You’re left with a few half-formed notes and a hazy memory, hardly enough for real analysis. Without an external coach to close that loop, you have no reliable way to see what you did well or poorly. And without that feedback, improvement stalls — founders and product managers keep repeating the same mistakes, unaware until it's too late and they've built the wrong thing. It’s easy to read what good interviews should look like; it’s much harder to recognize those principles in the middle of a conversation, when you’re juggling multiple cues and counterintuitive moves at once.

Why not just do more interviews? Wouldn't brute-forcing the number of interviews help founders and product managers get better without needing the feedback mechanism?

You might be reminded of the parable of the pottery class graded on the quantity of pottery they made improved faster than the other class graded on the quality of the pottery they made. Yet, the parable doesn't apply.

Clay gives perfect feedback; conversations don't. It's clear when clay doesn't hold a shape. It's not clear when words reflect ground truth. Finished pottery exists afterward; conversations are fleeting and ephemeral. Concrete evidence leads to higher fidelity feedback and analysis.

Hence, grinding on quantity only works when the medium gives perfect feedback. Customer interviews give noisy feedback: words are biased, memories are selective, and without transcripts at the very least, you're learning from low-fidelity notes. If your interviewing error rate stays high with leading questions, future hypotheticals, or failure to quantify budget or urgency, then doing more interviews mostly scales the noise. When you tamp down the error rate, the speed to ground truth increases, and that compounding effect takes over. The founder who learns to extract ground truth in a handful of interviews will outpace one who needs dozens just to see the same signal.

How do you leverage an AI coach in a customer development workflow? It's a straightforward two-step process. First, you convert the audio conversation of the interview into a transcript. Then, you use a prompt where the AI plays the role of a customer development coach to analyze the transcript and write an analysis of your performance.

Now that audio-to-text transcription works really well with the current crop of multi-modal LLMs, it's possible to get accurate transcripts of a conversation for cheap. The transcription can happen in real-time, on your own machine, so you don't even need to record the raw audio, or send it to remote servers. When I hop on a call for a customer interview, I'll ask the interviewee for permission to record the audio portion of the call to get a transcript. Nine out of ten people will agree to it. I think they appreciate being asked. I currently use Granola.ai, but MacWhisper or other audio transcript software will do. After the interview, I'll copy and paste the transcript into a markdown file named interview_transcript.md

Instead of taking notes that I'll never read again when reading a book on customer development, such as Jobs-to-be-done, the Mom Test, Sales Safari, or any number of resources written on customer development, I'll distill their principles into a ~2200 word prompt instead. The prompt has sections on the agent's objective, instructions, context and principles, and examples to help guide and guardrail the AI agent into doing the analysis correctly. This gets written into a file named cust_dev_coach.md

With this prompt in hand, I'll tell Claude Code (or OpenAI Codex):

This is just as easily done with ChatGPT. You can paste the agent prompt, follow it with the transcript, and then ask it to play the role of the prompt and analyze the transcript. The analysis is a high-fidelity, transcript-based introspection that you wouldn't have been able to do yourself in real-time, nor afterwards without great effort.

Is most of the benefit just having the transcript, and not the AI coach? It's true that a transcript makes an ephemeral conversation tangible. However, it takes a lot of effort to review a transcript post-interview. You'd have to relive the same conversation and manually map principle to practice. Most people don't do this because it's both taxing and time consuming. AI coaches lower this cost to almost nothing, while keeping the analysis specific and actionable.

Why not use a human coach? Customer development coaching doesn't seem to be a thing in Silicon Valley. Having lurked on Hacker News, Indie Hackers, and Bookface (YC's internal forum) for years, I've heard founders ask for recommendations for executive coaches, health and fitness coaches, and even couples therapy coaches, but never customer development coaches. Given customer development is often framed as just "talking to users," perhaps it's not a thing for the same reason ski instructors are more numerous than running instructors: running seems intuitive in a way that skiing doesn't.

But even if human coaching was available, it would be much more expensive. A human coach could cost on the order of thousands, whereas running a prompt would cost anywhere from under a dollar to $3. While human coaches could provide more value beyond pointing out mistakes, just avoiding mistakes will get you most of the benefit. As we've established earlier, a lowered error rate compounds.

There are additional benefits to using this workflow. By transcribing the audio into a transcript, you'll always have the raw information to perform additional analysis in the future, when your AI coach prompt improves or when there's another method of inferring customer demand that works better. And if all else fails, you can read the first-hand source material for insights you might have missed the first time.

So how do we know the AI coach is giving us good advice? Aren't LLMs prone to hallucination? The solution is to limit the AI coach to raising flags that are only hard-fail observable items. This includes:

• Leading/loaded question detection
• Future/hypothetical framing (“Would you…”, “Would you pay…”)
• Failure to ask budget/time/cost
• Failure to ask for the last concrete event (“Tell me about the last time…”)
• Interruption rate > X / min

It won't be giving judgements on "insightfulness", and it wouldn't be giving product advice.

Inexperienced LLM users expect AI to give exceptional performance with very little context. Like any human, the more relevant context you give them, the better they will do. Hence, detailed context and exposition on "what good looks like" in the prompt as gleaned from the source material on customer development would go a long way into guiding the LLM to flag these hard-fail items.

And if you do need to chase those edge cases where the AI agent fails, you can employ system evals. But that's rarely needed, because most of the mistakes founders make are common mistakes covered by the source material on customer development, rather than in rare edge cases.

Here are some examples of analysis from my own AI coach from multiple interviews I conducted. These were things that I missed while I was in the flow of the conversation. My brain never thought to pick up on these as problems, or I was too busy thinking of the next question to flag these as being problematic.

❌ Missing the Elephant in the Room

Throughout the conversation, you never asked the critical business question: "How much time and money does SOC2 compliance actually cost you per year?"

I spent all this time building automation, but I have no idea if this is a $1,000 problem or a $50,000 problem.

• ❌ Major omission. Without understanding the economic impact, you can't assess if this is a real business opportunity or just an engineering hobby project. This is the elephant question that could tank the business case.

How to improve: "Before we dive into the technical solution, help me understand the business impact. How much time do you spend on SOC2 prep? What would happen if you didn't get certified?"

❌ MISSED OPPORTUNITY: Failed to Probe Pain Points

You: "And then so because the two of you, you know, like, the other you know what you're doing. You know what you're not doing..."

Analysis: You completely glossed over their statement that "Linear was just too hard" and they abandoned structured project management. This is a massive signal about friction in their workflow that you didn't explore.

What you should have asked:

• "What made Linear too hard specifically?"
• "What did you try to do with it that didn't work?"
• "How do you handle knowing what the other person is working on now?"
• "What problems, if any, does this informal approach create?"

❌ Accepting Surface-Level Answers

Quote from transcript:

Them: "I don't feel like I have any problems."
You: "Okay. And then so have you showed this workflow to your other coworkers..."

They're thinking: That was easy - they moved on without digging deeper.

Consequences: You missed a golden opportunity to understand the real friction points. "No problems" is almost never true - you should have probed deeper into specific frustrations or limitations.

Better approach: "When you say no problems - walk me through the last time you had to create a complex PRD. What was the most time-consuming part?"

If every analysis was just a list of things I did wrong, it'd feel pretty bad and I might not want to read it. I asked the AI coach to also point out the good things I was doing in the interview, so I knew to keep on doing it. This shit sandwich makes it both fun and motivational to read the analysis after an interview. The anticipation of reading the post-mortem makes me want to talk to more users. Here's a single example of positive feedback.

✅ Great Questions That Revealed Insights

You: "what what has it cost you? Like, did it cost you, like, hours, or is it like, I can't do my work or I, like, can't collaborate with people or because of that? Packaging thing you're talking about? Or, like, what's what's the cost there?" ^block_1

Them: "I wouldn't say it's like like, of of all of problems in my career that have, like, like, a big kinda cost, like, I wouldn't say that that has been one of them."

Wait, so this isn't actually a big problem for me. I'm being asked to quantify something that feels minor compared to my real issues.

• ✅ Excellent follow-up. This question distinguished between "I-will-pay-to-solve-that" problems and "that's-kind-of-annoying-but-I-can-deal-with-it" problems. His response immediately deflated the notebook packaging issue as a major pain point, which is exactly the kind of reality check you need in customer development.

Despite knowing better in the abstract from reading about customer development repeatedly, in practice some of my mistakes are basic, classic mistakes. Without an AI coach, I wouldn't have noticed or remembered them. With this new awareness, I'm not making the same mistakes I was making before, and I'm much more confident of the ground truth information I'm gleaning from the interviews.

There are many aspects of picking the right startup idea that can lead a founder or product manager astray, but getting bad validation of an idea from customer development doesn't have to be one of them now. Keep practicing interviewing, but it can now be deliberate, with feedback, due to the help from AI. Scaling up interviewing without closing the loop on improvement is wasted time. Try running a two-week test: conduct five interviews, and run a hard-fail rubric. If your loaded questions and hypothetical probing don't drop by more than 50%, then you can try going back to brute-forcing interviews. Otherwise, keep it. Ground truth per hour compounds faster than "talking to more users."