Can ChatGPT replace customer interviews?

Can Chat-GPT replace customer interviews?

Heresy! That’s what I used to think, until I realized one thing…

(And yes, I do share my prompts.)

People will do anything to avoid talking to customers. It’s a tale as old as time, and it ends in tears. Hence, I was doubtful at first.

But then my friend, Mo Syed, pointed out something remarkable: ChatGPT’s training set isn’t facts so much as our cultural consensus. Therefore, if the information you want from a customer interview is part of our culture, a few carefully worded prompts could get you pretty far.

The Experiment

I tested Mo’s approach by staging a Jobs to be Done interview using ChatGPT for a service I know nothing about – Lindy Hop dance lessons. I’ll link to my full prompt list below. Here’s the process I followed:

1. Discover possible outcomes: I started by asking what outcomes people hope to achieve from these dance lessonsand got a plausible list.

2. Choose one outcome and explore that customer journey. I asked about social and emotional aspects and situational triggers that might spur someone to start pursuing this outcome.

3. Find traffic sources: I asked where somebody might look for ways to achieve this goal and got ideas for where to promote my dance school.

4. Uncover barriers to purchase: I explored customer anxieties and got advice on how to ease prospects’ fears, to help me improve the “product onboarding” for my dance school.

My surprising results

In our Coaching Program, I’ve seen hundreds of customer interviews, so my expectations were low. But the chatbot caught me off-guard. Like most things ChatGPT, it got 70% of the way there.

I checked with Tom Kerwin, an actual Lindy Hop instructor (and UX guru), and he said the actual reasons people try Lindy Hop are a subset of the ones ChatGPT suggested, and the actual reasons run deeper emotionally.

Next, I back tested Mo’s approach with some products where my team had completed good interviews and experimentally validated the customer outcome. My finding was the same as Tom’s – the actual reasons are a subset of the ones from ChatGPT, but the AI misses the emotional depth and nuance, and some important details.

My verdict:

1. It’s a great training tool that helps with the difficult mental shift from thinking about our product to customer outcomes. (That alone is incredibly valuable; I now use it as a teaching tool in our program).

2. It’s less effective with niche B2B, as less information is publicly available, especially on the social and emotional outcomes.

3. It gives a broad starter list. But the actual “winning” outcome is often missing or buried deep in the AI list, which often omits subtle but powerful emotional and social outcomes.

Bottom Line: It’s no substitute for listening to customers, but it’s a major upgrade over working from untested assumptions, which is what most people do. 😕

The Prompts

And here are the prompts I use, plus the Lindy Hop interview transcript, as a worked example.

If you find ways to improve this, let me know? If I share them in a future email, I’ll credit you, unless you'd prefer to languish in obscurity, like Mo.

I hope this helps!