Product Validation Biases And How To Avoid Them

There’s a fantastic presentation right now on Mind The Product with Jerome Ribot talking about Product Psychology and Design.

One brilliant idea is the ‘Pyramid of Learning’, where you present information to your target audience as “wide and easily accessible” to start with and then make this information more nuanced as time goes by.

Psychology doesn’t only apply to product design, it also applies to product validation. If your product team doesn’t have the budget for outside help, a Product Manager can often be left doing the validation themselves. Sometimes bias creeps in.

Here are 3 common product validation biases and practical tips to avoid them:

1. Conformity bias (users agreeing with the majority)

When you get a bunch of people together in a group setting, the natural tendency is conformity of opinion. Any first year psychology undergrad will tell you about Asch and his conformity experiments, along with the scary implications.

Conformity bias is very much at play within user research too. If you’re running group sessions, it’s important to have a moderator that understands how group dynamics work.

The quality of the group’s answers depends on your skill as a moderator to build a positive, creative atmosphere. If you have limited experience moderating, there are two things to bear in mind:

1. Forming, Storming, Norming and Performing is a simple, practical primer on how groups and teams form
2. Knowing when to influence the group if things are going off topic. Easy wins include:

• Actively encouraging quieter users to venture an opinion
• Breaking eye contact when you want more vocal users to stop talking
• Using humour to keep the energy levels high

2. Confirmation bias (finding evidence to support what you already believe)

his one is a doozy because we’ve all fallen victim to it at some point. The danger when you validate your own product is that you can unknowingly look for things that support your pre-existing beliefs and ideas.

When you have data from users, the temptation is to prove yourself right and that means you can miss what’s really going on.

If you’re looking at quantitative data, there are best practice things you can do like having a robust amount of users and opting for significance testing.

But for workshops and sprint sessions it’s a lot more difficult to be objective. So how do you do it?

The guys at Google Ventures have an excellent method of ‘Bringing the Troublemaker’ to any sprint:

‘Bringing the Troublemaker’ is a great tool for internal meetings and brainstorms too. Just make sure the troublemaker can be reined in and think twice before inviting them down the pub.

3. Interviewer bias (making the user think the same way you do)

When you’re explaining your product to users aim to be a scientist, not a salesperson.

You’ve slaved away lovingly on your product, so it feels natural to want to tell users how brilliant it is. But keep in mind the language you use to describe your product will influence how users evaluate it.

They can end up regurgitating your marketing spiel back to you.

Always aim for a spontaneous reaction from users before you tell them how your product is going to change their lives. You’ll learn a lot more.

Beware of biases

Biases are always going to play a role in product validation because it’s a people-driven process.

Keeping these biases top of mind when you’re engaging with users goes a long way in limiting them. Always remember to beat:

• Conformity bias with proactive moderation
• Confirmation bias with a designated troublemaker
• Interviewer bias with factual, concise language

There you have it, 3 simple and effective ways to validate with confidence.

But I’m biased, so I guess I would say that.