In companies' never ending quest to "Build something people want", Paul Graham advises:

Startups are better off talking to users than x, for almost all values of x.

— Paul Graham (@paulg) March 2, 2016

That may sound obvious, even easy. But it's often incredibly difficult to pull off.

I remember doing a series of customer interviews a few years ago. These weren't just casual interviews. We were heavily invested. We paid our customers to be there. Multiple stakeholders carved out an entire week of time listening to user feedback about our product and our brand. The opportunity cost was high.

Our first conversation went like this:

Us: Why did you choose to buy our product?
User: I liked it over your competition.
Us: Great. What was better about us?
User: I don't know. I just liked it.

Ugh. If we have a week of conversations like this, we'll end up with nothing. So what can we do about it?

Where did the term "brainstorming" come from? In 1942, Alex Faickney Osborn, one of the founding executives of the distinguished advertising agency BBDO, began publishing a series of books in which he outlined methods of creativity he and his team used at BBDO. In the book, "How to Think Up", he repurposed the term brainstorming to describe a group effort to come up with creative ideas.

Osborn's "Brainstorming" methodology had 2 main tenants:

1. Come up with as many ideas as possible.
2. Defer judgement.

Osborn argued that being critical of ideas during brainstorming hinders creativity. Since the point of brainstorming is to stimulate more idea generation, criticism was banned in a session.

And since then, brainstorming has become a household name. Who here hasn't been in some kind of group meeting where you've "brainstormed" a solution to something?

But, what most people don't realize about these "laws of brainstorming": Osborn made it all up.

That's a little harsh. I don't think Osborn ever made a claim that the principles behind brainstorming were laws derived from deep, thorough scientific research.

So it's interesting to actually do some deep, thorough, scientific research on "brainstorming".

Was Osborn right about how to get groups of people to be their most creative? That's what Dr. Charlan Nemeth, a professor of Psychology at Berkeley University, has spent a career studying.

In one of her experiments, she gave two different groups direction to come up with ideas to solve a traffic congestion problem. The control group was told: "come up with as many good solutions as you can to the problem." The second group however was additionally given some typical brainstorming instructions: "Come up with many solutions. Freewheeling is welcome; don’t be afraid to say anything that comes to mind."

This second group was then given one of Osborn's famous principles: You should NOT criticize anyone else’s ideas."

As Osborn would have predicted, the "brainstorming" group told to not criticize each other generated more ideas than the control group.

Ah, then, there was a third group in this experiment. The third group was also given additional brainstorming instruction, but this time they were told: most studies suggest that you SHOULD debate and even criticize each other’s ideas.

This third group, the Debate group, did even better than the brainstorming group. It turns out, healthy debate is actually good for getting people to come up with quantitatively more ideas.

But is debate good for just more ideas?

Dr. Nemeth did another study in which she gathered participants and primed them with a color. Her priming slides were blue. Except in one group Dr. Nemeth manipulated how the blue slides were presented and told participants that some people actually thought the slides were green. Then the groups were asked to come up with word associations with the colors "blue" or "green".

The people who were given this minority dissident viewpoint gave demonstrably more creative word associations. Given even a small amount of dissent or friction related to what you believe to already be true makes you significantly more creative.

Going back to those user interviews we were doing, we weren't getting very good feedback. So what our interviewer did was add some dissent.

User: I don't know. I just liked it.
Us: I don't believe it. I don't believe you even needed to even buy us. It sounds like you could have just used your previous solution and saved money and hassle.

My mouth dropped. Our interviewer was now in a debate with our customer. We're now even calling into question why our product even exists. Will this customer turn around and just admit "You know what, you're right. We don't really need your product."

But that's not what happened. Instead our customer became much more articulate about the problems she had with our competition and why she had a preference for us.

When our interviews became less about acquiescing to our customers and more of an interrogation where we debated our customers feedback and ideas, the conversations become incredibly fruitful and enlightening.

I see this same method of debate in play in our customer conversations here at Rockstar. When we get on a phone call, and a potential customer asks for X, Y, and Z. There's a natural inclination to just say yes. Saying yes, means we can get to sign that contract sooner. Checks get cut sooner.

But we don't just say "yes."

Before anything gets signed, we actually participate in some good natured debate with our customers about what they want built. We'll interrogate why X and Y are important. What assumptions have been made? What assumptions haven't been tested yet? Can we do something other than X and Y that are easier, cheaper, and quicker to build before we get to X and Y? Can we skip doing Z altogether?

Asking all these questions results in risk for us. It gives our customers more time to chat with competitors. It elongates the time until checks are cut. And it often results in projects that are smaller and cheaper than a client had originally budgeted for. That's not how agencies typically like to make money :)

But from years of doing this, we know our best work comes from challenging our customers assumptions to better articulate what they need. And the result is better projects that meet and often exceed expectations because they have our customers' most creative thinking behind them.

P.S. If you need any help building software, give us a shout. I’m sure we can help.