It’s not surprising that companies engaged in new product development feel the need to test the likely success of an idea before fully investing in its development and rollout. Hence the ongoing popularity of concept test and purchase intent surveys. But as any victim of a telephone or online survey can testify, answering numerous questions about your likelihood to do or buy something is not only tedious, but questionably accurate, even when responding in all good faith.
That’s why at Communispace, we have been actively exploring ways to incorporate some principles of “gamification” into the concept development, refinement, and testing that’s done in our private, online communities. We know from vast experience that when people are motivated to invest more in answering questions – when their hearts as well as minds are engaged in a task – the quality of their responses is likely to be better. And we hypothesized that a Prediction Market, in which participants invest points or play money in their predictions about which concepts are most likely to succeed in the market, would do just that. To put our theory to the test, we partnered with Consensus Point, makers of a prediction market platform called HuuNu, to evaluate the likely success of several new product ideas that had been co-created with consumers in one of our private online communities.
Member feedback confirmed that most participants found the market format to be highly engaging. As compelling, 287 participants in the prediction market – people who had not been screened to be representative of the target market in any way – reached almost the identical conclusions as 3,776 survey respondents who had. This suggests three somewhat provocative implications:
- Having a small sample may not matter with the right method.
- Having the “right” sample may not matter with the right method.
- Don’t ask people to offer judgments on concepts about which they do not have strong convictions
Are you wondering what I mean by each of the above? Well, you’ll have to read the case study to find out. Please do, then let me know what you think. Where are the flaws in our approach? And what follow-up questions are you left with?