Concept testing validates potential solutions, reveals how buyers value specific features, and verifies buyers’ purchase intent and willingness to pay
B-to-b organizations must determine whether potential offerings have the commercial potential to succeed in the market in the long term. This process of determination occurs through concept testing during the design stage of the SiriusDecisions Product Marketing and Management (PMM) Model, in which product managers systematically gather feedback from customers to gauge the commercial viability of product ideas and identify necessary changes. In this issue of SiriusPerspectives, we review the four key steps involved in conducting concept testing interviews.
The product manager should work with a cross-functional team – and potentially with external market research experts – to design the study:
• Agree on study objectives. Common objectives for a concept testing exercise include measuring how well the concept solves the buyer’s and user’s need, uncovering unrecognized concept benefits, identifying the value of key features and improvement areas, determining intent to buy and willingness to pay, and getting feedback on price ranges. Concept testing may reveal how the buyer defines the need and what metrics measure need extent (e.g. person hours lost, overtime hours consumed). This knowledge helps the product manager evaluate concepts and identify areas in need of improvement. These factors may be measured for several target market segments. To more clearly define objectives, frame potential research objectives as questions, such as “How many employees need access to this offering to ensure it delivers value?” “How and where will users access this offering?” This exercise confirms that research must be conducted to answer the questions, and allows the product manager to tailor the type and amount of research needed.
•Identify target candidates. If product managers hypothesize that several market segments will be interested in the offering, include representative buyers or users from each segment in the concept test – including customers, prospects and those who buy from the competition. Portfolio marketing, product management and sales should agree on the target market profile before recruiting study candidates; the product manager should create an interviewee profile (e.g. industry, company size, job role, reporting level, technology in use) to help identify candidates. Candidates may be sourced from the company database or from external sources (e.g. channel partners, industry databases).
The product manager is responsible for coordinating the concept testing and should complete the following activities before engaging with customers:
•Develop an interview guide or a survey. This ensures that questions are consistent and designed to achieve the objectives. Questions asked during a concept validation interview differ from those asked during interviews intended to uncover or validate needs. Include questions with quantitative scales (e.g. “How would you rate the usefulness of this feature on a five-point scale?”), as their responses facilitate analysis. Test the guide with several sales reps and customers before going live, and make sure the completed research will meet the objectives.
•Arrange for design of the offering prototype. Prototypes can range from a PowerPoint slide or one-page summary to a wireframe or a video explanation, and should be developed to fit the research methodology used. For the purposes of concept testing, the prototype should capture the basic offering concept and illustrate how the offering solves the buyer need. If testing the value of specific features, ensure the prototype demonstrates those features.
The product manager should adhere to the following guidelines when conducting concept testing interviews:
•Set ground rules and expectations. Share with the interviewee the objectives of the session, its duration and its confidential nature. A nondisclosure agreement should be signed as appropriate, and the interviewee should verbally confirm acceptance of the agreement.
•Establish a rapport with the interviewee and capture demographic information. Start by learning more about the interviewee by asking about his or her job role and years of experience. If identification of the selling company and the interviewee’s experience with the company are factors in the research, determine the interviewee’s relationship with and perception of the company. Encourage the interviewee to voice all opinions, including negatives ones.
•Introduce the need or business issue. Introduce the need the concept is designed to solve. Ask the interviewee to discuss the importance of addressing the need and the metrics used to size the problem or identify its existence. Use this discussion to validate the importance of meeting the need.
•Explain the concept. Verbally review the concept and explain how it addresses the need. This is not a sales pitch, so assure the interviewee that he or she doesn’t need to react positively. Ask the interviewee to rate how well the concept would address his or her need and if others in the company would benefit. Ask the interviewee to describe how the company currently addresses the need and the expected efficacy of the concept compared with that of the current approach.
•Present the prototype. Once the interviewee has responded to the concept’s rationale and benefits, present a diagram, wireframe or working prototype. Stop at specific points (e.g. function, feature) to ask if the capability would be useful. This portion of the interview offers an opportunity to refine the concept and ensure it aligns to needs. Ask the interviewee to score or prioritize functions and features on the basis of their value. At the end of the visual presentation, ask the interviewee to offer suggestions for improvement or alternative ways to use the offering. Negative responses can be illuminating; probe to determine if the cause of a lackluster or negative reaction is a lack of customer need, poor concept definition or simply the wrong audience.
•Discuss the potential economic benefits of the concept. Ask the interviewee about the concept’s tangible and intangible benefits (e.g. decreased risk, enhanced peace of mind). For product enhancements or add-ons, getting to the financial benefit can take time, so be prepared with some hypotheses to verify financial benefits.
•Ask about willingness to pay. The discussion of economic benefits presents an opportunity to get feedback about price and cost of ownership. One approach is to ask about price ranges. A second method is to use the Van Westendorp Price Sensitivity Meter – a standard approach that allows the interviewee to provide specific feedback on price ranges and identify price points that suggest a specific value for the offering. In addition to demonstrating the offering’s commercial viability, this feedback can help shape pricing strategy. In some cases, asking about willingness to pay may not be relevant – e.g. when interviewing users rather than buyers.
•Capture intent to purchase. The critical question is whether the interviewee would become a buyer. Although much of the feedback may appear positive and supportive, the bottom line is the offering’s commercial merit (or lack thereof). One approach is to ask the interviewee to rate his or her intent to buy on a scale of 1 (“I would never buy the product”) to 5 (“I would most definitely buy the product”). If the rating is 3 or lower, see if lowering the price point has an impact. If it doesn’t, there is likely low interest in the concept. When interviewing users or testing modifications to an offering that are not intended for sale, questions about purchase intent may not be applicable.
The results of the concept testing can best be synthesized into a scorecard made up of the following categories:
•Importance of solving the problem. Although this information is not directly related to the concept, it provides the buyer’s view on the importance of addressing the particular need discussed. The results may indicate that some segments place greater importance on meeting the need than others.
•Ability of the offering to address the need. Document interviewees’ indication of how well the offering addresses the need and whether they identified any barriers or prerequisites. If the selling organization was revealed during the research, capture feedback on the believability of the offering’s delivery.
•Comparison to alternatives. Indicate the most common alternatives interviewees use to address the need. If an interviewee’s company does not address that need, indicate whether he or she is aware of its existence. Provide ratings on how the concept compares to alternatives.
•Willingness to pay. Recap the results of the willingness-to-pay discussions, noting any differences between buying segments. These results can help drive offering configurations and bundling strategies.
•Purchase intent. Recap the feedback from purchase intent discussions, detailing buyer interest at various price points. Note that purchase intent feedback is often more optimistic than the purchasing that occurs in the market. If using this data for a business case, assume that only 50 percent of those who say they will purchase actually will.
One mistake organizations make is including only product management in concept testing. Although the offering in question is nowhere near ready for the market, asking marketing and sales to participate in these conversations helps the entire team better understand buyer needs, how to speak about those needs and how to quantify them. Exposure to the value conversation helps marketing and sales communicate the value of offerings more effectively once they are ready to go to market.