All posts

Concept testing: the complete guide to validating ideas before you build (+25 questions to ask)

Tania ClarkeUpdated July 202612 min
The dwarf galaxy LEDA 42160 glowing with bright bluish star-forming patches on its lower-right flank as it plows through the dense gas of the Virgo cluster.
Guide

Concept Testing

kill bad ideas in days, not quarters

Hubble · LEDA 42160 · 2024 · NASA
On this page

Concept testing helps answer a simple question: is this idea worth building?

It involves presenting an early concept, such as a sketch, mockup, wireframe, or written description, to your target audience before investing in development. The goal is to understand how people respond before significant time and resources are committed.

Concept testing helps teams validate ideas early, identify potential issues, and prioritize the concepts most likely to succeed. This guide explains how concept testing works, the four test structures, the most common research methods, what to measure, how many participants you need, and the questions that produce the most useful feedback.

TL;DR

Concept testing is the process of validating an idea with your target audience before investing in development. It helps product teams evaluate demand, identify potential issues, and make more informed product decisions before committing engineering resources.

An effective concept test includes:

  • A concept testing structure (monadic, sequential monadic, comparative, or proto-monadic)
  • The right research method, such as surveys, moderated interviews, unmoderated video, A/B preference testing, or fake door testing
  • Clear success metrics, including appeal, relevance, purchase or usage intent, uniqueness, believability, and willingness to pay
  • Representative participants who reflect your target audience
  • Predefined decision criteria to guide next steps

Common mistakes include testing too many concepts at once, using polished mockups that discourage honest feedback, recruiting the wrong participants, and interpreting results without clear success criteria.

What is concept testing?

Concept testing is a research method that evaluates an unfinished idea with your target audience before development begins. It sits between ideation and product development, allowing teams to validate concepts before investing in design and engineering.

A concept can take many forms, including a wireframe, written description, Figma mockup, or short video walkthrough. The format matters less than whether participants can understand the idea well enough to provide meaningful feedback.

You may also hear concept testing referred to as idea testing, concept validation, or concept research. Testing a feature that doesn't yet exist within your live product interface, such as adding a button for a feature that's still in development, is commonly called fake door or painted door testing, one of the methods covered later in this guide.

Concept testing vs usability testing vs A/B testing

Although these methods are often confused, they answer different questions at different stages of product development.

  • Concept testing: Would people want this to exist? Conducted before anything is built.
  • Usability testing: Can people successfully use it? Conducted with a prototype or working product.
  • A/B testing: Which version performs better? Conducted after a feature has launched using live traffic.

Rather than replacing one another, these methods work together throughout the product development process. Concept testing validates the idea, usability testing evaluates the experience, and A/B testing optimizes the finished product.

The four ways to structure a concept test

Before selecting a research method, decide how many concepts each participant will evaluate and in what order. This decision influences the quality of your results and aligns with the terminology commonly used in market research.

Monadic testing. Each participant evaluates a single concept. Because participants aren't comparing multiple options, the results are less susceptible to comparison bias. The trade-off is that each additional concept requires a separate participant group, increasing the required sample size.

Sequential monadic testing. Participants evaluate multiple concepts one at a time, providing feedback on each before moving to the next. This approach is more efficient than monadic testing but introduces the possibility of order bias and participant fatigue. Randomizing the presentation order and limiting sessions to two or three concepts can help reduce these effects.

Comparative testing. Participants compare two or more concepts directly and choose their preferred option. Comparative testing works well when selecting between similar alternatives, although it explains which option participants prefer rather than whether any individual concept is compelling on its own. Follow-up questions help uncover the reasoning behind each choice.

Proto-monadic testing. Proto-monadic testing combines both approaches. Participants first evaluate each concept independently before comparing them directly. This provides both individual concept ratings and head-to-head preferences, making it useful for higher-stakes product decisions where both perspectives are valuable.

Here's how the structures map onto the practical methods in the next section:

Structure
What it answers
Best paired with
MonadicIs this concept effective on its own?Surveys, moderated interviews
Sequential monadicHow do several concepts perform individually?Surveys, unmoderated video
ComparativeWhich concept do participants prefer?Preference tests, surveys
Proto-monadicWhich concept performs best overall?Surveys with a preference comparison

Five ways to run a concept test

The structure determines how concepts are presented. The method determines how you collect participant feedback.

Survey-based testing

Participants review a concept and answer structured questions about it. Surveys are fast, scalable, and easy to quantify, although they typically provide less context than interview-based methods. Careful participant screening is often more important than increasing sample size.

Unmoderated video testing

Participants record their screen and voice while interacting with a concept independently. This approach captures reactions, tone, and reasoning without requiring live scheduling, making it a practical option for collecting qualitative feedback at scale.

Moderated interviews

Researchers meet with participants one-on-one to discuss a concept and ask follow-up questions in real time. Although interviews require more time and coordination, they often produce the deepest insights because researchers can explore unexpected responses as they emerge.

A/B preference testing

Participants compare two concepts and select the option they prefer. This method works well when choosing between alternatives, particularly when the concepts differ in only one or two meaningful ways. Follow-up questions help explain why participants made their selection.

Fake door testing

Fake door testing introduces a feature that doesn't yet exist into a live product experience and measures real customer behavior, such as clicks or sign-up interest. Because participants are responding in a real product environment rather than a hypothetical scenario, this approach can provide a strong indication of actual demand.

What to measure: the six signals

The success of a concept test depends on measuring the right signals. Before writing survey or interview questions, identify what you want to learn so every question supports a clear research objective.

Appeal. How do participants respond to the concept overall? Appeal is typically measured using a 5- or 7-point rating scale and should always be paired with a follow-up question to understand the reasoning behind the score.

Purchase or usage intent. Would participants actually use or buy the product? While intent is one of the strongest indicators of potential demand, self-reported responses should be treated as directional rather than definitive.

Relevance. Does the concept solve a meaningful problem for your target audience? A concept can be appealing without addressing a genuine customer need.

Uniqueness. How different is the concept from existing solutions? High appeal combined with low uniqueness may indicate the idea doesn't offer meaningful differentiation.

Believability. Do participants believe the concept will deliver on its promise? Low believability may reflect unclear messaging rather than a weak product idea.

Willingness to pay. What are participants willing to exchange (whether money, time, or effort) to gain the benefits of the concept? Treat this as an indicator of perceived value rather than a pricing exercise.

Use a combination of scaled questions for quantitative comparison and open-ended questions to understand the reasoning behind participant responses.

How many participants do you need?

The ideal sample size depends on your research method and whether you need directional feedback or statistical confidence.

Method
Directional signal
Statistical confidence
Moderated interviews5-8 participantsNot intended for statistical analysis
Unmoderated video8-15 participantsNot intended for statistical analysis
Survey (single concept)30-50 participants100+ per concept
A/B preference30-50 participants100+
Fake doorUntil meaningful behavioral data is collectedVaries by traffic volume

Two principles matter more than any number:

  1. Participant quality outweighs sample size. A small group of representative customers often produces more valuable insights than a much larger group with little connection to your product. Recruiting from your own customer base (through a research CRM rather than an external panel) can significantly improve the quality of feedback.
  2. Match the sample size to the decision. Directional feedback is often sufficient for early-stage concept testing. Reserve larger sample sizes for decisions where statistical confidence is necessary.

Why concept testing matters (and when to skip it)

Concept testing helps teams validate ideas before investing significant time and development resources. Identifying weak concepts early can prevent costly product decisions and focus investment on ideas with stronger customer demand.

It also improves decision-making by replacing assumptions with customer evidence. Rather than relying solely on internal opinions, teams can use research findings to guide product prioritization and feature development.

Concept testing isn't always necessary. If customer behavior already provides a clear answer (for example, analytics consistently show users attempting an unsupported task), additional concept testing may add little value. Likewise, small interface improvements are often better evaluated through A/B testing after launch.

How to run a concept test

Set decision criteria before you start

Define success before collecting data. Establish clear thresholds for moving forward, iterating, or stopping a concept so results can be evaluated consistently.

For example:

  • 60% or more positive responses: Move forward.
  • 40-60% positive responses: Refine the concept and test again.
  • Below 40% positive responses: Reconsider or discontinue the concept.

Document these criteria before the study begins to reduce bias when interpreting results.

Choose your structure and method

Start by selecting the appropriate concept testing structure (monadic, sequential monadic, comparative, or proto-monadic), then choose the research method that best supports your objectives.

  • For speed, use surveys or preference tests.
  • For deeper qualitative insights, conduct moderated interviews.
  • For a combination of qualitative and quantitative feedback, begin with 8-15 unmoderated video sessions followed by a survey of 30-50 participants, or 100+ per concept if you need statistical significance.
  • For real-world behavioral validation, consider a fake door test.

Recruit your actual customers

Whenever possible, recruit participants from your existing customer base rather than relying solely on external research panels.

Customers already understand your product, workflows, and challenges, making their feedback more relevant than responses from general panel participants. For example, ServiceNow reduced participant recruitment time from 118 days to six after shifting from external panels to its own customer research CRM.

If external participants are necessary, use detailed screening criteria to ensure they closely match your target audience.

Keep concept materials rough

Early concepts don't need polished visuals. In fact, lower-fidelity concepts often encourage more honest feedback because participants recognize the idea is still evolving.

Wireframes, sketches, and written descriptions can help shift attention toward the concept itself rather than visual design details.

Ask questions that require real thinking

Avoid questions that encourage hypothetical or overly positive responses, such as "Would you use this?"

Instead, ask questions that prompt participants to think about real situations and decision-making:

  • "Walk me through how you'd use this in your current workflow."
  • "What would need to change for you to switch from your current solution?"
  • "Would you sign up for the beta waitlist today? Why or why not?"
  • "What was the first thing that confused you?"
  • "If you could keep only one part of this concept, what would it be?"

25 concept testing questions to ask participants

The questions you ask should align with your research objectives. The examples below are organized by the type of insight they help uncover.

First impressions

  1. What's your first reaction to this?
  2. What do you think this product or feature does?
  3. Who do you think this is designed for?
  4. What stands out to you most?
  5. Does anything confuse you at first glance?

Value and relevance

  1. Would this solve a problem you currently have?
  2. How would you use this in your day-to-day work?
  3. What would you expect to give up to get something like this?
  4. How does this compare to what you're using today?
  5. What would make this more useful to you?

Comprehension

  1. In your own words, what does this do?
  2. What features do you expect to see?
  3. Is anything missing that you'd need before using this?
  4. What would you need to know before trying this?
  5. Does anything feel unnecessary?

Preference (for comparative and proto-monadic tests)

  1. Which version do you prefer? Why?
  2. Which feels easier to understand?
  3. Which would you be more likely to try?
  4. What does version A do better than version B?
  5. If you could combine elements from both, what would you keep?

Purchase intent and next steps

  1. Would you sign up for this today? Why or why not?
  2. What would stop you from using this?
  3. Who else on your team would need to be involved in this decision?
  4. What would you need to see in a demo or trial?
  5. On a scale of 1-10, how likely are you to recommend this to a colleague?

Select the questions that best support your research objective rather than asking all 25. A focused set of 5-8 questions often produces deeper, more actionable insights.

Analyze and make the call

Compare your results against the decision criteria established before the study began. Concepts that meet your success threshold can move forward, while others may require iteration or should be reconsidered altogether.

Making the decision is often the most challenging part of concept testing. If participants consistently express confusion, limited interest, or difficulty explaining how they would use the concept, those findings should inform the next decision rather than be dismissed. Acting on the evidence gathered is what gives concept testing its value.

Where AI fits in concept testing

AI has become a practical part of concept testing, helping teams both conduct research and analyze results more efficiently.

Running the test

Traditionally, unmoderated concept tests captured only a participant's initial response. AI moderation expands on that by asking follow-up questions in real time, helping researchers understand the reasoning behind participant feedback without requiring a live moderator. This allows teams to collect richer qualitative insights while maintaining the speed and scalability of unmoderated research.

Synthesizing the results

Analyzing large volumes of qualitative feedback can quickly become time-consuming. AI theme clustering helps identify recurring patterns across interviews and survey responses, making it easier to surface common themes and objections.

Great Question automatically groups similar responses into themes and links each insight back to the supporting transcript or video clip. This gives teams a faster way to identify patterns while maintaining a clear connection to the underlying evidence.

AI doesn't replace researcher judgment. Instead, it reduces the manual work involved in organizing and synthesizing research so teams can spend more time interpreting findings and making product decisions.

Common mistakes

Testing too many concepts at once. Evaluating too many ideas in a single study can lead to participant fatigue and lower-quality feedback. Limit each round to two or three concepts whenever possible.

Over-polishing the concept. Highly polished mockups can shift attention toward visual design rather than the underlying idea. Early concepts such as wireframes or sketches often encourage more candid feedback.

Skipping decision criteria. Define success before collecting data so results can be interpreted consistently rather than retrospectively.

Ignoring order bias. In sequential monadic and comparative tests, presentation order can influence participant preferences. Randomizing the order helps reduce this bias.

Ignoring negative feedback. Critical feedback is often the most valuable outcome of concept testing. If participants consistently struggle to understand or see value in a concept, those findings should inform the next product decision.

Concept testing tools

The right tool depends on your research method and participant recruitment strategy. Different methods require different capabilities.

Surveys need image support, structured questionnaires, and audience segmentation. Unmoderated video needs screen and audio recording, playback, and AI-assisted highlights. Moderated interviews require scheduling, recording, transcription, and research organization.

Beyond individual features, consider how research is stored and shared over time. A centralized research repository makes it easier to revisit previous concept tests, compare findings, and build on existing customer knowledge.

Rather than relying on separate tools for recruitment, study execution, and analysis, many teams benefit from consolidating these activities into a single platform. This makes research easier to revisit and apply to future product decisions.

Organizations such as Asana have reduced recruitment timelines from approximately two weeks to two or three days, while Brex has expanded research participation from single digits to more than 100 people running research in the first year.

A concept test in practice

Imagine a B2B SaaS company evaluating three approaches for a new reporting dashboard:

  • A customizable drag-and-drop builder
  • Pre-built report templates
  • An AI-generated reporting experience

The team creates a simple wireframe for each concept and runs a proto-monadic survey with 75 existing customers. Participants evaluate each concept individually, rate its appeal, then select their preferred option and explain their reasoning.

Although the AI-generated concept receives the highest appeal scores, customers indicate they're more likely to use the pre-built templates because they feel more reliable for stakeholder reporting.

Based on those findings, the team prioritizes pre-built templates for the initial release and plans AI-generated reporting as a future enhancement.

The same approach can be applied in other contexts. For example, an e-commerce company considering built-in shipping insurance could run a fake door test by adding a "Protect this shipment" option during checkout that links to a "Coming soon" page. Measuring clicks and waitlist signups provides behavioral evidence of customer interest before development begins.

Frequently asked questions

What is concept testing?

Concept testing is a research method that evaluates an unfinished idea with your target audience before development begins. It helps determine whether a concept is worth investing in before significant time and resources are committed.

What are the main types of concept testing?

Concept testing can be categorized in two ways: test structure and research method. Test structures include monadic, sequential monadic, comparative, and proto-monadic testing. Research methods include survey-based testing, unmoderated video, moderated interviews, A/B preference testing, and fake door testing. The structure determines how concepts are presented, while the method determines how participant feedback is collected.

What should a concept test measure?

An effective concept test measures six key signals: appeal, purchase or usage intent, relevance, uniqueness, believability, and willingness to pay. Pair quantitative ratings with open-ended questions to better understand the reasoning behind participant responses.

What's the difference between concept testing and usability testing?

Concept testing evaluates whether an idea is worth building before development begins. Usability testing evaluates how easily people can use a prototype or finished product. Concept testing comes first, followed by usability testing once a concept has been validated.

How many participants do I need?

Unmoderated video: 8-15. Surveys: 30-50 for directional data, 100+ per concept for statistical significance. Moderated interviews: 5-8. Quality of participants matters more than quantity. If you're recruiting from your own customer base through a research CRM, even smaller samples produce higher-quality signal because participants have real context.

How long does a concept test take?

Survey: 3-5 days. Unmoderated video: 5-7 days. Interviews: 1-2 weeks. Preference test: 1-3 days. Fake door: 2-6 weeks. The biggest time sink is recruitment, which is why having a customer research panel ready makes such a difference.

What are good concept testing survey questions?

Focus on questions that encourage participants to think about real behaviors rather than hypothetical situations. For example:

"Walk me through how you'd use this in your workflow."

"What would you stop using if you had this?"

"What would need to change for this to fit how you work today?"

Questions grounded in real scenarios typically produce more actionable insights than asking, "Would you use this?"

Share