Market Research Challenges: Diagnose the Evidence Gap

The hardest market research challenges often come from an unclear decision, data that miss the target audience, or claims that have lost their source context. The remedy depends on the cause. A weak sample needs better evidence; two reports with different scopes may need a careful comparison first.

For example, a team may see one report favoring self-service onboarding and another favoring guided setup. Before choosing a direction, it should check whom each report studied, when, and what “favoring” meant.

Atlas can compare the selected reports with citations you open and check. An unsuitable sample still needs better data.

What makes market research fail

Start by locating the break in the chain from decision to evidence to claim. Four common failure modes call for different work: the question is vague, the people or instrument are a poor fit, the relevant sources are incomplete, or the final claim outruns what the sources show.

The Greenbook practitioner guide describes vague goals, weak samples, poor questions, late work, and findings that teams cannot use.

Those are symptoms. Find which one affects the decision at hand.

Table 1: The rubric is a triage tool. It does not rank the size of each problem. An unclear question can make a perfect sample irrelevant, while a clear question cannot repair a poor sample.

Diagnose the decision and sample

Write the decision as a sentence. “Should we offer guided setup to US small businesses next term?” is more useful than “What does the market want?” It identifies the choice, audience, and time frame. Then ask what evidence could change that choice.

Check who the data represent

Find the study's target buyers, sample source, recruitment method, field dates, and exclusions. A report on large firms may say little about small firms you have not reached.

A few interviews can reveal reasons to investigate. They do not show how common those reasons are in the market.

AAPOR's disclosure standards call for the population, recruitment method, sample size, dates, question wording, data quality steps, and limitations. These details let a reader judge whether the evidence fits the decision.

Inspect the question itself

A survey can recruit the right people yet ask a question that steers them. Compare the exact options and wording. Does “prefer self-service” mean choosing from a forced list, ranking several options, or describing an actual setup experience?

The Greenbook guide flags confusing questions and poor method fit. The ICC/ESOMAR code calls for research that fits its purpose and names its sources and limits. A summary cannot repair a bad survey question.

Resolve conflicting market evidence

Put each claim beside its source before calling the reports contradictory. Check geography, buyer segment, field date, population, definition, and method. Two headlines may use the same word for different measures.

Align the scope first

Suppose one fictional report covers US firms with fewer than 50 staff this year. Another describes interviews with UK enterprise buyers two years earlier. Their views may differ. That is a scope difference, so neither report directly overturns the other.

Even when the scope looks similar, read the method. Two surveys may reach different types of firms through different panels. AAPOR's disclosure requirements make those design details part of the report, not an optional appendix.

Investigate the remaining conflict

If both sources ask the same question of the same population in the same period, an opposite result needs closer work. Review recruitment, weighting, response options, exclusions, and the passage behind each headline. Keep the disagreement open if a method detail is unavailable.

The AAPOR Transparency Initiative promotes disclosure of research methods. AAPOR explicitly says that disclosure does not itself judge a study's quality. Use the disclosed details to make your own assessment.

For a broader method of comparing cases on the same questions, see comparative analysis. The scope check here serves a narrower decision: whether a discrepancy can be resolved from the available market sources.

Check data quality and provenance

Before trusting a headline, ask how the data were gathered. For surveys, check who joined, how they were screened, whether the questions were tested, and how poor responses were handled.

For interviews, check whom the team spoke to, what it asked, and whether quotes retain their context.

Read the method and limits

AAPOR's survey disclosure standards ask researchers to report checks for attention, logic, and duplicate responses. They also ask for exclusions and study limits.

The ICC/ESOMAR code keeps oversight with the people running the research, whatever tools they use.

These standards help you ask for the method. They do not certify a report just because it has a methods page. Check whether the disclosed design serves your decision and whether a stated limitation changes your conclusion.

Trace a claim to its passage

For each claim in the brief, save the report title, page or section, date, and a short account of what the passage says. Separate a source's observation from your inference. If the source is silent about a buyer segment, mark the gap. Silence cannot support a claim that the segment has no demand.

Scattered documents make this work slow. An industry guide from AlphaSense describes the effort of finding useful information. Its product remedy is a vendor claim. Your source trail still needs direct checks. A guide to cited AI answers explains why a citation starts the review.

If the source set is mainly published studies, synthesizing research papers can help turn checked findings into a coherent account. The sample and scope checks above still apply.

Choose the next research action

After the checks, make one of four decisions. State who will own the next step and what evidence would close the gap.

1. Clarify the question when the requested research does not map to a real decision, audience, or date.
2. Collect better data when the current sample, question, or coverage cannot answer it.
3. Reconcile existing sources when claims differ because of definitions, dates, or segments you can identify.
4. Retain a stated uncertainty when a material conflict remains and new data cannot be gathered before the decision.

When the checked finding points to a design problem, a How Might We statement can turn it into an open question for ideas. Keep the source and any counterexample beside the prompt so the team does not skip from one quote to a favored feature.

The fourth choice is still a decision. Say what the team knows, what it does not know, and how the unknown could change the action. Avoid giving a single confident answer just because a report has a sharp headline.

The ICC/ESOMAR code requires clear methods, sources, and limits. People remain responsible for the work.

If your next task is software choice after you name the evidence problem, see market research AI tools.

For a wider comparison of research workspaces, use the AI research assistants guide. Match the tool to the gap you found: source review, new data collection, or analysis.

A worked discrepancy log

The following reports are wholly hypothetical. The claims, dates, and document IDs were invented to show a review process. They are not observed market findings or a real Atlas output. A team is deciding how to support US small businesses during onboarding next term.

Table 2: The log changes the next action. A and B warrant a closer sample and question check. C supplies a possible question for new interviews, but it cannot vote in a US small-firm preference estimate. Nothing here justifies a fabricated percentage or a market-size forecast.

Write the decision brief

A checked note could say: “The supplied surveys make opposing preference claims for US small firms. Their eligible groups differ, and A's recruitment is not shown. We will request A's method and compare the overlapping segment before using either headline. C gives interview prompts, not a US estimate.”

That note retains the claim, its scope, the missing method, and the owner of the next step. It is more useful than merging the three headlines into one average.

Next step: check sources in Atlas

Add the reports you may use to one Atlas project. Mention the selected survey reports and interview summary with @. Ask for a source-by-source comparison using the exact fields in the log.

Compare these selected market reports on geography, buyer segment, field date, sample recruitment, question wording, and the meaning of “preferred.” Cite the passage behind each entry. Mark missing method details as unknown. Separate a scope mismatch from a remaining conflict, and list the next evidence request. Do not estimate market size.

Open each citation and read the method and surrounding text. If Atlas joins two sources or fills an empty field by guesswork, correct the log from the documents. Save the checked discrepancy and next-action note. The researcher still decides whether to seek new data or make a limited decision now.

The screenshot shows an unrelated research paper beside a cited Atlas answer. It illustrates opening a source for review. It does not depict these fictional market reports or validate the example claims.

Atlas can help compare the supplied text and retain checked notes. It cannot determine whether a sample represents the intended market, acquire missing respondents, validate a survey's statistics, or turn document agreement into market truth.