Critical Case Sampling: Check Why This Case Tests the Claim

Critical case sampling chooses a case because it has strategic bearing on a claim. The useful reading task is to trace why this case should tell you something about that claim, rather than accepting “critical” as a badge of importance.

A paper may choose a site where success should be especially likely or where it should be hard to achieve. Those choices carry different reasoning. Keep the claim, case conditions, and limits in the same evidence note.

What makes a case critical

A case can be a person, group, site, event, or process. It is critical relative to a question or claim, rather than simply large, urgent, or unusual. The selection argument explains why what happens there matters beyond the case itself.

Flyvbjerg's original methods argument defines critical cases by their strategic importance. It also stresses that finding such a case is difficult; a label does not settle the judgment.

Typical case sampling serves a different purpose: showing an ordinary instance. Laerd contrasts these purposes. A dramatic success or failure may be an extreme case without supplying a clear test of a broader claim.

The same case can serve more than one purpose. Ask which argument the authors actually make. A small sample size or limited budget alone does not establish that the chosen case is strategically decisive.

Trace the claim through the case

Read the stated claim alongside the case description and selection rationale. This table keeps each link in the reasoning visible rather than hiding assumptions in the word “critical.”

Table 1: Four links for checking a case-selection argument; this is a reading aid rather than a design score.

The Laerd guide says to extend findings with care. A case can help you judge a claim without showing how often the result occurs in a wider group. The force of the argument depends on which claim the case can test.

The worked synthesis table lists critical case choice as one way to select on purpose. For your paper, find the stated reason for the choice. A broad maxim cannot replace the facts that make this case a useful test.

A claim such as “this process works at all sites with trained staff” gives you conditions to check. “This is an important site” does not yet tell you what success or failure would mean for that claim.

Worked example at a favorable site

Imagine a fictional study of an advice service's new handoff checklist. The claim is that trained teams can use the checklist during their normal shift. The selected site has completed training and has ample time for handoffs. These facts and the report wording are invented for teaching.

The authors expect this site to be favorable for the claim because it has training and time. Suppose staff still cannot complete the handoff using the checklist. That finding would raise a serious question about the stated claim, subject to how well the study checked what happened.

Now change the outcome: the handoff works at that site. Success under favorable conditions does not show that it works where time is scarce or staff training differs. The educational account of critical selection needs this explicit condition when applied to a real argument.

If a report concludes “success here proves the checklist works at every site,” a bounded note could read: “The selected site had trained staff and ample handoff time. The reported success supports feasibility in that setting. It does not establish performance at sites with less time or different training.”

For a failure, the note would instead say that the outcome challenges the expectation of success at this favorable site. It would retain questions about use of the checklist and the study's evidence, rather than announcing that all sites must fail.

The example shows how the same selection rationale can give different weight to success and failure. It does not estimate an effect or document a real service outcome.

Keep wider inference conditional

“Most likely” and “least likely” must refer to a specific claim. Flyvbjerg explains this dependence: a case favorable for one proposition can have a different role for its opposite.

Check the facts behind the site ranking. Did staff get the same training at each site? Did more staff make this site unlike the rest? You need evidence about these conditions to judge the test, not just a label for the site.

New evidence can also change a case's role. In Flyvbjerg's account of his own case selection, deeper study changed a presumed critical case into an extreme case. Keep that possibility open when a paper's assumptions lack support.

Scribbr's sampling guide explains choosing a case for a decisive test. Use its examples to ask what makes the test work. A finding at one site does not give a rate for how often the outcome occurs elsewhere.

For comparison, homogeneous sampling narrows shared traits to study a group in depth. It does not need the same claim-testing rationale merely because the cases are similar.

Compare the case argument in Atlas

Add the claim passage, case description, methods rationale, and relevant guidance to one project. Use only documents you can share. Keep findings separate from what the authors expected before they studied the case.

The paper analysis workflow helps connect these passages with the broader research argument.

1. Select the sources with @. Ask: “What claim makes this case critical? Which facts about the case do the authors cite? What would success or failure mean for the claim? Keep source facts separate from assumptions.”
2. Open each citation and read the nearby text. Check whether the passage supports the case condition or merely states the author's expectation.
3. If the answer says “success proves it works everywhere,” inspect the extension passage. Revise the wording to retain the documented site conditions and untested settings.
4. Use New → Note to save the claim, case conditions, rationale, finding, corrected inference, and open question. Keep citations and wait for Saved.

The capture below shows an unrelated research paper beside a cited answer. It illustrates opening source context, rather than our fictional checklist study or a test of case selection. Check whether each passage supports a finding, a case condition, or only an expectation.

Open the cited passage before extending a finding beyond the conditions the case actually describes.

Screenshot: Atlas. Embedded paper: Yamada et al., The AI Scientist-v2, CC BY 4.0. Paper content unchanged within the product capture.

Save the argument and its limits

Keep what the study found apart from the reason it chose the case. Name an assumed condition that could change the meaning of the finding if it turns out to be wrong. This lets the next reader challenge the argument at the right point.

When reading several papers, synthesize their findings without merging distinct case rationales. Similar outcomes can come from studies designed to answer different claims.

In a literature review process, this is an appraisal of case choice inside a study. Your rules for choosing which papers to review remain separate.