Case Control Study Selection and Bias Checks

A case control study starts with people who have an outcome and a comparison group who do not. Researchers then compare their earlier exposures. The hard part is choosing groups that make the comparison meaningful.

When you read a published study, trace how each group entered the sample. A familiar disease name or a large case count does not explain why the controls are suitable. You need the selection rules, dates, and source population.

The matrix below compares two CDC outbreak studies, with questions for a hypothetical foodborne-illness study in its final column. Use those questions in expert review before you settle the plan. They do not describe a real sample or an approved study.

What makes a case-control study

Following the CDC glossary's definition, the defining feature is selection by outcome. Cases meet the study's case rules, while controls do not have the outcome under the relevant selection rules. Controls come from the population that gave rise to the cases. Researchers then compare earlier exposures between groups.

Controls can have the exposure being studied. If the question concerns a food, some well controls may have eaten it. Removing those people because they ate the food would change what the group represents. The CDC field manual explains why control selection should be independent of the exposure.

A cohort study uses a different starting point: a defined population or exposure groups, with outcomes then assessed. A retrospective cohort study can use past records. Thus, "looks backward" is not enough to identify a case-control design. Check the sampling basis in the methods, as described in the CDC's comparison of analytic designs.

Typical case-control samples also do not give you the full population counts needed to estimate disease risk. An odds ratio measures an association between exposure and outcome. It is not automatically a risk ratio, and its meaning depends on the sampling design. The CDC measures-of-association lesson explains this distinction. Statistical interpretation belongs with a qualified analyst.

Compare case and control selection

Use the same fields for each paper so you can compare like with like. Record what the authors say, where they say it, and what remains unknown. If the text leaves out a detail, look for more sources before you judge the study.

Write down the case rules

Find the rules for the outcome, place, dates, and who can join the study. A short label such as "Salmonella cases" may hide age limits, lab test rules, or groups left out.

The CDC glossary describes a case definition as standard rules for who counts as a case. Write the scope in your own words and keep the source passage beside it.

For your own study plan, leave rules open until they have been reviewed. Ask which records would show that someone meets the case definition. A rule used in one study may not be sound for your setting. Also keep the full outbreak count apart from the sample used in one part of the study.

Identify where controls came from

The source population is the group in which the cases arose. Ask whether a control could have entered the case group if they had developed the outcome. Then check the route used to find controls. People in the same town may differ in who can be reached, who can join, and who agrees to take part.

The STROBE case-control checklist asks how cases and controls were found and chosen, why, and what matching rules were used. Find the passages that address those fields. If the report leaves out a step, seek a fuller methods paper to check whether it was done. If the report instead samples a baseline subcohort, the case-cohort study guide explains why sampled membership and outcome status need separate checks.

Worked selection and bias-check matrix

The first example is the February case-control substudy in the 2006–2007 Salmonella Tennessee report. The second is the January 17–19 study in the 2008–2009 Salmonella Typhimurium report.

Compare their reported choices before deciding which questions your own proposal needs to answer.

Table 1: The historical columns summarize the two linked CDC reports. The last column contains new review questions, with no claim that either published selection method fits the proposed study.

The 2007 report also describes a January survey before the February substudy. Its date window does not settle the missing detail in the February text. Keep that cell open and seek the study's fuller methods before comparing date anchors.

Read the groups before the result

The 2007 report records peanut-butter consumption among both cases and controls. That is a useful check on the meaning of "control." Well people can share a suspected exposure.

Read the reported exposure comparison before turning the control group into an "unexposed" group in your notes.

The report also describes lab tests of food and samples from the plant. Keep those findings in their own notes. A row about how controls were found does not itself show what caused the outbreak. For each claim you save, name the kind of source that supports it.

Keep sequential studies separate

The 2009 report describes an earlier January 3–4 study before the second study used above. In the earlier study, controls reported exposures for the week before their interview.

In the second, their window was anchored to the matched case's onset. The report's separate methods passages support that distinction.

Suppose your comparison note says both studies used interview dates for controls. Correct the second-study entry and record the passage that led to the change. Do not blend the two samples into one set of selection rules. This kind of correction is a reason to synthesize papers with their differences visible, rather than save one broad summary.

Check what could distort the comparison

Now turn each choice into a clear review question. Familiar terms in a paper are not enough to give it a score. Ask what could skew the findings and what the report says about that concern.

Check selection and participation

A control source may differ from the case source in ways tied to the exposure. The CDC field manual discusses concerns with hospital controls and with choosing friends or relatives who share habits.

These are reasons to inspect a particular choice, rather than declare one control source suitable for every study.

For the phone-based examples, ask who could be reached and who agreed to take part. The matrix records the contact route. It does not prove that this route could reach all people who met the rules.

Look for groups left out, failed contacts, and people who declined. The STROBE participant-flow fields help you find who reached each stage of the study.

Check memory, measurement, and matching

If people recall past events, ask whether illness, the dates they are asked about, or the wording could change their answers. A case may spend more time trying to recall a suspect food.

You need to check this concern. It is not a finding that either example had recall bias. Look for how each group was asked and whether the same methods were used. The STROBE measurement fields point you to those details.

Matching also needs a reason and an analysis plan. A confounder is a factor that can skew the link you are studying. Matching groups on a factor does not by itself remove its effect from the findings.

Shared exposure patterns can also make matching a poor choice. The CDC guidance on matching covers these limits. Ask an epidemiologist and analyst how the choices affect your study.

End with three lists: what the authors did, concerns to check, and choices still open in your plan. This keeps a concern about bias from turning into a claim that the study is flawed.

Seek full methods and later reports through your literature review process when a short report leaves a question open.

Review cited methods in Atlas

Add the selected reports, method guidance, and your own clearly labeled proposal note to one project. Start a fresh chat, mention the relevant sources with @, and choose Project only beside + to scope new retrieval. Earlier outside material can remain in an existing chat's context, which is why a fresh chat is useful for this comparison.

1. Ask: "Compare the February 2007 substudy with the January 17–19, 2009 study. Give case eligibility, control source, matching, exposure dates, and sample counts. Cite each entry. Keep my proposed rules separate."
2. Open the citations. Inspect the named study's methods and surrounding text. A citation opens a related source, with precise page or passage navigation when available. Find the relevant paragraph yourself when needed.
3. Check the date-window error described above. If the answer assigns the first 2009 study's interview-date window to the second study, ask for the second study's exact methods passage and correct your matrix. Mark unsupported entries as unknown.
4. Select New, then Note. Enter a title and paste your checked matrix, source locations, and open questions. Wait for Saved before closing. Return to the sources when a later report changes the evidence.

For the saved note, include the study and substudy name, the reported rule, its source passage, your proposed choice, the unresolved concern, and the person who must decide it. These fields connect the comparison to the next research meeting. Organizing research notes around their sources also makes later corrections easier to trace.

The source page and cited answer in this Atlas screenshot concern an unrelated research paper. It illustrates the source-inspection layout, with no claim that a case-control study was tested or approved in the product.

Atlas product screenshot showing an excerpt from The AI Scientist-v2: Workshop-Level Automated Scientific Discovery via Agentic Tree Search by Yutaro Yamada and colleagues, licensed under CC BY 4.0. Screenshot and excerpt reproduced without changes.

Take unresolved choices to expert review

Take the matrix to a research meeting once you have checked each source passage and marked your own choices. It can still have open questions. Before you borrow a rule, check how the study's people, outcome, and dates differ from yours.

STROBE supports clear reporting. Its checklist does not prescribe how to design or conduct a study and does not rate study quality. A completed set of reporting fields therefore cannot validate your control source or remove the need for expert review.

The STROBE checklist guide shows how to trace reported items back to the study text. Use it when checking a paper's reporting, while keeping your design and validity review separate.

The research team and experts must decide who counts as a case, how to sample, and how to measure and analyze the data. They also own consent, required review, sample size, and judgments about cause.

Atlas helps you compare sources and save a checked note. It does not recruit people, validate diagnoses, estimate odds ratios, or give clinical advice. Name who must make the next choice before study work begins.