External Validity: Test Whether a Finding Fits Your Setting

External validity asks whether a study finding may hold for other people, places, or times. Start with the setting where you want to use the result. Then compare it with the study.

Loyka and colleagues suggest four checks: people, setting, outcome, and time.

Atlas can compare the methods and findings in papers you choose. It shows citations so you can check each claim. You still define the target and judge which gaps could change the result. The matrix below shows how to keep that work in view.

What external validity means

External validity asks how far a study's finding can reach beyond the people and setting it studied. The new target could be another school, a wider group, or a later year. Name that target first. A claim that a finding “generalizes” needs a clear destination.

Trochim's methods guide frames the question around people, place, and time. A well drawn sample may speak for the group it came from. A new group or place needs its own check. Ask how that target differs from the study.

External validity versus internal validity

Internal validity asks whether the study supports its own claim. In a causal study, that means checking for rival causes, such as confounding or bias in who joined.

External validity asks whether the finding may hold elsewhere. A sound trial result for its own students may change in another school. A weak original claim stays weak even when the new school looks similar. The trial methods review treats these as two linked questions. Check the study's claim, then name the target and test the gaps. Carry any doubt about the study design into your final judgment.

Compare the study with the target context

Write a clear target question: “Could this tutoring plan help students finish first-term courses at a rural college next autumn?” That question names the students, place, program, result, and time.

Loyka and colleagues ask readers to check people, setting, outcome, and time. Also check how the program ran.

The World Bank discussion shows that who took part, how work was done, and one-off events can change an effect.

Ask which differences could change the result

Some gaps matter more than others. If tutoring uses live video, home internet access may matter more than age. A visible group gap may have little effect on this program. Explain why a given gap might change the result. A simple count of matching traits cannot do that work.

The Murad and colleagues primer draws a useful line in health research. A result may speak for the group from which the sample came. A doctor may also ask whether it fits one patient. Fields vary in how they use generalizability and transportability. State the target and claim in plain terms.

Keep a source location for each cell

Check who could join, how people were found, where the work took place, what was done, what was measured, and when. Take these facts from the report. If it omits one, write “not reported.” The blank matters: you cannot treat it as a match.

Work an external validity example

This example is fictional. A rural college may offer live online tutoring next autumn. The team wants more students to finish their first-term courses.

In Study A, volunteers at an urban college have higher quiz scores after six weeks of live video tutoring. Study B reports course completion at two suburban sites. It leaves home internet access unclear.

The two study reports are made up for teaching. For real papers, each cell would cite the methods or results section. Keep what the paper reports apart from your judgment about the rural college.

Table 1: The team's conclusion should stay narrow: These made-up reports raise useful questions. They do not show that the rural college will gain more course completions.

Study B measures the right result, but it says little about access. The team needs local access data, more detail on how tutoring ran, and perhaps a small local test.

The matrix points to missing facts that could change the choice. Loyka and colleagues urge readers to match the people, place, result, and time to the real-world claim. Better quiz scores give only indirect support for a course-completion goal.

Find threats before drawing a conclusion

A threat to external validity is a reason a finding may change in the new setting. Trochim points to people, places, and times. Each threat calls for a check. Its mere presence does not settle the result.

Selection and population

Ask who could join, who was asked, who joined, and who stayed. Volunteers may have more free time than other students. That could change how well tutoring works at the target college. Dropout may also weaken the study's own result, so inspect both questions.

Setting and implementation

Check how the program ran. “Online tutoring” could mean live video, short recordings, trained tutors, or varied help with access. The World Bank examples show why place, how a program runs, and one-off events matter.

Outcome and timeframe

Check the measure and the date. Quiz scores, course grades, and finishing a degree tell different stories. A result from a brief crisis may change in a normal year. The study still has value for its own question. It may offer less for the new one.

Strengthen the evidence for a new setting

Clear study reports help readers make this check. Report who was asked and who joined. Describe who was left out, how the program ran, what was measured, and when.

Loyka and colleagues urge study teams to plan and report with the real-world claim in mind.

The target needs its own data. How many students can get online? Can staff offer the same help? Does the college track the same result?

If a gap could change the effect, test it with a local pilot or work across more sites. Trochim notes that repeat studies in varied places strengthen the case.

Some formal methods can estimate an effect for a new group. They need data on that group, good overlap with the study, and strong assumptions about what changes the effect. The methods review explains those limits. A reading table alone cannot give that estimate.

Check study details with Atlas

Add study reports you may use to an Atlas project. In chat, type @ and pick the reports to compare. Ask: “For each study, list who joined, where it ran, what was done, what was measured, and when. Cite each reported fact. Mark missing details.” The Atlas public synthesis guide documents source selection and citation checks.

Open each key citation in the paper. Check that its text backs the matrix cell. Read nearby caveats. If Atlas blends two papers or fills a gap, ask it to separate them and mark the fact as missing.

Mentions point to project items; the source still needs your review. The research-paper analysis guide shows a related single-paper check. The source-checking guide shows how to test a claim against its cited passage.

Save a note with the target, each source location, gaps that may change the result, and open questions. The paper synthesis guide explains how to retain links between source claims. Atlas helps find and compare supplied text. You decide what it means for the target.

State what the evidence can support

Write the conclusion for the named target. Try this form: “For [people and place], [finding] may help guide [choice]. The sources support [matches]. We still need [missing facts or a local test] because [gap] may change the result.” This makes the limits clear.

Keep the study's own limits in view. The confounding-variable guide covers a rival cause that can weaken a causal claim. The operationalization guide helps check whether two studies measured the same result. For a program evaluation, contribution analysis tests the proposed path from intervention to observed outcome and its rival explanations. A close group match cannot solve those issues.

External validity is a judgment about one move from study to target. Check the source, name the gaps, and state how far the finding can go. The answer may change as new data arrive.