A manipulation check asks whether a study changed the state or construct it meant to change. A construct is the concept under study, such as stress. If a task aims to make people frustrated, the check might ask how frustrated they feel afterward.
The main outcome asks a different question, such as whether people tried longer on a later puzzle. Read the results with the intended state apart from both the check and the outcome.
Start a note with what the study tried to change, the check wording, its timing and the reported result. Then state what that evidence can support.
Trace manipulation-check evidence in Atlas
Compare methods and results passages, then save a qualified evidence note.
What a manipulation check measures
Researchers often try to change an internal state through a task. They can give people a frustrating puzzle. Giving that puzzle alone does not show that people felt frustrated; the study needs evidence about their response.
The Experiment Basics textbook describes a manipulation check as a separate measure of the construct being changed. For a study of stress, that measure should address stress.
The check should address the proposed state. A question about what an instruction said may show that a person understood it. It need not show how frustrated they felt while doing the task.
Read the actual measure, rather than relying on the label. The Wiley reference entry describes checks of perception, comprehension or response to the manipulation. The wording tells you which of these a check targets.
Separate delivery, state and outcome
Delivery asks whether people received what the study meant to give them. A log can show that a video played, while leaving open how the person felt about it.
The state question asks whether that video changed the feeling under study. A sadness rating asks about a different fact from the playback log, even if the paper calls both checks.
The outcome answers the main research question. If the study asks about later helping behavior, the sadness rating and the helping measure serve different roles. Keep each measure with the question it was meant to answer.
In intervention research, treatment fidelity asks whether the treatment was delivered as intended. It helps keep evidence of delivery apart from evidence of the state it induced.
An attention check asks whether people follow instructions or attend to material. Passing an instruction question should not become evidence that a task made someone sad.
When labels overlap, use the wording and purpose to identify what a measure checks. Keep the authors' label in your note, then explain the narrower claim the procedure supports.
Work through a check evidence note
Imagine a fictional study with frustrating and neutral puzzle instructions. People rate their frustration right afterward, then try a persistence task. The report says frustration ratings were higher in the group given frustrating instructions.
The table below links each part of the report to the claim it supports. The passage labels are fictional teaching examples. No effect size or test result is invented. The table keeps the procedure, the recorded answer and the reading of that answer apart.
| Note component | Fictional passage | Supported reading | Limit to preserve |
|---|---|---|---|
| Intended manipulation | Methods paragraph 1 describes puzzle instructions | Instructions were intended to alter frustration | Intention does not establish the state changed |
| Check measure | Methods paragraph 2 gives a frustration rating | Self-reported frustration was measured | Exact wording and scale properties still matter |
| Check timing | Procedure places the rating before persistence | The rating preceded the main outcome | Completing the rating may affect later responses |
| Reported comparison | Results paragraph 1 says frustration was higher | The report describes a condition difference on the check | This alone does not establish the outcome mechanism |
Table 1: Each row keeps what the passage says apart from what you can infer from it.
“The frustration mechanism was confirmed” goes too far. The check compares a measured state. It does not itself show that frustration caused any difference in persistence.
Write instead that people reported more frustration in the named group at the stated time. Keep the main outcome and any claim about why it changed in their own sentences, with their own evidence.
If the paper gives an effect size or confidence interval, copy it with the comparison it describes. Do not use the outcome's statistic for the check just because both name the same groups.
If the check wording is missing, record its label and that gap. You can describe what the authors meant to measure without claiming the report shows how well the item measured frustration.
Read timing and validity limits
Asking someone to name their feelings can change what happens next. A check between the manipulation and outcome becomes part of their experience. It is not always a neutral way to observe them.
Hauser, Ellsworth and Gonzalez discuss this reactivity problem. A clear check can draw attention to the study's aim or alter the state the later task is meant to reflect.
Record whether the check came before, during or after the main outcome. Also keep a separate validation study apart from a check in the main experiment: the people and tasks may differ. Counterbalancing addresses the orders people follow; it does not show that a task changed their intended state.
Evidence from a pilot can support the account of a task. Keep its group and setting with the claim. A check after the outcome measures a later state, which may differ from how people felt during the task.
Chester and Lasko's construct-validation research examines how studies checked the constructs their tasks were meant to change. A plausible check item alone is not a full body of evidence that it measured the intended concept.
One frustration item may cover only part of that concept. Look for the authors' validation basis and evidence about other states the task might change. A single check does not rule out every other explanation.
A nonsignificant check does not prove the task had no effect. What it means also depends on the measure, uncertainty and design. Take questions about that analysis to a methods specialist rather than applying one rule to every study.
Do not automatically regroup people by their check answers in place of the randomized comparison. Record any exclusions or regrouping the authors used and the reasons they gave. A reviewer can then assess what those changes did to the comparison.
Compare the check passages in Atlas
Add the methods, results, supplement and validation material to an Atlas project, where you have permission to use them. Wait for processing to finish. Open a chat and use @ to select the sources about the check.
Ask: “Find the intended state, check wording, timing and reported group comparison. Cite each field. Keep the check apart from the main outcome and flag missing wording or timing.”
Open each citation and read the text around it. Make sure the passage is about the check, rather than the main outcome, an attention question or a delivery log. Keep the exact wording when the paper combines several measures.
Atlas interface screenshot showing source inspection. The visible ColPali paper by Manuel Faysse and colleagues is CC0 1.0; it is unrelated to the fictional frustration study. Lossless WebP re-encoding left the content unchanged.
If Atlas calls the check “validated,” open the cited source and see what it supports. Replace claims that lack support with the reported item and the evidence the source gives for it.
For the worked example, replace “mechanism confirmed” with the narrower claim about frustration ratings. Ask a follow-up for the exact check passage and a separate account of the persistence outcome.
Use New then Note to save the corrected statement and source references. Wait for Saved. Keep open questions beside each entry, so the reported finding and its limits stay together when you return to the note.
Use the same fields across papers, but keep each study's measure and timing. The research synthesis guide helps keep each source's findings clear in a comparison.
Write the supported interpretation
Name the intended state, how it was checked and when. State the reported comparison without turning it into a claim about every person or every possible cause of the outcome.
For the fictional study, write that the report describes higher self-rated frustration after the named instructions. The check came before persistence, so it may have affected how people responded next.
Keep the persistence result apart, with its own statistic and passage if given. If the authors propose frustration as the mechanism, attribute that claim and name the further evidence they give for it.
Before using the note in a review, check that each claim about a changed state points to the right measure. Keep missing wording, unclear timing and gaps in validation visible for the next reader.
Trace manipulation-check evidence in Atlas
Compare methods and results passages, then save a qualified evidence note.