Recall Bias: Compare the Memory Task Across Groups

Recall bias is systematic error when people's memories of past events are wrong or lack details. In a study with two groups, ask whether they faced the same memory task and whether their recall could differ.

The question goes beyond “Did the authors use one form?” People may be asked about different dates, get different prompts or know different things about the study.

Keep those details in a group comparison. A note about how people were asked does not measure the accuracy of their memories.

What recall bias means

The NCI definition concerns memories that are wrong or lack details. A forgotten detail can change an answer. That matters when answers are used to assess past exposure or compare groups.

Ask whether error differs between groups. People with an outcome may search their past for causes more closely than those without it. That could lead to different reports, but does not prove that either group recalls events falsely.

The Catalogue of Bias frames recall bias as systematic error. Check both ordinary lapses and errors that differ by group. Some missing memories do not by themselves show how much a group comparison was skewed.

Keep intention out of the diagnosis

A person can give an inaccurate answer while trying to recall events honestly. Do not treat a recall concern as proof of lying. Choosing an answer to look acceptable is a different response concern, even though both processes may occur in a survey.

Compare the recall task across groups

Period and time since the event

Find the period people were asked about. Was it last month, the year before a named event or their whole working life? Then find when they were asked. The past period and the time since it are two different checks.

NCI's guidance notes that time can affect recall. A survey taken months after a job change may pose a different task from one taken the next day. Write down the dates when the paper gives them.

Prompts, setting, and prior knowledge

Check the question stem, examples, calendar aids and follow-up prompts. Was one group shown a list of exposures while the other answered freely? Were interviews carried out in the same way?

Keep the plan in the protocol apart from what the report says took place. A stated plan to use the same prompts does not show that all staff followed it. If the concern is how staff interpreted or recorded answers, trace observer bias separately from the participant's memory task.

Also note what people knew at the time. Table 3-4 of the NTP handbook, on printed page 44, asks how knowing the outcome might affect exposure reports. Asking after an outcome does not, by itself, prove recall bias.

Worked check of past night work

Consider a made-up study of past night work among staff with and without a recorded fatigue outcome. The report says both groups used the same form. It claims that recall bias was thus avoided. This teaching case supplies no results or staff records.

Use these four questions to check whether the report supports that claim. Keep gaps in the methods apart from known differences between groups.

Table 1: The source details are fictional. A real comparison should cite the passage supporting each cell.

Correct the identical-form claim

Using the same form can keep wording consistent. It does not make the requested periods or time since events equal. Revise the claim to: “Both groups used the same form, but the supplied methods do not show matched recall windows or record checks.”

Do not turn a missing date into a known difference. If one group's interview date is absent, mark it as unknown. Ask for the schedule before deciding whether its recall task was harder or easier.

Keep source roles separate

The protocol may say that rota records will be checked. The report may only say that records were available. One describes a plan and the other a resource. Neither alone shows that each person's recalled shifts were checked.

A paper-analysis workflow helps you trace the plan, actual collection and final claim. Save each passage with its role instead of combining them into “exposure was validated.”

Separate recall risk from demonstrated bias

The misclassification guide asks how errors place people into the wrong groups. When past work is coded as “night work” or “no night work,” check how that label was assigned as well as the question asked.

Ask what the validation covers

Record checks can help when the records match the period and exposure being studied. Ask whose records were available, which details they contain and whether both groups had them. A rota can show planned shifts without proving which shifts someone worked.

The same NTP table asks whether an exposure measure fits the relevant time period. Check that before treating a second source as ground truth. A record from the wrong year does not check the requested period.

Leave direction and size unresolved

Recall error can skew a comparison, but which way it goes depends on the errors. The Catalogue's recall guide cautions that the observed link may grow or shrink. Do not claim recall bias must make a result larger.

A larger sample does not fill in missing event histories. More precise estimates can still depend on the same flawed recall task. A selection-bias check addresses a different question: who was invited, responded and entered the analysis.

A prospective label does not settle the issue either. As the specialist guide explains, a study following people forward may still ask them about earlier experiences. Check the specific measure, not just the design label.

Save a checked comparison in Atlas

1. Add the paper, protocol, questionnaire and any record-check or validation report to one project. Use permitted, suitably de-identified material. Keep group names and study versions consistent across files.
2. In Ask a question, type @ and select the sources. Ask: “For each group, what past period was asked about, on which date, with what prompts and in which setting? What did they know about the outcome? Cite each fact. Mark gaps and keep planned checks apart from reported checks.”
3. Open each numbered citation. Read the surrounding dates and group labels. If an answer treats the year before an event as the same as the previous calendar year, correct it. Preserve both time anchors in your note.
4. Revise the claim with the note above. For real material, keep record coverage marked as unknown until you find its passage. Use a source synthesis workflow to keep what the protocol and report say distinct.
5. Select New → Note, save the group comparison and wait for Saved. Atlas helps you organize the methods. It cannot recover forgotten events or turn these passages into a bias estimate.

The image shows source text beside a cited answer. Use that view to check dates and group names before saving a comparison. Its visible AI paper is unrelated to the night-work example; it does not show a recall study or memory reconstruction.

Check the time anchor and group named in the source before accepting a comparison of recall periods. Capture includes The AI Scientist-v2, Yutaro Yamada et al., CC BY 4.0, shown within Atlas; copied unchanged.

Ask for the missing time-window evidence

Ask for the schedule, full question stem or record-check report when one is missing. State what you know and the gap. “Interview date not supplied” gives the author a question they can answer. “Recall was biased” may go beyond your sources.

Keep the final note in three parts: reported facts, possible reasons for recall error and unknown effects. The NCI definition helps name the concern. The methods and any accuracy checks must support your judgment about the findings.