Implementation Research With a Worked Evidence Matrix

Implementation research studies how a policy, program, or practice is put into routine use. It asks what helps or hinders that work, what support people use, and what happens during delivery.

A report may say staff lacked time and a team planned shorter training. That records a concern and a plan. It does not show that training happened, solved the concern, or helped staff use the practice.

When reading studies together, keep each barrier, setting, change, and result tied to its source. The worked matrix below shows how. Atlas can help compare chosen records; you check their passages and save the note.

What implementation research asks

The focus is how a practice enters daily work: who uses it, with which support, in what setting, and for how long. WHO TDR's training guide starts with gaps in delivery, people involved, and study questions.

Whether a practice works is distinct from how people start and keep using it. Some studies ask both questions, as Mash and colleagues explain, but evidence of use alone does not show that the practice works.

Staff may start using a referral form while people still miss the help they need. To see both parts, keep records of staff use and what followed for visitors, with clear dates and units.

A process evaluation reviews how a program ran and can contribute to implementation research. When the question and design call for it, the inquiry may also compare support, settings, and reasons why use differs across sites.

Separate the practice from its support

Identify what is being introduced

Name the practice and its purpose before labeling barriers. In an invented library, that might be a referral from the front desk to a digital-help session. The referral and session are the new service.

Keep the practice version and steps with the source, since a shared program name can hide changes. A branch using a booking form may have a different version from one offering walk-in help.

Describe how people are supported

Staff briefings, prompt cards, and feedback on missed referrals could help staff use the service. Those supports are implementation strategies. The IRLM authors separate them from the practice itself.

Changing when staff get a briefing changes that support. Changing how a visitor gets referred changes the practice itself. Keep the two apart so a later review can say what changed.

Keep context and outcomes visible

Desk cover, booking systems, staff roles, and transport may affect use. Together, such conditions form the context. Staff can describe a problem, though an account alone cannot show its size or cause.

The context and design paper treats context as something that can change during a study. Keep its dates beside the finding rather than assume the setting stayed fixed.

The IRLM figure groups factors that might hinder or help use as determinants. Strategies supply support; mechanisms explain how that support might work. Its arrows lead to outcomes, but show a proposed path rather than proof.

Original Figure 1 by Smith, Li, and Rafferty, unchanged under CC BY 4.0. Its empty fields help plan a study, using 2020 domain labels that predate the updated CFIR.

Frame a question the sources can answer

Start with a decision

Write the choice behind the review. A team might need to decide what to check before changing staff support at another branch. That is narrower than asking which support works best everywhere.

The research objectives guide helps turn a broad topic into questions. Here, ask which barriers the records describe, what changed in response, and which results were measured.

Define the comparison unit

Choose whether each row concerns a study, site, team, period, or change. A paper can describe several branches, so one row per paper may merge findings that differ. Add a site or period to the source ID when needed.

Keep the group beside the unit. Staff who run a service and people who need it may face different barriers. A manager's account cannot stand in for every visitor's view.

Record stage and source coverage

Keep planning, early delivery, and later routine use apart. A meeting note can explain why a team chose training. It cannot show whether the service continued after the launch team left.

For each source, record its purpose, method, sample, dates, practice version, and passages used. WHO TDR's course includes methods, ethics, and working with the people involved. A matrix cannot replace those study tasks.

Choose methods and frameworks for the evidence

Match the method to the question

Interviews can show how staff describe a barrier, while watching the desk reveals what happens there. Logs may add referrals over time, so the methods address different parts of the question while each misses others.

To test a causal effect, choose a design that can address other causes. The context and methods paper discusses tradeoffs rather than one best design. Keep the study's limits when reviewing its records.

Choose a framework for its job

CFIR groups factors that may hinder or help use. The 2025 developer guide says it does not prescribe the delivery process or develop the practice. Choose terms that fit while leaving room for findings outside them.

An IRLM can map proposed links among those factors, support, and results. FRAME records changes to the practice; FRAME-IS records changes to the support strategies. These tools have different jobs and do not replace a study design.

Specify what the outcome means

Proctor and colleagues' outcome framework includes acceptability, adoption, feasibility, fidelity, cost, and sustainability. State what was measured, whose view it represents, its unit, and date.

Staff may like the referral process yet never start using it, which separates acceptability from adoption. Fidelity asks whether they do it as intended. Those measures still leave open whether visitors gained digital skills and whether the service caused any gain.

A worked implementation evidence matrix

Read the fictional source pack

Imagine a library introducing referrals to digital-help sessions. Study report S1 covers Branch A's launch; report S2 covers Branch B's later start. Context note C1, adaptation note A1, and outcome appendix O1 accompany the reports.

All five records, IDs, and findings below are invented teaching material. They are not real studies, quotes, measured Atlas output, or proof that this service works. The example shows how to read sources together.

S1 says staff felt pressed for time. Its desk note says referrals were missed even on a quiet shift. S2 describes staff concerns about desk cover. The issues may be related, but these accounts do not prove a single cause.

The matrix keeps context, change status, and limits next to each source. In your review, attach the page, passage, table, or permitted field-note location so readers can check each row.

Table 1: Fictional source comparison: the five rows preserve conflicting evidence, planned and delivered changes, and outcomes the records do not establish.

Revise the cross-source claim

A draft might say both branches lacked time and shorter training helped staff start using referrals. The rows do not support that claim. Branch A's quiet-shift note challenges the time account, and Branch B only proposed repeat briefings.

A narrower reading is that the records describe pressures around referral use and briefing access. One records a prompt card, another a plan, and a later note a rota change. Their effects remain unclear.

The IRLM paper notes that barriers and strategies rarely have one-to-one links. Here, a training plan may not address desk cover, form access, and every missed referral.

The current CFIR guide supports choosing factors that fit the study. Keep the quiet-shift case visible before giving all the accounts one shared label.

Trace adaptations before comparing outcomes

Name the changed object

Keep the old version and describe the new one. A1 moves the referral form into a desk system, changing how the practice is done. S2's planned repeat briefing changes how staff would be supported.

Choose the framework to fit the object that changed: FRAME tracks changes to the practice, while FRAME-IS addresses support strategies. Keep that object clear in the source record.

Keep the reason apart from the result

Record when a change was proposed, decided, and delivered, who took part, and why. If the note says it aimed to make referral easier, report that aim. Do not turn it into proof that referral became easier.

FRAME includes timing, people who chose the change, reasons, and links to fidelity. Keep fidelity unknown when you lack evidence about the steps that must stay intact.

Preserve the sequence

Put context and support changes on the same dated record. If desk cover improved before repeat briefings, later referral use could reflect either change, both, or another factor. Keep those options open unless the design can separate them.

FRAME-IS helps record changes to support strategies. The record can show the sequence without testing which change caused a result.

Keep explanations within the study design

Compare like measures

One report may count referrals, another staff using the form, and a third staff who intend to use it. These measures have different units and states. Putting them in one adoption column does not make them alike.

Proctor's outcome paper calls for clear measures, levels, and timing. Keep the source definitions beside the records before stating a pattern or pooling values.

Retain missing and opposing accounts

Silence about a barrier may mean it was not asked about, not reported, or absent. Mark it unreported until the source gives you a stronger basis. Check whether the studies asked similar questions and heard from people who stopped using the practice.

O1 leaves visitor skill results open, since a referral can still lead to a missed session. People unable to reach the desk may never appear in its log, leaving another gap in who the record describes.

Separate a pattern from a mechanism

If referral use rose after training, the sequence gives you a question to test. It does not rule out changed staffing, a new form, different visitors, or new logging rules. Report the pattern within the design's limits.

Mash and colleagues keep outcomes of use apart from effects on the people served. Keep those separate even when a report uses the broad word success for both.

The program theory guide helps state why a team expects support to change practice. Keep that path as a proposal until evidence tests it. It can guide the next study without becoming a finding.

Check what a tool study tested

The original IRLM study asked training attendees how useful they found the model. That is evidence about their reported experience with the tool, rather than a test that using it improved service results.

Apply that same question when reading a framework's evidence. Who used it, what did the study measure, and what was the comparison? A tool may help a team lay out its reasoning while leaving the proposed cause, the quality of its sources, and the effects on people open.

Compare the selected records in Atlas

Choose permitted sources

Open a project for the comparison. Add records you may use, with source IDs, dates, practice versions, and site names. Remove names and other details when the study's access rules require it, and keep the approved originals elsewhere.

Wait until sources finish processing. Start a chat, type @ in Ask a question, and select the reports and notes. Mention the context record as well as the study report so the request names both.

Choose Project only beside + to keep new searches within the project. Earlier chat context remains available, including outside sources already there. Start a fresh chat if that context could blur the chosen source set.

Ask for a source-specific matrix

Use a request that fixes both the comparison and the limits:

Compare these records for barriers to using the referral process. Keep each site and period separate. For each row, cite the reported barrier, context, response, and outcome. Distinguish a proposed response from confirmed delivery and a measured result. Retain conflicting accounts and unreported details. Do not infer that training caused better adoption or that referrals improved visitors' skills.

Select Send, then check what each source adds. If an answer combines accounts into one theme, ask for the separate passages before accepting it. A shared label can hide dates or kinds of evidence that differ.

Check and save the comparison

Open each numbered citation and read its passage in context. Suppose the answer says Branch B ran repeat briefings. If S2 only proposes them, change the row to planned and leave delivery unconfirmed. Citing the plan does not show who attended.

Check the quiet-shift note too. If a time-pressure theme omits it, restore it and note why it challenges that view. Read O1 before accepting any claim about skill gains or continued use.

For a missing or unhelpful citation, find the record by name and check it. Keep the passage location with your correction. Atlas can help read and compare; you decide what the records support and whether you need more evidence.

Select New, then Note in the project sidebar to keep the checked matrix, source locations, conflicts, and requests for missing records. Wait for Saved before closing the note.

Report the finding and next check

State which sources, sites, and periods the review covers. Name any framework version, explain how you grouped findings, and keep the passages that changed your view. The CFIR developer guide gives guidance for its own use.

For this pack, report staff concerns about referral use and briefing access, with the quiet-shift note leaving part of the explanation open. Repeat briefing delivery is unconfirmed, and the form change has no tested effect.

Name the next evidence needed and who will get it: briefing records, matched referral measures, nonusers' views, or later follow-up. The team can weigh these study tasks; they are not findings already in the pack.

Keep the checked comparison when new records arrive. Revisit the row and its claim rather than treat the first matrix as final. A new passage may fill a gap, show another context change, or require a narrower claim.