Pilot Study: Turn Findings into Plan Changes

A pilot study runs all or part of a planned study on a smaller scale to learn whether and how it can proceed. To review one, link each feasibility goal to what was observed, where the record supports it, and what should change next. A small study is not a useful pilot merely because it is small.

This guide builds a feasibility-observation-revision note from fictional interview-pilot records. It keeps a smooth rehearsal distinct from a ready-to-run main study and shows how missing evidence changes the next step. Use the structure with your own permitted plan, notes, and reports.

What a pilot study can answer

A pilot asks practical questions about a planned study: can the team carry out a procedure, and can people follow the task? Look for the point where the process breaks down. CASP's pilot study guide describes use in both qualitative and quantitative work. Its purpose is to learn about the process before a larger study, not simply to produce a smaller set of findings.

In the trial framework used by Eldridge and colleagues, feasibility asks whether a future trial can be done, should be done, and how. A pilot rehearses all or part of it on a smaller scale. State the purpose and design when using either term, since names alone can hide different kinds of work.

A rehearsal can reveal an unclear prompt, a slow handoff, or a step that needs more staff time. It may also leave the main question open. If you test an interview guide with a small group but do not test the recruitment route, your notes cannot show that the route will supply the main sample.

A feasibility pilot does not prove an intervention works. NCCIH's uses and misuses guidance warns against drawing efficacy conclusions from these studies. Keep the claim tied to the question the pilot was designed to answer, and resist turning encouraging early results into a verdict about the main study.

Start with the stated feasibility goals

Read the plan version in force when the pilot began. Extract its goals before reading the final conclusion. That order helps you see whether the report answers the planned questions or substitutes easier ones after the results are known.

1. Name the uncertainty. State what the team needed to learn, such as whether the session fits the time slot.
2. Find the planned check. Record what would count as relevant evidence and any criteria set in advance.
3. Find the observation. Locate the note, log, or result that addresses the goal, with its source location.
4. Bound the meaning. Say what the observation supports, what it leaves open, and which setting it covers.
5. Propose the next step. Keep a plan change, its reason, owner, and recheck together.

The CONSORT pilot extension links specific trial goals to methods and results. It also asks authors to document changes and their reasons.

These are useful documentation principles; the extension's formal scope is randomized pilot trials before a future definitive trial, not every interview rehearsal.

When you have too many possible questions, a learning agenda can help choose which unknowns matter for the next decision.

Keep that choice separate from the evidence check. A goal can be important even when the pilot has not yet produced the evidence needed to answer it.

Record absent criteria as absent. If the team chose a rule after seeing the notes, label it as a proposed rule for the next round. Do not rewrite the old plan to make the result look as though it passed a test set in advance.

Build a fictional pilot revision note

Imagine a team preparing interviews about commuter students' travel routines. The example below is fictional: no interviews were conducted, and every observation, source label, and proposed change is invented for teaching. The pilot aims to check timing, prompt clarity, recruitment records, and the recording process.

The fictional plan, P1, has a 40-minute session slot. It calls for a timing log, notes on unclear prompts, a recruitment log, and a recording check. The table links those goals to four distinct records. None of the rows shows how common a view is among commuter students.

Table: Fictional feasibility-observation-revision note. Labels, observations, and changes are invented; this is not a real pilot report or evidence of a successful study.

The timing row supports a proposed change to the session, not a claim that the team should drop the last topic. That topic may be central to the main question. Keep its purpose visible while reviewing ways to free time or change the approved format.

The recording row has a similar limit. A file opening does not show that the whole recording is usable. You need the relevant check before claiming the process worked. The CASP discussion of pilot limitations also cautions that pilot work cannot guarantee later success.

For real records, preserve the exact source wording apart from your proposed change. A useful row lets a reviewer disagree with your interpretation without losing the observation. That is why “what happened” and “what we should do” occupy separate fields.

Correct an unsupported readiness claim

Suppose the draft conclusion says: “The interviews ran smoothly, so the main study is ready.” The table does not support that broad claim. Timing needs revision, the reference day is unclear, recruitment evidence is missing, and the recording check is incomplete.

Use the stated goals as review criteria. Ask whether each has an observation, whether that observation answers the goal, and whether any required check remains open. This is a review of the supplied evidence, not a new score invented to make the pilot pass.

A better note is: “The supplied records suggest revisions to the opening and prompt wording. Recruitment readiness is not established, and recording completeness needs checking. Review those points before deciding whether the revised plan can proceed.” This conclusion names a next step without hiding the gaps.

The extended feasibility framework treats uncertainty as a reason to choose further work and describes paths that need not be linear. You may need to revisit one part of the design after another check reveals a new problem.

A revision note should leave room for that return.

Do not make the correction stronger than the records allow. “No log in the supplied set” does not mean no recruitment took place. It means this evidence set cannot resolve that goal. Locate the missing record, narrow the claim, or plan an appropriate check with the study owner.

Keep progression and efficacy apart

Progression asks whether and how to move to the next stage. Efficacy asks whether an intervention produces the intended effect. A feasibility note may guide the former without answering the latter.

NCCIH's guidance also warns that unstable pilot effect sizes should not drive power calculations for a main efficacy study.

There is no universal pilot sample size in this guide. Choose the size for the feasibility question and justify it with suitable methods support. Testing one technical step and estimating a recruitment rate are different tasks; they need not call for the same people, measures, or amount of data.

Internal and external pilots also differ. In the 2023 framework, internal pilot data contribute to the main trial dataset, while external pilot data do not enter that trial's final analysis.

Follow the approved design and analysis plan rather than merging early records by default.

Before preparing a formal report, check the right guideline for the design. EQUATOR's current pilot-extension entry identifies its randomized-trial scope and lists generic CONSORT 2025 alongside the extension. A completed checklist cannot certify that the pilot or main study will succeed.

Changes that affect consent, burden, access, or the approved procedure need the relevant research review. A proposed row in a note is not approval to act. Keep that status visible so a useful suggestion cannot be mistaken for a released plan.

Compare pilot sources in Atlas

Add the plan and permitted pilot records to one project. Wait until they have finished processing, then start a chat and type @ to select each source.

Use clear versions so the old plan and the revised one cannot silently swap roles in the comparison.

Choose Project only beside + if you want to stop new outside retrieval. Earlier chat context remains available, including outside evidence already present. A fresh chat can help keep this review focused when earlier material could blur the source set.

Ask for a note that preserves the source-to-change chain:

Compare the stated feasibility goals in this plan with these pilot notes. For each goal, cite the observation, explain what it supports and leaves open, and propose a revision only where supported. Keep proposed changes distinct from approvals. Mark missing records as not found.

Open each citation and check the surrounding text. The citation-tracking guide helps explain why source links matter.

A link gives you a way to inspect the claim; it does not mean that a proposed revision is sound or that every relevant record was found.

Atlas source-check capture, reused unchanged. Paper: Faysse and colleagues, ColPali, version 6 (2025), CC0 1.0. It does not show the fictional student pilot or a feasibility evaluation.

If the answer says recruitment passed, ask for the exact log passage. When the supplied set lacks it, correct the row and retain the gap. Use the research synthesis guide for broader source comparisons after the pilot-specific rows have been checked.

For scans or complex layouts, review the reading needs described in the AI PDF reader guide. Inspect the relevant page yourself.

Atlas can assist with reading and notes; it does not conduct the pilot, run statistics, or authorize changes to a research plan.

Review changes before the next study

Review each proposed change with its source and reason visible. Check that the new wording still serves the main question and that the named owner can carry out the recheck. A change that fixes timing may create a new gap in coverage, so read the revised plan as a whole.

Create a note through New then Note, and wait for Saved before closing. Keep the source versions, goal, observation, proposed change, review status, and remaining questions together. Preserve the old decision rather than replacing it with a claim that the process always worked.

Before releasing a revised plan, resolve the checks and approvals it requires. Share open questions as open questions. The useful result of a pilot review can be a narrower next step: revise one procedure, gather a missing record, or pause the main study until the evidence supports progression.