Questionnaire design with a checked item-alignment note

Questionnaire design connects what you want to learn with questions that people can understand and answer. Start by defining each idea, then check whether the wording, answer choices, time frame, and route through the form fit that purpose.

A question can read well while asking about the wrong thing. For example, being pleased with a library is different from finding a seat easily. Keep an item-alignment note that links each draft question to its purpose and the source requirements behind your choices.

The fictional library draft below shows wording corrections and open checks. Use Atlas to compare source support, then test the questions and judge what their answers can support.

What questionnaire design needs to connect

A questionnaire contains questions, answer choices, and instructions. A survey also needs a wider plan for whom to ask, how to gather replies, and how to analyze them. Statistics Canada starts with objectives and data needs before the form is designed and tested.

An item is a question or prompt. A construct is the idea you want to study, such as perceived ease of finding a seat. Define what that idea includes so you can judge the fit of each item.

The NCVO questionnaire guide connects questions to the indicators a study needs. That connection is a reason to review an item. It does not prove that the draft measures the idea well.

Keep the full question and its answer options together. A question about days needs a way to report days, while a question about ease needs choices that express ease and difficulty. The source-alignment note should explain why you chose each form.

Define constructs and wording criteria

In this fictional example, a campus team wants to learn about students' recent attempts to find a study seat at R Main Library. Its main construct is perceived ease of finding a seat during the most recent attempt. A separate visit-count item helps describe recent library use.

The team defines a study seat as a place where the student tried to sit and study. Test what students count, including group-work seats, computer desks, and silent-area desks.

Use research objectives to state the need before writing questions. The Government Analysis Function guidance starts with data needs and checks how research aims will be addressed. Its page is marked under review, so check source versions and compare advice with other methods sources.

The conceptual framework guide helps define broad ideas and proposed links. Here, ease, comfort, satisfaction, and expected future success need distinct meanings.

Set evidence criteria for the item review:

• Purpose: Which defined idea does this question address, and why is it needed?
• People: Who can answer from their own experience?
• Time and place: Which visit, dates, and library does it concern?
• Answer choices: Can people express their experience without guessing or choosing a misleading option?
• Source and tests: Which methods requirement guides the wording, and what must still be tested?

Review each item against these criteria before it reaches respondents.

A worked construct-item-source alignment table

The fictional items and decisions below apply source guidance to a teaching example. They remain untested and provide no evidence that this library scale is valid.

Table 1: E1 combines two ideas that a student could rate differently. E2 assumes a benefit that the study has not shown. V1 leaves the meaning of regularly to each person, making replies hard to compare.

The table leaves out questions that do not serve the current aim. Asking about satisfaction or future confidence would need a reason and a plan for those answers.

The Cross-Cultural Survey Guidelines chapter sets out question types and calls for records of item origins and changes. Wording used with one group or mode may need review in another.

Correct wording and answer choices

Review each wording change with its options and route:

• Split combined ideas. Replace E1 with a question about how easy or difficult it was to find a seat on the most recent attempt. A comfort question would need its own purpose and a route for people who used a seat.
• Make the period clear. Replace V1 with: "Between 1 and 7 September 2026, on how many days did you visit R Main Library?" Allow a count from zero to seven. These dates belong to the teaching example and must match the real study period.
• Remove the assumed benefit. Replace E2 with: "On your most recent visit to R Main Library to find a study seat, how easy or difficult was it to find one?" Review failed attempts separately instead of assuming everyone found a seat.

The Pew Research Center wording guidance explains why one-concept questions, plain words, and answer choices matter. Its own process includes pretesting. You still need to learn how students understand your stems.

For the ease item, a draft set might range from very easy through neither easy nor difficult to very difficult, with every point labeled. A failed-search answer needs review: it may be meaningful to route those students to a separate question about what happened.

Do not treat "I did not find a seat" as automatically equal to the most difficult scale point. Likewise, zero visit days differs from being unsure of the count. Test how people choose among those cases before deciding the final options or codes.

Keep routing and context with each item

Routing controls which questions a person sees. A student who never tried to find a seat cannot report that experience, even if they visited to return a book.

Check whether the person tried to find a seat before asking about the search. Keep failed attempts visible in the draft's routes.

Original annotated example from the Department for Transport's NTS 2024 questionnaire documentation, Appendix A1, page 2. Crown copyright, reused under the Open Government Licence v3.0. The labels keep block rules, routing, item name, and text distinct. This travel-survey example illustrates documentation, with no claim about the library draft's quality.

In the source example, rules name the child age group and who answers. Labels point to routing, the question name, and opening text. The owner's text version explains block rules, staff instructions in capitals, and text fills.

Keep who sees each item, what it asks, and what follows beside your draft. Review the actual online or paper version to check how its instructions and routes work. For open-ended items, plan qualitative survey analysis that retains the exact prompt and full answer, including mixed responses and blanks.

Test the full draft with respondents

Source review can flag a likely wording problem. Test the draft with its target group to learn how people answer. Keep planned checks and test results in distinct parts of the note.

Different checks serve different purposes:

• Cognitive pretesting: Explore how people understand the words, recall a visit, make a judgment, and choose an answer.
• Usability testing: Check how people read the form, follow instructions, use controls, and move through its routes.
• Piloting: Try the wider process, including how to reach people, record data, time the work, and use the planned analysis.
• Measurement review: Judge the support needed for this use of the item or scale, with help from methods experts.

Test with non-users, people who found a seat, and people who tried but failed. Check what they count as a visit or seat and whether the choices fit. These cases remain planned.

Statistics Canada's questionnaire testing section covers wording, order, instructions, choices, and the full form. Use the literature review process to locate past instruments and their methods evidence when reuse fits.

If you adapt an item, keep its origin and check its allowed use, group, mode, and past tests. Wording or context changes may alter what replies mean. A clearer draft or small pilot alone does not prove that it measures the idea well or does so consistently.

Review item support in Atlas

Bring the defined ideas, draft, and sources to one project for text review. Atlas can help compare them. The researcher decides on the items and runs the tests.

1. Add sources you can use. Include the defined constructs, full draft, choices, routes, and methods guidance.
2. Name the comparison set. In Ask a question, use @ to select the items and sources. Project only bounds new retrieval to the project. Earlier chat context remains available.
3. Ask for alignment. Request: "Compare each item with its defined idea. Cite the wording rules, flag mixed ideas or assumed gains, and mark missing source support. Keep planned tests separate from results."
4. Inspect the support. Select Send, then open the important citations. Read the surrounding text. A PDF citation can open a precise place when a location is available.
5. Correct and save. Revise items that do not fit and remove unsupported claims. Choose New, then Note, save the reviewed table and open questions, and wait for Saved.

If the answer calls E2 validated because it follows a wording rule, correct that claim. The rule supports removing an assumed gain. It gives no test results for this new item.

Keep the changed stem and its test-needed status together. A reviewer can check the design reason and see what remains to be tested.

Save the note and testing decisions

For each version, keep the item ID, construct, stem, options, route, source, reason for change, and pending test. Record test findings when they exist, along with the action taken.

Return to the note when a reviewer questions the seat definition or a test reveals a missing answer case. Organizing research notes helps keep sources and decisions together. Choose the final form from the checked design work and test results for its planned use.