How to Use the Cognitive Interview Protocol Prompt to Pretest a Survey Before Fielding

A survey question can look perfectly clear to the team that wrote it and still mean something different to the people answering it. One respondent counts telehealth visits as doctor visits and another does not. Someone rates their access to mental health care as very good without ever having looked for it. Cognitive interviewing is the standard way to catch these problems before a survey goes into the field. The Survey Cognitive Interview Pretest Protocol Writer: Think Aloud and Verbal Probes Mapped to Comprehension, Retrieval, Judgment, and Response, a Probe Matrix, Interviewer Script, Rounds, Problem Codes, and a Revision Log prompt writes the whole protocol: think aloud instructions, probes mapped to the stages of answering a question, an interviewer script, rounds and recruitment, a problem coding scheme, and a log that connects findings to revisions.

What the prompt produces

1. An approach note choosing concurrent think aloud, retrospective probing, or a mix based on your survey mode and population.
2. A probe matrix that lists, for each item, the stage most at risk (comprehension, retrieval, judgment, or response), scripted probes for that stage, and cues for spontaneous probes when a participant hesitates.
3. An interviewer script with welcome, consent, a practice think aloud exercise, item administration, probes, and debrief.
4. Interviewer rules for reading items exactly, probing neutrally, and never teaching what a term means.
5. A sample and rounds plan with recruitment by subgroup and at least two rounds with revisions between them, framed as qualitative evidence.
6. A problem coding scheme with codes by stage plus instructions, navigation, and sensitivity, each defined with an example from your items.
7. A per item summary template for what participants said and how often each problem appeared.
8. A findings to revision log that tracks each problem from evidence to revised wording to retest.

How to fill the inputs

DraftItems pastes the numbered questions with response options and any skip logic.

TargetPopulation describes who will take the survey and which subgroups may struggle, such as speakers of another language, people with lower literacy, or older adults.

SurveyMode says whether the survey is web, phone, paper, or interviewer administered, and how long it is. Interviews should mirror the real mode as closely as possible.

ItemConcerns lists what the team already suspects, often from past data that looked odd.

Resources covers interviewers, session length, incentive, timeline, and consent rules for recording.

Reading the example output

The example tests three health survey items with adults in a rural county:

• The approach mixes methods: participants think aloud while answering on their own phones, then answer retrospective probes, which is easier on older participants.
• Each item gets probes for its riskiest stage: a recall strategy probe for the visit count, a paraphrase probe for "usual source of care," and a basis probe for the access rating.
• Spontaneous probe cues are concrete, such as a participant counting on their fingers or answering very fast.
• The practice exercise is unrelated to the survey, counting the windows in your home, so participants learn to think aloud without priming.
• Problem codes come with examples from these items, such as answering a rating question with no relevant experience.
• The revision log starts with the team's suspicions as pending entries, with proposed wording to test in round two. No findings are invented before interviews happen.

Tips for better results

• Pretest the items in the language and format respondents will really see, including on phones if that is the mode.
• Recruit people who resemble the subgroups most likely to struggle, not just convenient colleagues.
• Record sessions when consent allows, and write notes the same day.
• Revise between rounds. Testing the same wording twice teaches you less than testing a fix.

Mistakes to avoid

• Do not explain a term when a participant asks what it means. Their confusion is the finding.
• Do not use leading probes like "That was clear, right?"
• Do not report counts as percentages of a population. Cognitive interviews are qualitative.
• Do not skip the debrief. Participants often mention the most important problem at the end.

Who it is for

Survey researchers, public health and education evaluators, graduate students designing questionnaires, and government or nonprofit teams preparing a survey for fielding.

Related PromptDig links

Open the Survey Cognitive Interview Pretest Protocol Writer: Think Aloud and Verbal Probes Mapped to Comprehension, Retrieval, Judgment, and Response, a Probe Matrix, Interviewer Script, Rounds, Problem Codes, and a Revision Log prompt and paste your draft items. For more research prompts, Browse more prompts. If you have a methods prompt that helps your team, Share a prompt.