🔬 Research

Survey Cognitive Interview Pretest Protocol Writer: Think Aloud and Verbal Probes Mapped to Comprehension, Retrieval, Judgment, and Response, a Probe Matrix, Interviewer Script, Rounds, Problem Codes, and a Revision Log

Plan cognitive interviews to pretest a questionnaire before fielding: think aloud instructions, scripted probes for each item mapped to the four stage response model, room for spontaneous probes, a full interviewer script, sample size and iterative rounds, a problem coding scheme, and a findings to revision log.

0.0
0Reviews
P
October 7, 2026

Prompt

Act as a survey methodologist who designs and conducts cognitive interviews to pretest questionnaires for public health, education, and government surveys, and who uses the four stage response model (comprehension, retrieval, judgment, response) to find out why an item fails, not just that it fails.

Inputs:
- Draft questionnaire items to test, numbered, with response options and skip logic: [DraftItems]
- Target population and the subgroups most likely to struggle (language, literacy, age, region): [TargetPopulation]
- Survey mode and length: web, phone, paper, or interviewer administered: [SurveyMode]
- Known concerns about specific items from the team or past data: [ItemConcerns]
- Resources: interviewers, session length, incentive, timeline, recording consent rules: [Resources]
- Output format: [Format]

Generate:
1. An approach note: concurrent think aloud, retrospective probing, or a mix, chosen for SurveyMode and TargetPopulation, with one sentence on why.
2. A probe matrix: for each item in DraftItems, the response stage most at risk, one or two scripted probes per stage worth testing (for example a paraphrase probe for comprehension, a recall strategy probe for retrieval, a confidence probe for judgment, a fit of options probe for response), and a cue for spontaneous probes when the participant hesitates or changes an answer.
3. The interviewer script: welcome, consent and recording, a think aloud practice exercise unrelated to the survey, item administration in the survey's own mode, the probes, and a closing debrief.
4. Interviewer rules: read items exactly as written, use neutral probes, do not teach the meaning of a term, note nonverbal hesitation, and how to probe without leading.
5. Sample and rounds: a recruitment plan by subgroup, participants per round within ranges commonly used in the cognitive interviewing literature, and at least two iterative rounds with revisions in between. Say that this is qualitative evidence and not a statistical sample.
6. A problem coding scheme: codes by stage (comprehension, retrieval, judgment, response, plus instructions, navigation, and sensitivity), with a definition and an example from these items.
7. A summary template per item: what participants said, codes applied, how many participants showed each problem, and whether it was seen across subgroups.
8. A findings to revision log: item, problem, evidence quote, proposed revision, round tested, outcome.

Constraints:
- Do not invent findings; use placeholders until real interviews happen. Probes must never suggest a correct answer.
- Respect consent and privacy rules in Resources. No em dashes.

Instructions

Replace every [bracket] with your details before running. Works on ChatGPT, Claude, and Gemini.

Generated Output

This image was generated using the prompt above.

Survey Cognitive Interview Pretest Protocol Writer: Think Aloud and Verbal Probes Mapped to Comprehension, Retrieval, Judgment, and Response, a Probe Matrix, Interviewer Script, Rounds, Problem Codes, and a Revision Log - Result

Examples

Example Input

DraftItems: Q1 In the past 12 months, how many times did you visit a health care provider? (number). Q2 Do you have a usual source of care? (yes, no). Q3 How would you rate your access to mental health services? (very good to very poor)
TargetPopulation: adults in a rural county, including Spanish speakers and adults over 70
SurveyMode: self administered web survey on phones, about 12 minutes
ItemConcerns: past data show Q2 yes rates higher than expected; Q3 has many very good answers from people who never sought care
Resources: two bilingual interviewers, 60 minute sessions, $40 gift card, six weeks, audio recording allowed with written consent
Format: approach, probe matrix, script, rules, rounds, codes, summary template, revision log

Example Output

Approach: concurrent think aloud while participants answer on their own phone, followed by retrospective probes on each item, because the real survey is self administered on phones and older participants may find continuous think aloud tiring.

Probe matrix
| Item | Stage at risk | Scripted probes | Spontaneous cue |
| Q1 visits | Retrieval, comprehension | How did you come up with that number? Which visits did you count? Did you count dentists, telehealth, urgent care? | Participant pauses or counts on fingers |
| Q2 usual source | Comprehension | In your own words, what is this question asking? What does usual source of care mean to you? Would an urgent care clinic count? | Participant says it depends |
| Q3 access rating | Judgment, response | What were you thinking about when you chose that answer? Have you ever looked for mental health care? How easy was it to pick one option? | Fast very good with no prior care mentioned |

Interviewer script (excerpt)
1. Welcome. "We are testing the questions, not you. There are no wrong answers."
2. Consent and recording form; confirm language preference.
3. Practice: "Please think aloud while you tell me how many windows are in your home."
4. Participant answers Q1 to Q3 on their phone, thinking aloud.
5. Retrospective probes from the matrix, then spontaneous probes where cues occurred.
6. Debrief: "Was any question confusing or hard to answer honestly?" Thank and give the $40 card.

Interviewer rules: read items exactly; use neutral follow ups such as "Tell me more"; never explain usual source of care; note hesitations with a timestamp.

Sample and rounds
- Round 1: about 8 to 10 participants: 3 Spanish speakers, 3 adults over 70, others mixed. Round 2: a similar number after revisions. Sizes follow ranges commonly used in cognitive interviewing practice; findings are qualitative, not a statistical sample.

Problem codes
| Code | Definition | Example from these items |
| C1 term unclear | key term read differently | usual source read as any place ever visited |
| C2 scope | unclear what to include | telehealth left out of Q1 |
| R1 recall | cannot recall over reference period | 12 month count guessed |
| J1 no basis | answers without relevant experience | Q3 rated with no care sought |
| P1 options | answer does not fit options | no option for never needed |
| S1 sensitivity | discomfort answering | mental health stigma |

Summary template per item
Item / what participants said / codes / number showing each problem / subgroups affected / recommendation.
Placeholder until interviews run: Q2: [quotes], codes [C1], [n] of [N], [subgroups].

Revision log
| Item | Problem | Evidence | Proposed revision | Round | Outcome |
| Q2 | C1 suspected per ItemConcerns | to collect | add definition: a doctor's office or clinic you usually go to when sick, not the emergency room | test in round 2 | pending |
| Q3 | J1 suspected | to collect | add filter: In the past 12 months, did you try to get mental health care? then ask Q3 only of yes | test in round 2 | pending |

Reviews (0)

Please login to leave a review.
Loading reviews...