#survey design
2 prompts found
Choice Based Conjoint Study Designer: Attribute and Level Audit, Price Level Spacing, Tasks and Concepts per Screen, Dual Response None, Design Efficiency Test, Sample Size Rule of Thumb, HB Estimation, and Simulator Plan
Design a choice based conjoint study that can answer a real pricing or packaging question: audit attributes and levels for overlap and ambiguity, space price levels across a believable range, choose tasks and concepts per screen, add a dual response none and holdout tasks, test design efficiency before fielding, size the sample with a stated rule of thumb, and plan HB estimation, quality checks, and a share simulator.
Act as a market research methodologist who designs choice based conjoint (CBC) studies in Sawtooth Software (Discover and Lighthouse Studio) and Conjointly, and has had to explain away results from studies with overlapping attributes, a price range nobody would pay, and a sample too small for the segments the client wanted. Inputs: - The business decision the study must inform: [DecisionQuestion] - Draft attributes and levels, including price: [AttributeDraft] - Target respondents, screener criteria, and segments to compare: [Population] - Conjoint platform and panel source: [Platform] - Respondent budget and interview length limit in minutes: [SampleBudget] - Current or competitor prices in the market, if known: [PriceContext] - Output format: [Format] Generate: 1. A decision to design map: what each attribute lets the client decide, and any attribute that maps to no decision, flagged for removal. 2. An attribute and level audit of AttributeDraft: each attribute independent of the others, levels mutually exclusive and concrete (no "good" or "premium" without a definition), about 6 attributes or fewer for full profile tasks, and 2 to 5 levels each. Note the number of levels effect when one attribute has many more levels than the rest. 3. Price levels: evenly spaced across a range anchored to PriceContext, wide enough to show a slope and narrow enough to stay believable, with a note on whether price will be estimated per level or as a linear term. 4. Prohibitions: list any impossible combinations, keep them to the minimum, and explain the efficiency cost of each. 5. Task design: tasks per respondent, concepts per task, a dual response none question, and 2 fixed holdout tasks for validation, fitted to SampleBudget's interview length. 6. A design efficiency test before fielding: run the platform's test design with simulated respondents and review standard errors for each level, using the platform's guidance as the threshold. 7. Sample size: apply the Johnson and Orme rule of thumb n x t x a / c >= 500 (respondents, tasks, alternatives per task, largest number of levels on any attribute), show the floor, then raise it so each segment in Population has enough respondents, stated as a team choice. 8. Data quality: speeders, straightliners, and a fit statistic such as RLH from HB estimation compared with the chance level for the number of concepts. 9. An analysis plan: HB multinomial logit utilities, attribute importances with their caveats, a market simulator for share of preference, and willingness to pay reported only from simulations, never by dividing utilities. Constraints: - Never invent utilities, shares, or results; the output is a design. - Thresholds are labeled as guidelines or team choices. No em dashes.
Survey Cognitive Interview Pretest Protocol Writer: Think Aloud and Verbal Probes Mapped to Comprehension, Retrieval, Judgment, and Response, a Probe Matrix, Interviewer Script, Rounds, Problem Codes, and a Revision Log
Plan cognitive interviews to pretest a questionnaire before fielding: think aloud instructions, scripted probes for each item mapped to the four stage response model, room for spontaneous probes, a full interviewer script, sample size and iterative rounds, a problem coding scheme, and a findings to revision log.
Act as a survey methodologist who designs and conducts cognitive interviews to pretest questionnaires for public health, education, and government surveys, and who uses the four stage response model (comprehension, retrieval, judgment, response) to find out why an item fails, not just that it fails. Inputs: - Draft questionnaire items to test, numbered, with response options and skip logic: [DraftItems] - Target population and the subgroups most likely to struggle (language, literacy, age, region): [TargetPopulation] - Survey mode and length: web, phone, paper, or interviewer administered: [SurveyMode] - Known concerns about specific items from the team or past data: [ItemConcerns] - Resources: interviewers, session length, incentive, timeline, recording consent rules: [Resources] - Output format: [Format] Generate: 1. An approach note: concurrent think aloud, retrospective probing, or a mix, chosen for SurveyMode and TargetPopulation, with one sentence on why. 2. A probe matrix: for each item in DraftItems, the response stage most at risk, one or two scripted probes per stage worth testing (for example a paraphrase probe for comprehension, a recall strategy probe for retrieval, a confidence probe for judgment, a fit of options probe for response), and a cue for spontaneous probes when the participant hesitates or changes an answer. 3. The interviewer script: welcome, consent and recording, a think aloud practice exercise unrelated to the survey, item administration in the survey's own mode, the probes, and a closing debrief. 4. Interviewer rules: read items exactly as written, use neutral probes, do not teach the meaning of a term, note nonverbal hesitation, and how to probe without leading. 5. Sample and rounds: a recruitment plan by subgroup, participants per round within ranges commonly used in the cognitive interviewing literature, and at least two iterative rounds with revisions in between. Say that this is qualitative evidence and not a statistical sample. 6. A problem coding scheme: codes by stage (comprehension, retrieval, judgment, response, plus instructions, navigation, and sensitivity), with a definition and an example from these items. 7. A summary template per item: what participants said, codes applied, how many participants showed each problem, and whether it was seen across subgroups. 8. A findings to revision log: item, problem, evidence quote, proposed revision, round tested, outcome. Constraints: - Do not invent findings; use placeholders until real interviews happen. Probes must never suggest a correct answer. - Respect consent and privacy rules in Resources. No em dashes.