Factorial Survey Design

The factorial survey design (“vignette analysis”) is a method that integrates multi-factorial experimental set-ups into surveys. Respondents are asked to evaluate hypothetical situations, objects, or persons. By systematically varying attributes of the descriptions, it is possible to determine their influence on respondents’ stated attitudes, decisions, or choices. The experimental variation of the stimuli makes it possible to estimate the influence of each attribute on the evaluation. The factorial survey method is therefore an appropriate instrument for testing theoretical predictions. Moreover, because the experiment is embedded in a survey questionnaire, it is possible to reach heterogeneous sample populations.

This course provides a theoretical and practical overview of factorial survey methods. You will gain practical insights into all the single steps necessary to design a factorial survey experiment: (1) construction of vignettes, (2) selection of an experimental design, (3) drafting and programming of questionnaires (for online surveys as well as paper and pencil surveys), (4) data management, and (5) data analysis techniques (e.g., multilevel analyses, willingness to pay estimates). The course is structured as follows: Instructors will provide an overview on factorial survey experiments and explain practical tasks. You work on the tasks in individual “hands on” exercises. For the practical exercises, you may choose a research question related to your own research (e.g., PhD project).

For practical analyses, the statistical software package Stata will be used (prior knowledge required!). For setting up experimental designs and programming of questionnaires we additionally use the software packages SAS and QuestBack (no prior knowledge required). The method is NOT connected to (confirmatory or explorative) factor analysis. Moreover, the course does not cover anchoring and video vignettes.

Course leader

Katrin Auspurg holds a full professorship in Sociology at the LMU Munich, Germany. Carsten Sauer holds a full professorship in Sociology at Bielefeld University, Germany. Alisia Bauer is a research assistant at the LMU Munich, Germany.

Target group

You will find the course useful if:
- you want to learn about survey-experimental designs to study attitudes, preferences, or behavioral intentions,
- you have initial ideas for your own research questions that could be realized using a factorial survey,
- you plan to conduct a factorial survey in your own (PhD) projects,
- you want to deepen your knowledge of experimental designs and quantitative statistical methods,
- you want to learn how to analyze data from experimental designs and factorial surveys and evaluate the quality of such data.

Prerequisites:
- You should be familiar with the statistical software package Stata before the course starts (i.e., command structure, do-files, use of ados).
- You should have basic knowledge of questionnaire design and experimental methods.
- Methodical knowledge of data management and quantitative data analyses (e.g., linear regression techniques, coding of variables, merging of data sets) is required.

Course aim

By the end of the course, you will:
- have learned and discussed the features, typical applications, advantages, and shortcomings of factorial survey methods,
- have acquired practical insights into all single steps that are needed to set up factorial survey designs, to implement them into (computer assisted) questionnaires, to analyze resulting data, and report on results,
- be familiar with practical methods to evaluate data quality gained by factorial survey methods,
- have gained some insights into related experimental survey methods such as conjoint analyses and choice experiments,
- be able to apply factorial survey methods on your own.