23 August 2019
This course provides researchers with the necessary methods and techniques to evaluate, test, and modify surveys to reduce measurement error. This is achieved through both lectures and in-class exercises. In the lectures, we introduce different pretesting approaches and discuss examples of untested as well as pretested and improved survey questions. The in-class exercises familiarize participants with different pretesting methods. First, we will discuss the overall goal of pretesting surveys and how to fit pretesting into the survey development process. Then, we will discuss the following pretesting methods in detail:
- expert review & questionnaire appraisal systems
- focus groups
- cognitive interviewing
- web probing
- usability testing & eye tracking
- other pretesting methods (e.g., behavior coding)
For each method, participants will receive practical advice on how to conduct pretesting projects and how to decide which pretesting methods should be selected in a given research situation. Participants will learn how the information gained from these techniques can be used to improve data quality and reduce respondent burden. We will also discuss the pros and cons of different pretesting methods and mixed-method approaches. Furthermore, the course provides an introduction to conducting cross-cultural pretesting projects.
Dr. Katharina Meitinger is an assistant professor in methods and statistics at Utrecht University.
Emily Geisen is the Cognitive/Usability Laboratory manager at RTI International in Research Triangle Park, North Carolina.
Participants will find the course useful if they:
- develop their own questionnaires for own data collection;
- work in a survey organization and work on questionnaire design and evaluation;
- use survey data and wish to understand the importance of pretesting to reduce measurement error.
- basic knowledge in questionnaire design; however, some practical experience in conducting surveys will be beneficial;
- there are no statistical prerequisites.
Participants are asked to bring their own laptops for the part of usability testing. No particular software access is needed.
By the end of the course participants will:
- be familiar with current pretesting methods;
- learn how to apply pretesting methods to their work;
- know the pros and cons of the different approaches to test survey questions;
- be able to make an informed decision about when to use which pretesting method and the ways in which several methods can be combined within a pretesting project.
- Certificate of attendance issued upon completion.
- 4 ECTS points via the University of Mannheim for regular attendance and satisfactory work on daily assignments and for submitting a paper/report of about 5000 words to the lecturer(s) up to 4 weeks after the end of the summer school (EUR 50).
EUR 300: Student/PhD student rate.
EUR 450: Academic/non-profit rate.
The rates include the tuition fee, course materials, the academic program, access to library and IT facilities, coffee/tea, and a number of social activities.
10 DAAD scholarships are available via the Institute of Sociology and Social Psychology (ISS) of the University of Cologne.
5 scholarships for participation in one main course are available from the European Survey Research Association (ESRA).