Designing Experiments in the Social Sciences

About

Workshop contents and objectives

An experiment is a method of data collection under controlled conditions. By manipulating the explanatory variable, controlling potentially confounding factors, and randomizing the assignment of participants to different treatments and control conditions, the method is well-suited to generate data for rigorously testing hypotheses derived from theories. Given the opportunity to create a wide variety of social contexts in the laboratory, it also enables researchers to explore counterfactual situations. Likewise, it allows us to systematically study the effects of interventions in everyday situations and study attitudes, behavioral intentions, and behavior that would be susceptible to social desirability bias in ordinary surveys.

The course offers a broad introduction to the design of experiments in sociology, political science and economics. Its objective is to enable participants to develop and implement their own experiments. While going through the full circle of developing a research question and a theory-based hypothesis, designing an experiment that captures the essential elements to test the hypothesis, implementing the experiment and analyzing the data, it discusses the considerations relevant in each stage of this process. For this test, the participants design a laboratory, a field and a survey experiment. In doing this, they are introduced to the particularities of each type and the differences between them. The course further elaborates on issues such as validity and the role of incentives as well as ethical questions. Software programming of experiments is not part of the course, but participants will be briefly introduced to some available options.

Workshop design

The course consists of a mixture of lectures, practical tasks and discussions. Typically, the mornings focus on lectures and discussions, whereas the afternoons are reserved for practical work in groups of two or three participants and the discussion of that work in plenum. On day 4, each group implements one of the experimental designs prepared in the previous days and participants serve as subjects for each other’s experiments, perhaps (and hopefully) complemented by other summer school participants who are recruited by advertisement. Or they may even unwittingly participate in a field experiment administered on campus. Groups analyze their data and present the results in the afternoon of the final day.

Detailed lecture plan (daily schedule)

Day 1 Morning
45 min Introduction
90 min Practical session (plenum): Example of laboratory experiment
Afternoon
45 min Theory in experiments
90 min Practical session (group): Theory development
90 min Practical session (group): Derivation of testable hypothesis

Day 2 Morning
90 min Causality and experimental design
90 min Laboratory experiments
Afternoon
90 min Field experiments
90 min Practical session (group): Design of own laboratory or field experiment (paper and pencil)
90 min Practical session (plenum): Group discussion of experimental designs

Day 3 Morning
90 min Validity
90 min Survey experiments: Principles
Afternoon
90 min Survey experiments: Design issues
90 min Practical session (group): Design of own experiment (survey)
90 min Practical session (plenum): Group discussion of experimental designs

Day 4 Morning
90 min Statistical analysis of experimental data
90 min Practical session (individual): Models for analyzing experimental designs
Afternoon
90 min Practical session (group): Implementation of one of the own experiments
90 min Practical session (group): Implementation of one of the own experiments
90 min Practical session (group): Data analysis of the implemented experiment

Day 5 Morning
60 min Incentives
60 min Ethics and deception
30 min Natural experiments
30 min Software
Afternoon
120 min Practical session (plenum): Presentation of results

Class materials

All materials will be provided online.

**The Summer School cannot grant credits. We only deliver a Certificate of Participation, i.e. we certify your attendance.**

If you consider using Summer School workshops to obtain credits (ECTS), you will have to investigate at your home institution (contact the person/institute responsible for your degree) to find out whether they recognise the Summer School, how many credits can be earned from a workshop/course with roughly 35 hours of teaching, no graded work, and no exams.

Make sure to investigate this matter before registering if this is important to you

Course leader

Prof Bernhard Kittel, Department of Economic Sociology, University of Vienna

Target group

graduate students, doctoral researchers, early career researchers

Basic understanding of statistics: tests for categorical, ordinal and interval data, regression.

Fee info