empirical-methods
Homepage for 17-803 "Empirical Methods" at Carnegie Mellon University
Project maintained by bvasiles Hosted on GitHub Pages — Theme by mattgraham
(thanks for the promo, @JoshQuicksall!)
This is the Spring 2026 offering of this course. For older versions, see here: Spring 2024 • Fall 2022 • Spring 2021 • Fall 2018.
Overview
Empirical methods play a key role in the design and evaluation of tools and technologies, and in testing the social and technical theories they embody. No matter what your research area is, chances are you will be conducting some empirical studies as part of your work. Are you looking to evaluate a new algorithm? New tool? Analyze (big) data? Understand what challenges practitioners face in some domain?
This course is a survey of empirical methods designed primarily for computer science PhD students, that teaches you how to go about each of these activities in a principled and rigorous way. You will learn about and get hands-on experience with a core of qualitative and quantitative empirical research methods, including interviews, qualitative coding, survey design, and many of the most useful statistical analyses of (large-scale) data, such as various forms of regression, time series analysis, and causal inference. And you will learn how to design valid studies applying and combining these methods.
There will be extensive reading with occasional student presentations about the reading in class, homework assignments, and a semester-long research project for which students must prepare in-class kickoff and final presentations as well as a final report.
After completing this course, you will:
- become a more sophisticated consumer of empirical research, both in your field and outside
- develop the methodological skills that can help you design and carry out empirical components in your own research program
- be able to analyze empirical data, draw conclusions, and present results
- be able to read, summarize, present, but most importantly critique academic empirical research papers on a deep technical level
As a side effect, this course helps you develop a healthy dose of skepticism towards scientific results in general. Does the study design really allow the authors to make certain claims? Does the analysis technique? Is the evidence provided as strong as it could be? Are there fundamental flaws and threats to validity?
Coordinates
- Lectures: Tuesdays & Thursdays 3:30-4:50pm, Eastern Time in GHC 5222
- Assignments, private announcements: Canvas
- Slides: this website
- Reading materials: Shared Google Drive folder accessible with CMU login
- Instructor: Bogdan Vasilescu
- Teaching assistant: Claudia Mamede
Course Syllabus and Policies
The syllabus covers course overview and objectives, evaluation, time management, late work policy, and collaboration policy.
Learning Goals
The learning goals describe what I want students to know or be able to do by the end of the semester. I evaluate whether learning goals have been achieved through assignments, written project reports, and in-class presentations.
Schedule
Below is a preliminary schedule for Spring 2026. Each link points to a dedicated page with materials and more details. All videos are published on this YouTube channel.
Note: The schedule is subject to change and will be updated as the semester progresses.
| Date | Topic | Notes |
|---|---|---|
| Tue, Jan 20 | Introduction | slides • video |
| Thu, Jan 22 | Formulating research questions | slides • video |
| Tue, Jan 27 | Literature review | slides • video |
| Thu, Jan 29 | Role of Theory & Paper discussion | slides • slides P1, P2 🔜 • video |
| Tue, Feb 3 | Interviewing | slides • video |
| Thu, Feb 5 | Paper discussion | slides P3, P4, P5 • video |
| Tue, Feb 10 | Qualitative data analysis | slides • video |
| Thu, Feb 12 | In-class activity: qualitative coding & thematic analysis | slides • video |
| Tue, Feb 17 | Paper discussion & Mixed Methods & Survey Design | slides P6 • slides MM, slides SD • video |
| Thu, Feb 19 | Paper discussion & Survey Design | slides • slides P7 • video |
| Tue, Feb 24 | Paper discussion | slides P8, P9, P10 • video |
| Thu, Feb 26 | Survey Design | slides • video |
| Tue, Mar 10 | Designing Experiments I | slides • video |
| Thu, Mar 12 | Paper discussion & In-class activity: Designing Experiments II | P11 • slides • video |
| Tue, Mar 17 | Paper discussion | P12 • video |
| Thu, Mar 19 | Paper discussion & Designing Experiments III | P13 • slides • video |
| Tue, Mar 24 | Regression Modeling I | slides • video |
| Thu, Mar 26 | Regression Modeling II, | slides • 🔜 |
| Tue, Mar 31 | Regression Modeling III & Mixed Effects | materials • video |
| Thu, Apr 2 | In-class activity | materials w/ sol • video |
| Tue, Apr 7 | Time series | materials |
| Tue, Apr 14 | Guest lecture (Bogdan @ICSE) | |
| Thu, Apr 16 | Guest lecture (Bogdan @ICSE) | |
| Tue, Apr 21 | Final presentations (I) | |
| Thu, Apr 23 | Final presentations (II) |