empirical-methods
Homepage for 17-803 "Empirical Methods" at Carnegie Mellon University
Project maintained by bvasiles Hosted on GitHub Pages — Theme by mattgraham
L3: The Role of Theory (pdf, video)
You gotta have a theory!
Theories are the building blocks of scientific knowledge. They explain how and why certain phenomena occur, and allow predictions to be made. In this lecture we start by discussing two papers that use theory in interestingly different ways.
We then talk about how different philosophical stances differ in their ideas about the role of theory. To the positivist (e.g., the Lampe, Ellison, and Steinfield paper), science is the process of verifying theories by testing hypotheses derived from them. To the constructivist (e.g., the Hausman paper, at least partly; the Bogart et al paper from last lecture), science is the process of seeking local theories that emerge from (and explain) the data.
Finally, we argue that thinking about theories is unavoidable even in very applied, tool-building-oriented disciplines like Software Engineering.
The bonus slides define “theory” and relate it to the concepts of “theoretical framework” and “conceptual framework” often encountered in research papers.
Exemplars
Hausman, A. (2000). A multi‐method investigation of consumer motivations in impulse buying behavior. Journal of Consumer Marketing.
Lampe, C. A., Ellison, N., & Steinfield, C. (2007, April). A familiar face (book) profile elements as signals in an online social network. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 435-444).
Methodological Readings
Easterbrook, S., Singer, J., Storey, M. A., & Damian, D. (2008). Selecting empirical methods for software engineering research. In Guide to Advanced Empirical Software Engineering (pp. 285-311). Springer, London.
This chapter gives a very readable, albeit somewhat brief, overview of how theory fits into empirical research in general, and how different philosophical stances differ in their ideas about the role of theory.
Varpio, L., Paradis, E., Uijtdehaage, S., & Young, M. (2020). The distinctions between theory, theoretical framework, and conceptual framework. Academic Medicine, 95(7), 989-994.
This paper goes into much more detail about how theory fits into positivist (objectivist deductive) and constructivist (subjective inductive) research. This is also where the discussion of differences between “theory,” “theoretical framework,” and “conceptual framework” comes from.
Further Readings
“You gotta have a theory!” Keynote talk by Steve Easterbrook. Doctoral Symposium at the 2006 ACM SIGSOFT Symposium on the Foundations of Software Engineering (FSE).
The talk addresses the question of how we should evaluate the claims we make for our research in software engineering. The central idea Easterbrook presents in the talk is that the only way to judge a “contribution to knowledge” is to relate the research to new or existing theories, and to collect evidence that helps to support or refute these theories. However, in software engineering, we very rarely articulate our theories. The talk explores the nature of theories, and surveys a range of empirical methods that can be used to investigate and/or develop theories in software engineering.
Stol, K.-J., & Fitzgerald, B. (2015). Theory-oriented software engineering. Science of Computer Programming, 101, 79-98.
This paper discusses the importance of theory and conceptualization, and argues for developing stronger theoretical foundations within Software Engineering research. It talks about theory as both a driver for, and a result of empirical research, similarly to the sources above. And it gives examples of different possible research pathways involving theory.
Trockman, A., Zhou, S., Kästner, C., & Vasilescu, B. (2018). Adding sparkle to social coding: an empirical study of repository badges in the npm ecosystem. In Proceedings of the 40th International Conference on Software Engineering (pp. 511-522).
This was the first paper from a series looking at the role of signals on the GitHub platform. It uses signaling theory as part of a theoretical framework developing and testing specific hypotheses about repository badges in the context of the GitHub platform. Think of this as an example of a positivist use of theory.