empirical-methods
Homepage for 17-803 "Empirical Methods" at Carnegie Mellon University
Project maintained by bvasiles Hosted on GitHub Pages — Theme by mattgraham
L3: The Role of Theory (pdf, video)
You gotta have a theory!
Theories are the building blocks of scientific knowledge. They explain how and why certain phenomena occur, and allow predictions to be made. In this lecture we start by defining “theory” and relating it to the concepts of “theoretical framework” and “conceptual framework” often encountered in research papers. We then talk about how different philosophical stances differ in their ideas about the role of theory. To the positivist, science is the process of verifying theories by testing hypotheses derived from them. To the constructivist, science is the process of seeking local theories that emerge from (and explain) the data. The Bogart et al paper from last lecture is a good example of a constructivist use of theory. In class we went over an example of a positivist use of theory – the application of signaling theory to studying repository badges on the GitHub platform.
Lecture Readings
Easterbrook, S., Singer, J., Storey, M. A., & Damian, D. (2008). Selecting empirical methods for software engineering research. In Guide to advanced empirical software engineering (pp. 285-311). Springer, London.
This chapter gives a very readable, albeit somewhat brief, overview of how theory fits into empirical research in general, and how different philosophical stances differ in their ideas about the role of theory.
Varpio, L., Paradis, E., Uijtdehaage, S., & Young, M. (2020). The distinctions between theory, theoretical framework, and conceptual framework. Academic Medicine, 95(7), 989-994.
This paper goes into much more detail about how theory fits into positivist (objectivist deductive) and constructivist (subjective inductive) research. This is also where the discussion of differences between “theory,” “theoretical framework,” and “conceptual framework” comes from.
Trockman, A., Zhou, S., Kästner, C., & Vasilescu, B. (2018). Adding sparkle to social coding: an empirical study of repository badges in the npm ecosystem. In Proceedings of the 40th International Conference on Software Engineering (pp. 511-522).
This was the first paper from a series looking at the role of signals on the GitHub platform. It uses signaling theory as part of a theoretical framework developing and testing specific hypotheses about repository badges in the context of the GitHub platform.
Further Readings
“You gotta have a theory!” Keynote talk by Steve Easterbrook. Doctoral Symposium at the 2006 ACM SIGSOFT Symposium on the Foundations of Software Engineering (FSE).
The talk addresses the question of how we should evaluate the claims we make for our research in software engineering. The central idea Easterbrook presents in the talk is that the only way to judge a “contribution to knowledge” is to relate the research to new or existing theories, and to collect evidence that helps to support or refute these theories. However, in software engineering, we very rarely articulate our theories. The talk explores the nature of theories, and surveys a range of empirical methods that can be used to investigate and/or develop theories in software engineering.
Stol, K.-J., & Fitzgerald, B. (2015). Theory-oriented software engineering. Science of Computer Programming, 101, 79-98.
This paper discusses the importance of theory and conceptualization, and argues for developing stronger theoretical foundations within Software Engineering research. It talks about theory as both a driver for, and a result of empirical research, similarly to the sources above. And it gives examples of different possible research pathways involving theory.