empirical-methods
Homepage for 17-803 "Empirical Methods" at Carnegie Mellon University
Project maintained by bvasiles Hosted on GitHub Pages — Theme by mattgraham
L7: Qualitative Analysis (pdf, video)
You’ve designed the perfect interview guide (or open-ended survey instrument), carried out, recorded, and transcribed the interviews (or collected the survey responses), and are now sitting in front of a big pile of text. How do you go from here to publishable science?
In this lecture we discuss how to rigorously conduct qualitative content analysis, including coding and finding higher-level patters in your codes, with the goal of identifying themes or building theory from the ground up.
We also cover best practices for establishing trustworthiness in qualitative research results, focusing on credibility, transferability, and dependability.
Readings
Miles, M. B., Huberman, A. M., & Saldaña, J. (2018). Qualitative data analysis: A methods sourcebook. Sage publications. Chapter 4 “Fundamentals of Qualitative Data Analysis” (PDF) & Chapter 11 “Drawing and Verifying Conclusions”
Main source for the lecture. Chapter 4 focuses on coding data segments for category, theme, and pattern development. Analytic memos and the formulation of assertions and propositions are also discussed. Within-case and cross-case analysis are compared.
Chapter 11 covers analytic tactics for generating meaning from data and for testing or confirming findings. Standards for assessing the quality of conclusions are proposed, along with methods for documenting a researcher’s analytic processes.
Nowell, L. S., Norris, J. M., White, D. E., & Moules, N. J. (2017). Thematic analysis: Striving to meet the trustworthiness criteria. International Journal of Qualitative Methods, 16(1), 1609406917733847.
To be accepted as trustworthy, qualitative researchers must demonstrate that data analysis has been conducted in a precise, consistent, and exhaustive manner through recording, systematizing, and disclosing the methods of analysis with enough detail to enable the reader to determine whether the process is credible. The process of conducting a thematic analysis is illustrated through the presentation of an auditable decision trail, guiding interpreting and representing textual data.
Hsieh, H. F., & Shannon, S. E. (2005). Three approaches to qualitative content analysis. Qualitative Health Research, 15(9), 1277-1288.
Rather than being a single method, current applications of qualitative content analysis show three distinct approaches: conventional, directed, or summative. All three approaches are used to interpret meaning from the content of text data. The major differences among the approaches are coding schemes, origins of codes, and threats to trustworthiness. In conventional content analysis, coding categories are derived directly from the text data. With a directed approach, analysis starts with a theory or relevant research findings as guidance for initial codes. A summative content analysis involves counting and comparisons, usually of keywords or content, followed by the interpretation of the underlying context.
Onwuegbuzie, A. J., & Leech, N. L. (2007). Validity and qualitative research: An oxymoron?. Quality & Quantity, 41(2), 233-249.
The paper makes the case that in every qualitative inquiry, findings, interpretations, and conclusions should be assessed for truth value, applicability, consistency, neutrality, dependability, credibility, confirmability, transferability, generalizability, or the like. Further, legitimacy should not only be undertaken, but documented and delineated in the final research report, so that qualitative research can be made public, instead of the private status that it tends to have.
DuBois, J. M., Strait, M., & Walsh, H. (2018). Is it time to share qualitative research data?. Qualitative Psychology, 5(3), 380.
Roller, M. R., & Lavrakas, P. J. (2018). A total quality framework approach to sharing qualitative research data: Comment on Dubois et al. (2018). Qualitative Psychology, 5(3), 394–401.
The first paper discusses a series of concerns with sharing qualitative research data such as the importance of relationships in interpreting data, the risk of re-identifying participants, issues surrounding consent and data ownership, and the burden of data documentation and depositing on researchers.
The second paper provides researchers with an efficient way to think about and organize the types of information to share about their qualitative studies.
Paradis, E., & Sutkin, G. (2017). Beyond a good story: from Hawthorne Effect to reactivity in health professions education research. Medical Education, 51(1), 31-39.
The paper reviews the literature on the Hawthorne Effect (defined as a research participant’s altered behaviour in response to being observed) – a common source of bias in observational research. The authors conclude that “evidence of a Hawthorne Effect is scant, and amounts to little more than a good story.”