empirical-methods

Homepage for 17-803 "Empirical Methods" at Carnegie Mellon University

Project maintained by bvasiles Hosted on GitHub Pages — Theme by mattgraham

L9: Qualitative Analysis with Large Language Models (pdf, video)

Lecture9-LLMs

To what extent can we use large language models and chatbots for qualitative analysis of text? How much can we trust the results? And what prompting strategies are most effective? In this lecture we discuss the emerging literature on this topic.

Lecture Readings

Dunivin, Z. O. (2024). Scalable Qualitative Coding with LLMs: Chain-of-Thought Reasoning Matches Human Performance in Some Hermeneutic Tasks. arXiv preprint arXiv:2401.15170.

Torii, M. G., Murakami, T., & Ochiai, Y. (2024). Expanding Horizons in HCI Research Through LLM-Driven Qualitative Analysis. arXiv preprint arXiv:2401.04138.

Zhang, H., Wu, C., Xie, J., Lyu, Y., Cai, J., & Carroll, J. M. (2023). Redefining qualitative analysis in the AI era: Utilizing ChatGPT for efficient thematic analysis. arXiv preprint arXiv:2309.10771.

Dengel, A., Gehrlein, R., Fernes, D., Görlich, S., Maurer, J., Pham, H. H., … & Eisermann, N. D. G. (2023, October). Qualitative Research Methods for Large Language Models: Conducting Semi-Structured Interviews with ChatGPT and BARD on Computer Science Education. In Informatics (Vol. 10, No. 4, p. 78). MDPI. Chicago

Jalali, M. S., & Akhavan, A. (2024). Integrating AI Language Models in Qualitative Research: Replicating Interview Data Analysis with ChatGPT. Available at SSRN 4714998. Chicago

Liu, A., & Sun, M. (2023). From Voices to Validity: Leveraging Large Language Models (LLMs) for Textual Analysis of Policy Stakeholder Interviews. arXiv preprint arXiv:2312.01202.

Nguyen-Trung, K. ChatGPT in Thematic Analysis: Can AI become a research assistant in qualitative research?.