Using Natural Language Processing to Rhetorically Contextualize Audiences Vaccine Sentiment Analysis of Newspaper Comments, 2017–2023
Main Article Content
Abstract
This article demonstrates the value of sentiment analysis for contextualizing audiences in Rhetoric of Health and Medicine (RHM) by comparing vaccine related newspaper comments to non-vaccine related comments in the New York Times from 2017–2023 (n = 22,330,999). Our results show that while all comments skew negative, following a similar trend line, after the emergence of COVID-19, vaccine related comments decouple from the negative trend of baseline non-vaccine comments, becoming more negative and volatile. These results raise additional questions about the nature of the negativity for vaccine related comments, and we provide a properly sampled dataset for follow-up research to encourage iterative investigation into the public response to vaccine policy. In addition to these findings, this article calls for broader engagement with Natural Language Processing (NLP) and data science in RHM.
Article Details
References
Burowoy, Michael. (2009). The extended case method: Four countries, four decades, four great transformation, and one theoretical tradition. University of California Press.
Campeau, Kari L. (2019). Vaccine Barriers, Vaccine Refusals: Situated Vaccine Decision-Making in the Wake of the 2017 Minnesota Measles Outbreak. Rhetoric of Health & Medicine, 2(2), 176–207. https://doi.org/10.5744/rhm.2019.1007
Center for Disease Control. (2023). CDC Recommends Updated COVID-19 Vaccine for Fall/Winter Virus Season. Center for disease control. https://www.cdc.gov/media/releases/2023/p0912-COVID-19-Vaccine.html
Chen, Gina. M., & Lu, Shuning. (2017). Online Political Discourse: Exploring Differences in Effects of Civil and Uncivil Disagreement in News Website Comments. Journal of Broadcasting & Electronic Media, 61(1), 108–125. https://doi.org/10.1080/08838151.2016.1273922
Devlin, Jacob., Chang, Ming-Wei, Lee, Kenton., & Toutanova, Kristina. (2019). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding (No. arXiv:1810.04805). arXiv. https://doi.org/10.48550/arXiv.1810.04805
DiCaglio, Josh. (2021). Scale Theory: A Nondisciplinary Inquiry. University of Minnesota Press.
Duncan, Michael. (2011). Polemical Ambiguity and the Composite Audience: Bush’s 20 September 2001 Speech to Congress and the Epistle of 1 John. Rhetoric Society Quarterly, 41(5), 455–471. https://doi.org/10.1080/02773945.2011.596178
Dyson, Anne. H., & Genishi, Celia. (2005). On the Case: Approaches to Language and Literacy Research. Teachers College Press.
Gallagher, John R., Chen, Yinyin., Wagner, Kkyle., Wang, Xuan., Zeng, Jingyi., & Kong, Alyssa. L. (2020). Peering at the internet abyss: Using big data audience analysis to understand online comments. Technical Communication Quarterly, 29(2), 155–173. https://doi.org/10.1080/10572252.2019.1634766
Gallagher, John R., & Lawrence, Heidi. Y. (2020). Rhetorical Appeals and Tactics in New York Times Comments About Vaccines: Qualitative Analysis. Journal of Medical Internet Research, 22(12), e19504. https://doi.org/10.2196/19504
Gallagher, John R. (2024). Case study research in the digital age. Routledge.
Gesualdo, Francesco., Marino, Francesco., Mantero, Jas., Spadoni, Andrea., Sambucini, Luigi., Quaglia, Giammarco., Rizzo, Caterina., Sahinovic, Isabelle., Zuber, Patrick. L. F., & Tozzi, Alberto. E. (2020). The use of web analytics combined with other data streams for tailoring online vaccine safety information at global level: The Vaccine Safety Net’s web analytics project. Vaccine, 38(41), 6418–6426. https://doi.org/10.1016/j.vaccine.2020.07.070
Horton, Zachary. (2020). Viral zoom: COVID-19 as multi-scalar immune failure. International Journal of Performance Arts and Digital Media, 16(3), 319–340. https://doi.org/10.1080/14794713.2020.1827205
Horton, Zachary. (2021). The cosmic zoom: Scale, Knowledge, and mediation. University of Chicago Press.
Houston, J. Brian., Hansen, Glenn. J., & Nisbett, Gwendelyn. S. (2011). Influence of User Comments on Perceptions of Media Bias and Third-Person Effect in Online News. Electronic News, 5(2), 79–92. https://doi.org/10.1177/1931243111407618
Hutto, C. J., & Gilbert, Eric. (2014). VADER: A Parsimonious Rule-based Model for Sentiment Analysis of Social Media Text. Eighth International AAAI Conference on Weblogs and Social Media, 216–225. https://doi.org/10.1210/en.2011-1066
Kennedy, Helen. (2012). Perspectives on Sentiment Analysis. Journal of Broadcasting & Electronic Media, 56(4), 435–450. https://doi.org/10.1080/08838151.2012.732141
Kenzie, Daniel., & Anderson, Murphy. (2024). Don’t Read the Comments: Discourse About COVID-19 Vaccines in a State Health Department’s Social Media Comments. Journal of Technical Writing and Communication, 00472816241279821. https://doi.org/10.1177/00472816241279821
Kim, Youngju. (2015). Exploring the effects of source credibility and others’ comments on online news evaluation. Electronic News, 9(3), 160–176. https://doi.org/10.1177/1931243115593318
Lawrence, Heidi. (2018). When Patients Question Vaccines: Considering Vaccine Communication through a Material Rhetorical Approach. Rhetoric of Health & Medicine, 1(1–2), 161–178. https://doi.org/10.5744/rhm.2018.1010
Lawrence, Heidi. (2020). Vaccine Rhetorics. The Ohio State Univerity Press.
Lee, N. Yeon., & McElroy, Kathleen. (2019). Online comments: The nature of comments on health journalism. Computers in Human Behavior, 92, 282–287. https://doi.org/10.1016/j.chb.2018.11.006
Liu, Yinhan., Ott, Myle., Goyal, Naman., Du, Jingfei., Joshi, Mander., Chen, Danqi., Levy, Omer., Lewis, Mike., Zettlemoyer, Luke., & Stoyanov, Veselin. (2019). RoBERTa: A Robustly Optimized BERT Pretraining Approach (No. arXiv:1907.11692). arXiv. https://doi.org/10.48550/arXiv.1907.11692
Malkowski, Jennifer., & Melonçon, Lisa. (2019). The Rhetoric of Public Health for RHM Scholarship and Beyond. Rhetoric of Health & Medicine, 2(2), iii–xiii. https://doi.org/10.5744/rhm.2019.1010
McInnis, B., Ajmani, L., Sun, L., Hou, Y., Zeng, Z., & Dow, S. P. (2021). Reporting the Community Beat: Practices for Moderating Online Discussion at a News Website. Proceedings of the ACM on Human-Computer Interaction, 5(CSCW2), 1–25. https://doi.org/10.1145/3476074
Melonçon, Lisa., & St.Amant, Kirk. (2019). Empirical research in technical and professional communication: A 5-Year examination of research methods and a call for research sustainability. Journal of Technical Writing and Communication, 49(2), 128–155. https://doi.org/10.1177/0047281618764611
Molloy, Cathryn. (2019). Durable, Portable Research through Partnerships with Interdisciplinary Advocacy Groups, Specific Research Topics, and Larger Data Sets. Technical Communication Quarterly, 28(2), 165–176. https://doi.org/10.1080/10572252.2019.1588375
Moriarty, Devon., Núñez De Villavicencio, Paula., Black, Lillian. A., Bustos, Monica., Cai, Helen., Mehlenbacher, Brad., & Mehlenbacher, Ashley. R. (2019). Durable research, portable findings: Rhetorical methods in case study research. Technical Communication Quarterly, 28(2), 124–136. https://doi.org/10.1080/10572252.2019.1588376
Nemes, László., & Kiss, Attila. (2021). Social media sentiment analysis based on COVID-19. Journal of Information and Telecommunication, 5(1), 1–15. https://doi.org/10.1080/24751839.2020.1790793
Noble, Safiya. U. (2018). Algorithms of oppression: How search engines reinforce racism. New York University Press.
Perelman, Chaim., & Olbrechts-Tyteca, Lucie. (1969). The New Rhetoric: A Treatise on Argumentation. University of Notre Dame Press.
Qorib, Miftahul., Oladunni, Timothy., Denis, Max., Ososanya, Esther., & Cotae, Paul. (2023). COVID-19 vaccine hesitancy: Text mining, sentiment analysis and machine learning on COVID-19 vaccination Twitter dataset. Expert Systems with Applications, 212, 118715. https://doi.org/10.1016/j.eswa.2022.118715
Saha, Subrata., Showrov, H. Imran., & Rahman, Motinur. (2023). VADER vs. BERT: A Comparative Performance Analysis for Sentiment on Coronavirus Outbreak. In Md. S. Satu, M. A. Moni, M. S. Kaiser, & M. S. Arefin (Eds.), Machine Intelligence and Emerging Technologies—First International Conference, MIET 2022, Proceedings (Vol. 490, pp. 371–385). Springer. https://doi.org/10.1007/978-3-031-34619-4_30
Stake, Robert. E. (1995). The art of case study research. Sage Publications.
Stake, Robert. E. (2006). Multiple case study analysis. The Gilford Press.
St.Amant, Kirk., & Graham, S. Scott. (2019). Research that resonates: A perspective on durable and portable approaches to scholarship in technical communication and rhetoric of science. Technical Communication Quarterly, 28(2), 99–111. https://doi.org/10.1080/10572252.2019.1591118
Tausczik, Yla. R., & Pennebaker, James. W. (2010). The Psychological Meaning of Words: LIWC and Computerized Text Analysis Methods. Journal of Language and Social Psychology, 29(1), 24–54. https://doi.org/10.1177/0261927X09351676
Teston, Christa., & Torrence, Addison. (2024). Durability, Portability, and Responsivity in Rhetorics of Health and Medicine (RHM): A Scoping Study of RHM Research (2006–2020). Rhetoric of Health & Medicine, 7(4), 371–403.
Underwood, Ted. (2014). Theorizing Research Practices We Forgot to Theorize Twenty Years Ago. Representations, 127(1), 64–72. https://doi.org/10.1525/rep.2014.127.1.64
Wankhade, Mayur., Rao, Annavarapu. C. S., & Kulkarni, Chaitanya. (2022). A survey on sentiment analysis methods, applications, and challenges. Artificial Intelligence Review, 55(7), 5731–5780. https://doi.org/10.1007/s10462-022-10144-1
Wood, Simon. N. (2017). Generalized Additive Models: An Introduction with R. Routledge.
Yin, Hui., Song, Xiangyu., Yang, Shuiqiao., & Li, Jianxin. (2022). Sentiment analysis and topic modeling for COVID-19 vaccine discussions. World Wide Web, 25(3), 1067–1083. https://doi.org/10.1007/s11280-022-01029-y
Yin, Robert. K. (2009). Case study research: Design and methods. Sage Publications.