Abstract

Background/purpose – ChatGPT, an AI-powered chatbot designed with generative pre-trained transformer architecture, have intrigued millions of people from diverse backgrounds since its first release, and generated excitement with its groundbreaking performance in numerous use cases they have been tested. Its ability to generate coherent and contextually relevant responses across a wide range of topics have also attracted the attention of researchers, and several scholars have investigated the potential uses of this AI-based tool to promote scientific work. The current study was designed with a similar purpose, and aimed to conduct scientific research on the effects of COVID-19 pandemic on education with a collaborative analysis of human-artificial intelligence.

Materials/methods – The data were generated from simultaneous interviews with ChatGPT-3.5 and 4, and their responses were evaluated comparatively in terms of the accuracy, clarity, conciseness, and breadth of information provided for (1) the definition of the COVID-19 pandemic, (2) the negative impact of the COVID-19 pandemic on education, (3) the positive impact of the COVID-19 pandemic on education, (4) recommendations to improve the effectiveness of teaching and learning during the COVID-19 pandemic. We evaluated the responses using a trichotomous rating system, and also calculated Cohen’s kappa values to assess inter-rater agreements for each category of evaluation.

Results – The results indicated that both versions demonstrated a strong potential to generate accurate, clear and concise information with a satisfactory breadth. As can be expected, ChatGPT-4 provided more categorized and synthesized information, and was better at demonstrating critical reasoning.

Conclusion – These results indicate that ChatGPT is a promising tool that can support scientific research process in collaboration with human intelligence, and its newer versions could be developed in a way that these chatbots could be ethically and safely integrated into different stages of scientific work.

Keywords: ChatGPT, COVID-19 pandemic, artificial intelligence, global health crisis, education, chatbot, OpenAI

Adatara, P.; Kuug, A.K.; Nyande, F.K.; Klutsey, E.E.; Johnson, B.B.; Nyefene, M.K.; Amooba, P.A.; Achaliwie, F.; Maalman, R.S.E.; Sedinam, G.B.; Prempeh, E.B.A.; Kodjo, M.M. A Qualitative Study on Frontline Nurses’ Experiences and Challenges in Providing Care for COVID-19 Patients in the Volta Region of Ghana: Implications for Nursing Management and Nursing Workforce Retention. Healthcare 2023, 11, 1028. https://doi.org/10.3390/healthcare11071028

Almazova, N., Krylova, E., Rubtsova, A., & Odinokaya, M. (2020). Challenges and opportunities for Russian higher education amid COVID-19: Teachers’ perspective. Education Sciences, 10(12), 368.

Azevedo, J.P.; Hasan, A.; Goldemberg, D.; Geven, K.; Iqbal, S.A. (2021) Simulating the Potential Impacts of COVID-19 School Closures on Schooling and Learning Outcomes: A Set of Global Estimates. The World Bank Research Observer, 36(1), 1-40, https://doi.org/10.1093/wbro/lkab003

Bawaneh, A. K., & Malkawı, E. (2023). Stem Faculty Members’ perspectives and challenges towards dıstance learnıng and virtual classes during COVID-19 outbreak. Turkish Online Journal of Distance Education, 24(1), 246-261.

Betthäuser, B. A., Bach-Mortensen, A. M., & Engzell, P. (2023). A systematic review and meta-analysis of the evidence on learning during the COVID-19 pandemic. Nature Human Behaviour, 7(3), 375-385. https://doi.org/10.17605/osf.io/u8gaz 

Bratianu, C., & Bejinaru, R. (2021). COVID‐19 induced emergent knowledge strategies. Knowledge and Process Management, 28(1), 11-17. https://doi.org/10.1002/kpm.1656

Brom, C., Lukavský, J., Greger, D., Hannemann, T., Straková, J., & Švaříček, R. (2020). Mandatory home education during the COVID-19 lockdown in the Czech Republic: A rapid survey of 1st-9th graders' parents. Frontiers in Education 5, p. 103-119.

Carvalho, I. and Ivanov, S. (2023), ChatGPT for tourism: applications, benefits and risks, Tourism Review, Vol. ahead-of-print No.ahead-of-print. https://doi.org/10.1108/TR-02-2023-0088

Cascella, M., Rajnik, M., Aleem, A., Dulebohn, S. C., & Di Napoli, R. (2022). Features, evaluation, and treatment of coronavirus (COVID-19). Statpearls [internet].

Castelvecchi, D. (2022). Are ChatGPT and AlphaCode going to replace programmers? Nature. doi: https://doi.org/10.1038/ d41586-022-04383-z.

Chierichetti, M., & Backer, P. (2021). Exploring faculty perspectives during emergency remote teaching in engineering at a large public university. Education Sciences, 11(8), 419.

Choi, J., Park, Y., Kim, H. E., Song, J., Lee, D., Lee, E., ... & Lee, Y. (2021). Daily life changes and life satisfaction among Korean school-aged children in the COVID-19 pandemic. International journal of environmental research and public health, 18(6), 3324.

Chui, M., Roberts, R. and Yee, L. (2022), Generative AI is here: how tools like ChatGPT could change your business, McKinsey and Company, available at: www.mckinsey.com/capabilities/quantumblack/our-insights/generative-ai-is-here-how-tools-like-chatgpt-could-change-your-business (accessed 13 March 2023).

Cui, Y., Ma, Z., Wang, L., Yang, A., Liu, Q., Kong, S., & Wang, H. (2023). A survey on big data-enabled innovative online education systems during the COVID-19 pandemic. Journal of Innovation & Knowledge, 8(1), https://doi.org/10.1016/j.jik.2022.100295

Delouya, S. (2022), I asked ChatGPT to do my work and write an insider article for me. It quickly generated an alarmingly convincing article filled with misinformation, Business Insider, available at: www.businessinsider.com/i-asked-chatgpt-to-write- insider-story-it- was-convincing-2022-12 (accessed 16 March 2023).

Erbay, A. (2020). Uzamış COVID: Yeni bir tanım. Bozok Tıp Dergisi.

Eringfeld, S. (2021). Higher education and its post-coronial future: utopian hopes and dystopian fears at Cambridge University during Covid-19. Studies in Higher Education, 46(1), 146-157. https://doi.org/10.1080/03075079.2020.1859681

Filip, R., Gheorghita Puscaselu, R., Anchidin-Norocel, L., Dimian, M., & Savage, W. K. (2022). Global Challenges to Public Health Care Systems during the COVID-19 Pandemic: A Review of Pandemic Measures and Problems. Journal of Personalized Medicine, 12(8), 1295. http://dx.doi.org/10.3390/jpm12081295

Francis, A. I., Ghany, S., Gilkes, T., & Umakanthan, S. (2022). Review of COVID-19 vaccine subtypes, efficacy and geographical distributions. Postgraduate medical journal, 98(1159), 389-394.

Gandamayu, I. B. M., Antari, N. W. S., & Strisanti, I. A. S. (2022). The level of community compliance in implementing health protocols to prevent the spread of COVID-19. International Journal of Health and Medical Sciences, 5(2), 177-182.

Garcez, A., Silva, R., & Franco, M. (2022). Digital transformation shaping structural pillars for academic entrepreneurship: A framework proposal and research agenda. Education and Information Technologies, 1-24. https://doi.org/10.1007/s10639-021-10638-5

Gozalo-Brizuela, R. and Garrido-Merchan, E.C. (2023), ChatGPT is not all you need. A state of the art review of large generative AI models, arXiv, doi: 10.48550/arxiv.2301.04655 (accessed 19 April 2023).

Haleem, A., Javaid, M., & Vaishya, R. (2020). Effects of COVID-19 pandemic in daily life. Current medicine research and practice, 10(2), 78.

Huang, C., Wang, Y., Li, X., Ren, L., Zhao, J., Hu, Y., Zhang, L., Fan, G., Xu, J., Gu, X., & Cheng, Z. (2020). Clinical features of patients infected with 2019 novel coronavirus in Wuhan, China. The Lancet, 395(10223), 497–506. https://doi.org/10.1016/S0140-6736(20)30183-5  

Hume, S., Brown, S. R., & Mahtani, K. R. (2023). School closures during COVID-19: an overview of systematic reviews. BMJ Evidence-Based Medicine. http://dx.doi.org/10.1136/bmjebm-2022-112085

Iyer, P., Aziz, K., & Ojcius, D. M. (2020). Impact of COVID‐19 on dental education in the United States. Journal of dental education, 84(6), 718-722.

Jones, P., & Comfort, D. (2020). The COVID-19 crisis, tourism and sustainable development. Athens Journal of Tourism, 7(2), 75-86.

Karakose, T. (2021). The impact of the COVID-19 epidemic on higher education: Opportunities and implications for policy and practice. Educational Process: International Journal, 10(1), 7-12. https://doi.org/10.22521/edupij.2021.101.1

Karakose, T. (2021b). Emergency remote teaching due to COVID-19 pandemic and potential risks for socioeconomically disadvantaged students in higher education. Educational Process: international journal, 10(3): 53-61. https://dx.doi.org/10.22521/edupij.2021.103.4

Karakose, T., Yirci, R., Basyigit, H., & Kucukcakir, A. (2021). Investigation of associations between the effects of COVID-19 fear on school administrators and nutrition and problematic eating behaviors. Progress in Nutrition, 23(27): e2021187. https://doi.org/10.23751/pn.v23i2.11656

Karakose, T., Yirci, R., Papadakis, S. (2022). Examining the associations between COVID-19- related psychological distress, social media addiction, COVID-19-related burnout, and depression among school principals and teachers through structural equation modeling. International Journal of Environmental Research and Public Health, 19, 1951. https://doi.org/10.3390/ijerph19041951

Karakose, T. (2023). The Utility of ChatGPT in Educational Research—Potential Opportunities and Pitfalls. Educational Process: International Journal, 12(2): 7-13. https://dx.doi.org/10.22521/edupij.2023.122.1

Karakose, T., Demirkol, M., Yirci, R., Polat, H., Ozdemir, T.Y., & Tülübaş, T. (2023). A Conversation with ChatGPT about Digital Leadership and Technology Integration: Comparative Analysis Based on Human–AI Collaboration. Administrative Sciences, 13(7), 157. https://doi.org/10.3390/admsci13070157

Kim, Y.; Shin, J.M.; Yoo, S.H.; Keam, B. Challenges in Care for Non-COVID-19 Patients with Severe Chronic Illnesses during COVID-19 Pandemic: A Qualitative Study of Healthcare Providers Working around Acute Care Hospitals in South Korea. Healthcare 2023, 11, 611. https://doi.org/10.3390/healthcare11040611

Kotowski, S. E., Davis, K. G., & Barratt, C. L. (2022). Teachers feeling the burden of COVID-19: Impact on well-being, stress, and burnout. Work, (Preprint), 1-9.

Landis, J.R., & Koch, G.G. (1977). The measurement of observer agreement for categorical data. Biometrics 33, 159-174.

Li, J., & Che, W. (2022). Challenges and coping strategies of online learning for college students in the context of COVID-19: A survey of Chinese universities. Sustainable Cities and Society, 83, 103958.

Liu, K., Chen, Y., Lin, R., & Han, K. (2020). Clinical features of COVID-19 in elderly patients: A comparison with young and middle-aged patients. Journal of Infection, 80(6), e14-e18.

Lu, R., Zhao, X., Li, J., Niu, P., Yang, B., Wu, H., ... & Tan, W. (2020). Genomic characterisation and epidemiology of 2019 novel coronavirus: implications for virus origins and receptor binding. The lancet, 395(10224), 565-574.

Min, B., Ross, H., Sulem, E., Veyseh, A.P., Nguyen, T.H., Sainz, O., Agirre, E., Heinz, I. and Roth, D. (2021), Recent advances in natural language processing via large pre-trained language models: a survey, arXiv, doi: 10.48550/arxiv.2111.01243 (accessed 21 March 2023).

Oliveira, G., Grenha Teixeira, J., Torres, A., & Morais, C. (2021). An exploratory study on the emergency remote education experience of higher education students and teachers during the COVID‐19 pandemic. British Journal of Educational Technology, 52(4), 1357-1376.

OpenAI (2022), ChatGPT: optimizing language models for dialogue, available at: https://openai.com/blog/chatgpt/

Oppenheimer D (2023) ChatGPT has arrived – and nothing has changed |Times Higher Education. https:// www. times highe reducation. com/ campus/ chatg pt- has- arriv ed- and- nothi ng- has- chang ed. (accessed 18 March 2023)

Pokhrel, S., & Chhetri, R. (2021). A literature review on impact of COVID-19 pandemic on teaching and learning. Higher education for the future, 8(1), 133-141.

Rahman, M. M., Masum, M. H. U., Wajed, S., & Talukder, A. (2022). A comprehensive review on COVID-19 vaccines: development, effectiveness, adverse effects, distribution and challenges. Virusdisease, 33(1), 1-22.

Restu, N. K., Suryana, C., Herman, T., & Mulyati, T. (2023). Learning Loss pada Kemampuan Pemahaman Konsep dan Representasi Matematis Peserta Didik Kelas V SD di Era Post Pandemic Covid-19. Jurnal Basicedu, 7(1), 288-294.

Sacco P.L, De Domenico M. Public health challenges and opportunities after COVID-19. Bull World Health Organ. 2021, 99(7), 529-535. http://dx.doi.org/10.2471/BLT.20.267757

Sadjadi, E.N. Challenges and Opportunities for Education Systems with the Current Movement toward Digitalization at the Time of COVID-19. Mathematics 2023, 11, 259. https://doi.org/10.3390/math11020259

Sahu, P. (2020). Closure of universities due to coronavirus disease 2019 (COVID-19): impact on education and mental health of students and academic staff. Cureus, 12(4).

Saliba, T., Boitsios, G. ChatGPT, a radiologist’s perspective. Pediatr Radiol 53, 813–815 (2023). https://doi.org/10.1007/s00247-023-05656-z

Schuurman, T. M., Henrichs, L. F., Schuurman, N. K., Polderdijk, S., & Hornstra, L. (2023). Learning loss in vulnerable student populations after the first COVID-19 school closure in the Netherlands. Scandinavian Journal of Educational Research, 67(2), 309-326. https://doi.org/10.1080/00313831.2021.2006307

Segev-Jacubovski, O. (2023). Changes in Daily Occupations Patterns, Optimism, and Positive Affect During COVID-19 Lockdown. Canadian Journal of Occupational Therapy, https://doi.org/10.1177/00084174231166260  

Shah, C. (2022), The Rise of AI Chat Agents and the Discourse With Dilettantes. Information Matters, 2 (12). https://informationmatters.Org/2022/12/The-Rise-Of-Aichat-Agents-And-The-Discourse-WithDilettantes/ 

Shen, Y., Heacock, L., Elias, J., Hentel, K.D., Reig, B., Shih,G. andMoy, L. (2023), ChatGPT and other large languagemodels are double-edged swords, Radiology, Vol. 230163, doi: 10.1148/radiol.230163.

Sriram, D., Sanjay, L., Nayak, N., Sathish, S., Kodipalli, A., Anil, P.N. (2023). Analysis of Effectiveness of Online Classes During COVID. In: Kumar, R., Verma, A.K., Sharma, T.K., Verma, O.P., Sharma, S. (eds) Soft Computing: Theories and Applications. Lecture Notes in Networks and Systems, vol 627. Springer, Singapore. https://doi.org/10.1007/978-981-19-9858-4_70  

Stokel-Walker, C. 2022. AI bot ChatGPT writes smart essays — should professors worry? Nature. https://doi.org/10.1038/ d41586-022-04397-7.

Sun, S. (2011). Meta-analysis of Cohen’s kappa. Health Services and Outcomes Research Methodology 11, 145-163. http://doi.org/10.1007/s10742-011-0077-3

Tang, K. H. D. (2023). Impacts of COVID-19 on primary, secondary and tertiary education: a comprehensive review and recommendations for educational practices. Educational Research for Policy and Practice, 22(1),23-61. https://doi.org/10.1007/s10671-022-09319-y

Tulaskar, R., & Turunen, M. (2022). What students want? Experiences, challenges, and engagement during Emergency Remote Learning amidst COVID-19 crisis. Education and information technologies, 27(1), 551-587.

Tülübaş, T., Demirkol, M., Ozdemir, T.Y., Polat, H., Karakose, T., & Yirci, R. (2023). An Interview with ChatGPT on Emergency Remote Teaching: A Comparative Analysis Based on Human–AI Collaboration. Educational Process: International Journal, 12(2): 93-110. https://dx.doi.org/10.22521/edupij.2023.122.6 

Vallance, C. 2022. ChatGPT: New AI chatbot has everyone talking to it, https://www.bbc

van Dis, E.A.M., Bollen, J., Zuidema, W., van Rooij, R. and Bockting, C.L. (2023), ChatGPT: five priorities for research, Nature, Vol. 614 No. 7947, pp. 224-226, doi: 10.1038/d41586-023-00288-7.

Verdecchia, P., Cavallini, C., Spanevello, A., & Angeli, F. (2020). COVID-19: ACE2centric infective disease?. Hypertension, 76(2), 294-299.

Vieira, S. M., Kaymak, U., & Sousa, J. M. (2010, July). Cohen's kappa coefficient as a performance measure for feature selection. In International Conference on Fuzzy Systems (pp. 1-8), July, 18-23, 2010 - CCIB, Barcelona, Spain

Viner, R. M., Russell, S. J., Croker, H., Packer, J., Ward, J., Stansfield, C., ... & Booy, R. (2020). School closure and management practices during coronavirus outbreaks including COVID-19: a rapid systematic review. The Lancet Child & Adolescent Health, 4(5), 397-404.

Waluyo, E. M. (2022). Healthy Community Movement (GERMAS) And Distance, Wearing Mask, And Washing Hands (3m) In the Elderly to Prevent the Spread of Covid 19. ABDIMAS: Jurnal Pengabdian Masyarakat, 5(1), 2172-2176.

Wan, M., Kelemen, T. K., Zhang, Y., & Matthews, S. H. (2022). An island of sanity during COVID-19 pandemic: Does pet attachment support buffer employees’ stress due to job insecurity?. Psychological reports, 00332941221109105.

Wang, L., Didelot, X., Yang, J. et al. Inference of person-to-person transmission of COVID-19 reveals hidden super-spreading events during the early outbreak phase. Nat Commun 11, 5006 (2020). https://doi.org/10.1038/s41467-020-18836-4

Warrens, M. J. (2015). Five ways to look at Cohen's kappa. Journal of Psychology & Psychotherapy 5(4): 1. http://dx.doi.org/10.4172/2161-0487.1000197 

Willems, J. (2023), Chatgpt At Universities- The Least of our Concerns. Https://Ssrn.Com/Abstract=4334162. Http://Dx.Doi.Org/10.2139/Ssrn.4334162

Winkelmann, J., Webb, E., Williams, G. A., Hernández-Quevedo, C., Maier, C. B., & Panteli, D. (2022). European countries' responses in ensuring sufficient physical infrastructure and workforce capacity during the first COVID-19 wave. Health Policy, 126(5), 362-372. https://doi.org/10.1016/j.healthpol.2021.06.015

World Health Organization, 2. (2020). Transmission of SARS-CoV-2: implications for infection prevention precautions: scientific brief, 09 July 2020 (No. WHO/2019-nCoV/Sci_Brief/Transmission_modes/2020.3). World Health Organization.

World Health Organization. (2020). WHO Director-General’s opening remarks at the media briefing on COVID-19.

Yirci, R., & Özdemir, T. Y. (2021). Covid-19 pandemisinin sosyoekonomik ve psikolojik göstergeleri ile Türk eğitim sistemi üzerindeki etkileri. Tarih Okulu Dergisi, 14(53), 2440-2466. http://dx.doi.org/10.29228/joh.52016

Zhao, W., Li, H., Li, J., Xu, B., & Xu, J. (2022). The mechanism of multiple organ dysfunction syndrome in patients with COVID‐19. Journal of medical virology, 94(5), 1886-1892.

Zhuo, T.Y., Huang, Y., Chen, C. and Xing, Z. (2023), Exploring AI ethics of ChatGPT: a diagnostic analysis, arXiv, doi: 10.48550/arxiv.2301.12867 (accessed 20 March 2023).