CHATGPT, SMART WRITING ASSISTANT CHATBOT FOR STUDENTS: AN ANALYSIS OF ITS DRAWBACKS

S. Rizal Yazid, Dzulfikri Dzulfikri

Abstract


This research examines the drawbacks associated with ChatGPT, a smart writing assistant chatbot, within the context of student writing. Employing qualitative methods, notably in-depth interviews with 30 participants, the study meticulously analyzes the challenges and constraints entailed in relying on ChatGPT for writing assistance. Centered around five key themes — understanding, dependency, feedback, writing style, and plagiarism — the findings clarify that students struggle with comprehending ChatGPT's output, potentially leading to confusion and misinterpretation. The feedback provided by the AI writing assistant emerges as generic and lacking specificity, constraining its efficacy in guiding students' writing processes. Participants report an obvious shift towards more predictable and impersonal writing styles attributed to their usage of ChatGPT—furthermore, apprehensions surface concerning the unintentional risk of plagiarism when leveraging the tool. The implications underscore educators' need to acknowledge the limitations of AI writing assistants, offering targeted support to help students navigate these challenges while fostering critical thinking, creativity, and independent writing skills. The findings promote ongoing research and improvement in AI writing assistants to improve understanding, elevate feedback quality, and address identified limitations. By mitigating these drawbacks, educators, students, and developers can optimize the utility of AI writing assistants, ensuring they serve as valuable tools for supporting writing development while upholding critical thinking, creativity, and ethical writing practices. This research not only reveals the drawbacks of ChatGPT as a writing assistant but also advocates for continual research and development in the field of AI writing assistance.

Keywords


AI writing assistants; ChatGPT; drawbacks; student writing

Full Text:

PDF

References


Adeoye-Olatunde, O. A., & Olenik, N. L. (2021). Research and scholarly methods: Semi-structured interviews. JACCP: Journal of The American College of Clinical Pharmacy, 4(10), 1358–1367. https://doi.org/https://doi.org/10.1002/jac5.1441

Alejandrina Sotelo Muñoz, S., Gutiérrez Gayoso, G., Caceres Huambo, A., Domingo Cahuana Tapia, R., Layme Incaluque, J., Nacional Federico Villarreal, U., Cielo Ramírez Cajamarca, J., Enrique Reyes Acevedo, J., Victor Huaranga Rivera, H., & Luis Arias-Gonzáles, J. (2023). Examining the Impacts of ChatGPT on Student Motivation and Engagement. Social Space Journal, 23(1), 1–27. https://orcid.org/0000-0001-8821-6209

Ali, J. K. M., Shamsan, M. A. A., Hezam, T. A., & Mohammed, A. A. Q. (2023). Impact of ChatGPT on Learning Motivation: Journal of English Studies in Arabia Felix, 2(1), 41–49. https://doi.org/10.56540/jesaf.v2i1.51

Aljanabi, M., Ghazi, M., Ali, A. H., & Abed, S. A. (2023). ChatGpt: Open Possibilities. Iraqi Journal for Computer Science and Mathematics, 62–64. https://doi.org/10.52866/20ijcsm.2023.01.01.0018

Barrot, J. S. (2023). Using ChatGPT for second language writing: Pitfalls and potentials. Assessing Writing, 57, 100745. https://doi.org/https://doi.org/10.1016/j.asw.2023.100745

Bjelm, L. (2022). Teachers’ trust in AI-based education technology: A study in higher education in Singapore. 13. http://urn.kb.se/resolve?urn=urn:nbn:se:kth:diva-321223

Bozkurt, A., Xiao, F., Lambert, S. R., & Pazurek, A. (2023). Speculative Futures on ChatGPT and Generative Artificial Intelligence (AI): A Collective Reflection from the Educational Landscape. Asian Journal of Distance Education, 18(1), 53–130. https://doi.org/10.5281/zenodo.7636568

Creswell, J. W., & Creswell, J. D. (2018). Research design: Qualitative, quantitative, and mixed methods approaches (H. Salmon, C. Neve, & M. O’Heffernan, Eds.; 5th ed., Vol. 5). SAGE Publications, Inc.

Deng, J., & Lin, Y. (2022). The Benefits and Challenges of ChatGPT: An Overview. Frontiers in Computing and Intelligent Systems, 2(2), 81–83.

Dwivedi, Y. K., Kshetri, N., Hughes, L., Slade, E. L., Jeyaraj, A., Kar, A. K., Baabdullah, A. M., Koohang, A., Raghavan, V., Ahuja, M., Albanna, H., Albashrawi, M. A., Al-Busaidi, A. S., Balakrishnan, J., Barlette, Y., Basu, S., Bose, I., Brooks, L., Buhalis, D., … Wright, R. (2023). “So what if ChatGPT wrote it?” Multidisciplinary perspectives on opportunities, challenges and implications of generative conversational AI for research, practice and policy. International Journal of Information Management, 71. https://doi.org/10.1016/j.ijinfomgt.2023.102642

Farrokhnia, M., Banihashem, S. K., Noroozi, O., & Wals, A. (2023). A SWOT analysis of ChatGPT: Implications for educational practice and research. Innovations in Education and Teaching International, 1–15. https://doi.org/10.1080/14703297.2023.2195846

Firat, M. (2023). How Chat GPT Can Transform Autodidactic Experiences and Open Education? https://doi.org/10.31219/osf.io/9ge8m

Fitria, T. N. (2021). Grammarly as AI-powered English Writing Assistant: Students’ Alternative for Writing English. Metathesis: Journal of English Language, Literature, and Teaching, 5(1), 65. https://doi.org/10.31002/metathesis.v5i1.3519

Fu, Q.-K., Zou, D., Xie, H., & Cheng, G. (2022). A review of AWE feedback: types, learning outcomes, and implications. Computer Assisted Language Learning, 1–43. https://doi.org/10.1080/09588221.2022.2033787

Fyfe, P. (2023). How to cheat on your final paper: Assigning AI for student writing. AI & SOCIETY, 38(4), 1395–1405. https://doi.org/10.1007/s00146-022-01397-z

Haleem, A., Javaid, M., & Singh, R. P. (2022). An era of ChatGPT as a significant futuristic support tool: A study on features, abilities, and challenges. BenchCouncil Transactions on Benchmarks, Standards and Evaluations, 2(4), 100089. https://doi.org/10.1016/j.tbench.2023.100089

Husband, G. (2020). Ethical Data Collection and Recognizing the Impact of Semi-Structured Interviews on Research Respondents. Education Sciences, 10(8). https://doi.org/10.3390/educsci10080206

Javaid, M., Haleem, A., Singh, R. P., Khan, S., & Khan, I. H. (2023). Unlocking the opportunities through ChatGPT Tool towards ameliorating the education system. BenchCouncil Transactions on Benchmarks, Standards and Evaluations, 3(2), 100115. https://doi.org/https://doi.org/10.1016/j.tbench.2023.100115

Kasneci, E., Sessler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., Krusche, S., Kutyniok, G., Michaeli, T., Nerdel, C., Pfeffer, J., Poquet, O., Sailer, M., Schmidt, A., Seidel, T., … Kasneci, G. (2023). ChatGPT for good? On opportunities and challenges of large language models for education. Learning and Individual Differences, 103, 102274. https://doi.org/https://doi.org/10.1016/j.lindif.2023.102274

Liao, Z., Wang, J., Shi, Z., Lu, L., & Tabata, H. (2023). Revolutionary Potential of ChatGPT in Constructing Intelligent Clinical Decision Support Systems. Annals of Biomedical Engineering. https://doi.org/10.1007/s10439-023-03288-w

Lin, X. (2023). Exploring the Role of ChatGPT as a Facilitator for Motivating Self-Directed Learning Among Adult Learners. Adult Learning, 10451595231184928. https://doi.org/10.1177/10451595231184928

Lund, B. D., Wang, T., Mannuru, N. R., Nie, B., Shimray, S., & Wang, Z. (2023). ChatGPT and a new academic reality: Artificial Intelligence-written research papers and the ethics of the large language models in scholarly publishing. Journal of the Association for Information Science and Technology, 74(5), 570–581. https://doi.org/https://doi.org/10.1002/asi.24750

Malinka, K., Peres’{i}ni, M., Firc, A., Hujnák, O., & Janus, F. (2023). On the Educational Impact of ChatGPT: Is Artificial Intelligence Ready to Obtain a University Degree? Proceedings of the 2023 Conference on Innovation and Technology in Computer Science Education V. 1, 47–53. https://doi.org/10.1145/3587102.3588827

Nazari, N., Shabbir, M. S., & Setiawan, R. (2021). Application of Artificial Intelligence powered digital writing assistant in higher education: randomized controlled trial. Heliyon, 7(5). https://doi.org/10.1016/j.heliyon.2021.e07014

Neumann, A. T., Arndt, T., Köbis, L., Meissner, R., Martin, A., de Lange, P., Pengel, N., Klamma, R., & Wollersheim, H.-W. (2021). Chatbots as a Tool to Scale Mentoring Processes: Individually Supporting Self-Study in Higher Education. Frontiers in Artificial Intelligence, 4. https://doi.org/10.3389/frai.2021.668220

Popenici, S. A. D., & Kerr, S. (2017). Exploring the impact of artificial intelligence on teaching and learning in higher education. Research and Practice in Technology Enhanced Learning, 12(1). https://doi.org/10.1186/s41039-017-0062-8

Roshaidai, S., & Arifin, M. (2018). Ethical Considerations in Qualitative Study. In International Journal of Care Scholars (Vol. 1, Issue 2).

Saunders, B., Sim, J., Kingstone, T., Baker, S., Waterfield, J., Bartlam, B., Burroughs, H., & Jinks, C. (2018). Saturation in qualitative research: exploring its conceptualization and operationalization. Quality and Quantity, 52(4), 1893–1907. https://doi.org/10.1007/s11135-017-0574-8

Seo, K., Tang, J., Roll, I., Fels, S., & Yoon, D. (2021). The impact of artificial intelligence on learner–instructor interaction in online learning. International Journal of Educational Technology in Higher Education, 18(1), 54. https://doi.org/10.1186/s41239-021-00292-9

Shi, S., Zhao, E., Tang, D., Wang, Y., Li, P., Bi, W., Jiang, H., Huang, G., Cui, L., Huang, X., Zhou, C., Dai, Y., & Ma, D. (2022). Effidit: Your AI Writing Assistant. https://mp.weixin.qq.com/s/b-kPSR3aFPKHpUnFv7gmeA

Shidiq, M. (2023). The Use of Artificial Intelligence-Based Chat-GPT and Its Challenges For The World of Education; From The Viewpoint of The Development of Creative Writing Skills. In Society and Humanity (Vol. 01, Issue 01).

Yuan, A., Coenen, A., Reif, E., & Ippolito, D. (2022). Wordcraft: Story Writing With Large Language Models. 27th International Conference on Intelligent User Interfaces, 841–852. https://doi.org/10.1145/3490099.3511105

Zhai, X. (2023). ChatGPT and AI: The Game Changer for Education. AI4STEM Education Center, 1–5. https://ssrn.com/abstract=4389098

Zhai, X., Chu, X., Chai, C. S., Jong, M. S. Y., Istenic, A., Spector, M., Liu, J. B., Yuan, J., & Li, Y. (2021). A Review of Artificial Intelligence (AI) in Education from 2010 to 2020. In Complexity (Vol. 2021). Hindawi Limited. https://doi.org/10.1155/2021/8812542

Zhao, X. (2022). Leveraging Artificial Intelligence (AI) Technology for English Writing: Introducing Wordtune as a Digital Writing Assistant for EFL Writers. In RELC Journal. SAGE Publications Ltd. https://doi.org/10.1177/00336882221094089




DOI: https://doi.org/10.18860/abj.v9i1.24459

Refbacks

  • There are currently no refbacks.


Copyright (c) 2024 S. RIZAL YAZID, Dzulfikri Dzulfikri

License URL: https://creativecommons.org/licenses/by-sa/4.0/

...............................................................................................................................................................

Mailing Address:

Ruang Jurnal Abjadia, Fakultas Ilmu Tarbiyah dan Keguruan (FITK)
Universitas Islam Negeri (UIN) Maulana Malik Ibrahim Malang
Jalan Gajayana 50 Malang 65144, Jawa Timur, Indonesia

Phone/Faximile: (+62341) 552398, Phonsel: +6281333666063
Website: http://ejournal.uin-malang.ac.id/index.php/abjadia
Email: abjadia@uin-malang.com

...............................................................................................................................................................

Abjadia: International Journal of Education. This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

...............................................................................................................................................................

Indexed by: