Are Pre-Service EFL Teachers Ready for AI-Assisted Assessment? The Role of Assessment Literacy in the Digital Era

https://doi.org/10.19105/panyonara.v7i2.21416
Abstract views: 70 , PDF downloads: 23
Keywords: Language Assessment Literacy, AI-based Assessment Readiness, EFL

Abstract

Artificial Intelligence (AI) is transforming educational assessment, particularly in English as a Foreign Language (EFL) settings. As the use of AI is becoming increasingly rapid, pre-service teachers must become proficient with both Language Assessment Literacy (LAL) and preparation to utilize AI tools. Therefore, this study explored the relationship between LAL and AI-based assessment readiness among Indonesian EFL pre-service teachers. Adopting a mixed-method explanatory sequential research design, 200 respondents across 60 universities in Indonesia participated through a questionnaire survey followed by interviews with the highest and lowest AI readiness scores. The quantitative data found a significant moderate positive correlation, implying that higher LAL is associated with greater readiness to use AI-based assessment. Moreover, the descriptive data indicated that while most participants demonstrated high LAL, their AI readiness was only moderately high. Qualitative data revealed that the respondents with better LAL have critical views about practices when it comes to assessment. The research concluded that to effectively integrate AI into assessment practices, pre-service teachers not only need technology skills training but also a solid assessment knowledge. These results have implications for the curriculum in teacher education, for which there is demand for integrated frameworks that link assessment theory with ethical AI implementation.

Downloads

Download data is not yet available.

Author Biography

Ive Emaliana, Universitas Brawijaya

Ive Emaliana's teaching and research interests include EFL epistemic beliefs, inclusive education, language assessment, and English for specific purposes. She is author and coauthor of 9 books including From Literacy to Multiliteracies (2020), Evaluasi Pembalajaran Bahasa Asing pada Pendidikan Tinggi (2019),  Academic Platforms for Students-Researchers (2018), and Success Stories in English Language Teaching and Learning (2014). Her papers were published in some edited books, conference proceedings, and national, international, Scopus-index journals. She can be contacted via e-mail at ive@ub.ac.id.

References

Abfalter, D., Mueller-Seeger, J., & Raich, M. (2021). Translation decisions in qualitative research: A systematic framework. International Journal of Social Research Methodology, 24(4), 469–486. https://doi.org/10.1080/13645579.2020.1805549

Al-Akbari, S., Nikolov, M., & Hódi, Á. (2025). EFL teachers’ language assessment literacy training needs. Social Sciences & Humanities Open, 11, 101254. https://doi.org/10.1016/j.ssaho.2024.101254

Arefian, M. H. (2022). Perceptions of self-assessment literacy and self-directed reflection during online learning for Iranian EFL student teachers. Reflective Practice, 23(6), 623–634. https://doi.org/10.1080/14623943.2022.2096584

Arslan, S. (2025). English-as-a-foreign language university instructors’ perceptions of integrating artificial intelligence: A Turkish perspective. System, 131, 103680. https://doi.org/10.1016/j.system.2025.103680

Ayanwale, M. A., Sanusi, I. T., Adelana, O. P., Aruleba, K. D., & Oyelere, S. S. (2022). Teachers’ readiness and intention to teach artificial intelligence in schools. Computers and Education: Artificial Intelligence, 3, 100099. https://doi.org/10.1016/j.caeai.2022.100099

Belford, N., Tudball, L., & Kewalramani, S. (2020). Research-informed ‘immersive’ professional learning with pre-service teachers in schools. Cogent Education, 7(1), 1733223. https://doi.org/10.1080/2331186X.2020.1733223

Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77–101. https://doi.org/10.1191/1478088706qp063oa

Brown, A. L., Myers, J., & Collins, D. (2021). How pre-service teachers’ sense of teaching efficacy and preparedness to teach impact performance during student teaching. Educational Studies, 47(1), 38–58. https://doi.org/10.1080/03055698.2019.1651696

Celik, I. (2023). Towards Intelligent-TPACK: An empirical study on teachers’ professional knowledge to ethically integrate artificial intelligence (AI)-based tools into education. Computers in Human Behavior.

Cirocki, A., Anam, S., Drajati, N., & Soden, B. (2025). Assessment Literacy Among Indonesian Pre-service English Language Teachers: A Mixed-Methods Study. Iranian Journal of Language Teaching Research, 13(1). https://doi.org/10.30466/ijltr.2025.55691.2851

Cohen, L., Manion, L., & Morrison, K. (2018). Research Methods in Education (Eight edition). Routledge.

Cope, B., Kalantzis, M., & Searsmith, D. (2021). Artificial intelligence for education: Knowledge and its assessment in AI-enabled learning ecologies. Educational Philosophy and Theory, 53(12), 1229–1245. https://doi.org/10.1080/00131857.2020.1728732

Creswell, J. W., & Clark, Vicki. L. P. (2018). Designing and Conducting Mixed Methods Research (Third edition). Los Angeles: SAGE.

Creswell, J. W., & Creswell, J. D. (2022). Research Design: Qualitative, Quantitative, and Mixed Methods Approaches (6th ed.). SAGE Publications, Inc.

Cui, Y., Liu, Y., Yu, H., & Gao, Y. (2022). Developing English teachers’ language assessment literacy in an EAP reform context through test design: A case study. System, 109, 102866. https://doi.org/10.1016/j.system.2022.102866

Fitriyah, I., Masitoh, F., & Widiati, U. (2022). Classroom-based language assessment literacy and professional development need between novice and experienced EFL teachers. Indonesian Journal of Applied Linguistics, 12(1), 124–134. https://doi.org/10.17509/ijal.v12i1.46539

Guan, L., Lee, J. C.-K., Zhang, Y. E., & Gu, M. M. (2025). Investigating the tripartite interaction among teachers, students, and generative AI in EFL education: A mixed-methods study. Computers and Education: Artificial Intelligence, 100384. https://doi.org/10.1016/j.caeai.2025.100384

Helmiatin, Hidayat, A., & Kahar, M. R. (2024). Investigating the adoption of AI in higher education: A study of public universities in Indonesia. Cogent Education, 11(1), 2380175. https://doi.org/10.1080/2331186X.2024.2380175

Kasneci, E., Sessler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., … Kasneci, G. (2023). ChatGPT for good? On opportunities and challenges of large language models for education. Learning and Individual Differences, 103, 102274. https://doi.org/10.1016/j.lindif.2023.102274

Kohnke, L. (2025). Preparing future educators for AI-enhanced classrooms: Insights into AI literacy and integration. Computers and Education: Artificial Intelligence, 8, 100398. https://doi.org/10.1016/j.caeai.2025.100398

Lan, C., & Fan, S. (2019). Developing classroom-based language assessment literacy for in-service EFL teachers: The gaps. Studies in Educational Evaluation, 61, 112–122. https://doi.org/10.1016/j.stueduc.2019.03.003

Lander, N., Mazzoli, E., Orr, J., Verswijveren, S. J. J. M., Contardo Ayala, A. M., Brown, T., & Muir, T. (2025). TransformEd: An innovative model to support professional growth of pre-service teachers. Teaching Education, 1–22. https://doi.org/10.1080/10476210.2025.2462312

Malterud, K., Siersma, V. D., & Guassora, A. D. (2016). Sample Size in Qualitative Interview Studies: Guided by Information Power. Qualitative Health Research, 26(13), 1753–1760. https://doi.org/10.1177/1049732315617444

Mekheimer, M., & Abdelhalim, W. M. (2025). The digital age students: Exploring leadership, freedom, and ethical online behavior: A quantitative study. Social Sciences & Humanities Open, 11, 101325. https://doi.org/10.1016/j.ssaho.2025.101325

Özer-Altınkaya, Z., & Yetkin, R. (2025). Exploring pre-service English language teachers’ readiness for AI-integrated language instruction. Pedagogies: An International Journal, 0(0), 1–17. https://doi.org/10.1080/1554480X.2025.2451299

Pardo, R., García-Pérez, D., & Panadero, E. (2024). Shaping the assessors of tomorrow: How practicum experiences develop assessment literacy in secondary education pre-service teachers. Teaching and Teacher Education, 152, 104798. https://doi.org/10.1016/j.tate.2024.104798

Pastore, S., & Andrade, H. L. (2019). Teacher assessment literacy: A three-dimensional model. Teaching and Teacher Education, 84, 128–138. https://doi.org/10.1016/j.tate.2019.05.003

Prestes, P. A. N., Silva, T. E. V., & Barroso, G. C. (2021). Correlation analysis using teaching and learning analytics. Heliyon, 7(11), e08435. https://doi.org/10.1016/j.heliyon.2021.e08435

Proudfoot, K. (2023). Inductive/Deductive Hybrid Thematic Analysis in Mixed Methods Research. Journal of Mixed Methods Research, 17(3), 308–326. https://doi.org/10.1177/15586898221126816

Rabbianty, E. N., Virdyna, N. K., Azizah, S., Roda’i, M., & Shidqi, Z. A. (2025). Breaking barriers: Integrating Padlet to introduce environmental education to pre-service teachers. PANYONARA: Journal of English Education, 7(1), 77–102. https://doi.org/10.19105/panyonara.v7i1.17849

Roslan, R., Nishio, Y., & Jawawi, R. (2022). Analyzing English language teacher candidates’ assessment literacy: A case of Bruneian and Japanese universities. System, 111, 102934. https://doi.org/10.1016/j.system.2022.102934

Saputra, E. R., Hamied, F. A., & Suherdi, D. (2020). The development of beliefs and practices of language assessment literacy: Does a professional learning community help? Journal of Education for Teaching, 46(3), 414–416. https://doi.org/10.1080/02607476.2020.1761250

Shafiee Rad, H., Alipour, J., Jafarpour, A., & Hashemian, M. (2024). Unlocking the magic of digital adaptivity: Unleashing students’ writing skills and self-determination motivation through digital literacy instruction. System, 125, 103454. https://doi.org/10.1016/j.system.2024.103454

Shahid, M. K., Zia, T., Bangfan, L., Iqbal, Z., & Ahmad, F. (2024). Exploring the relationship of psychological factors and adoption readiness in determining university teachers’ attitude on AI-based assessment systems. The International Journal of Management Education, 22(2), 100967. https://doi.org/10.1016/j.ijme.2024.100967

Swiecki, Z., Khosravi, H., Chen, G., Martinez-Maldonado, R., Lodge, J. M., Milligan, S., … Gašević, D. (2022). Assessment in the age of artificial intelligence. Computers and Education: Artificial Intelligence, 3, 100075. https://doi.org/10.1016/j.caeai.2022.100075

Taghizadeh, M., & and Ejtehadi, A. (2023). Investigating pre-service EFL teachers’ and teacher educators’ experience and attitudes towards online interaction tools. Computer Assisted Language Learning, 36(8), 1633–1667. https://doi.org/10.1080/09588221.2021.2011322

Viberg, O., Mutimukwe, C., Hrastinski, S., Cerratto-Pargman, T., & Lilliesköld, J. (2024). Exploring teachers’ (future) digital assessment practices in higher education: Instrument and model development. British Journal of Educational Technology, 55(6), 2597–2616. https://doi.org/10.1111/bjet.13462

Wulandari, M., & Purnamaningwulan, R. A. (2024). Exploring Indonesian EFL pre-service teachers’ experiences in AI-assisted teaching practicum: Benefits and drawbacks. LLT Journal: A Journal on Language and Language Teaching, 27(2), 878–894. https://doi.org/10.24071/llt.v27i2.8690

Xi, X. (2023). Advancing Language Assessment with AI and ML–Leaning into AI is Inevitable, but Can Theory Keep Up? Language Assessment Quarterly, 20(4–5), 357–376. https://doi.org/10.1080/15434303.2023.2291488

Xie, H., Chu, H.-C., Hwang, G.-J., & Wang, C.-C. (2019). Trends and development in technology-enhanced adaptive/personalized learning: A systematic review of journal publications from 2007 to 2017. Computers & Education, 140, 103599. https://doi.org/10.1016/j.compedu.2019.103599

Yalcin Arslan, F. (2019). Reflection in pre-service teacher education: Exploring the nature of four EFL pre-service teachers’ reflections. Reflective Practice, 20(1), 111–124. https://doi.org/10.1080/14623943.2018.1564652

Zaim, M., Arsyad, S., Waluyo, B., Ardi, H., Al Hafizh, Muhd., Zakiyah, M., … Hardiah, M. (2024). AI-powered EFL pedagogy: Integrating generative AI into university teaching preparation through UTAUT and activity theory. Computers and Education: Artificial Intelligence, 7, 100335. https://doi.org/10.1016/j.caeai.2024.100335

Zhang, L., Zheng, Y., & Yang, D. (2023). Exploring Chinese university English teachers’ language assessment literacy: A mixed-method study. Asia Pacific Journal of Education, 0(0), 1–19. https://doi.org/10.1080/02188791.2024.2354684

Published
2025-09-30
How to Cite
Ayuningtryas, V., & Emaliana, I. (2025). Are Pre-Service EFL Teachers Ready for AI-Assisted Assessment? The Role of Assessment Literacy in the Digital Era. PANYONARA: Journal of English Education, 7(2), 267-294. https://doi.org/10.19105/panyonara.v7i2.21416
Issue
Vol. 7 No. 2 (2025): PANYONARA: Journal of English Education
Section
Articles

The journal uses an Open Access policy under a Creative Commons Attribution-NonCommercial 4.0 International License. Authors who publish with this journal agree to the following terms:

  1. Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this journal.
  2. Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this journal.
  3. Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work.