Abstract
Although researchers have investigated the effects automated writing evaluation (AWE) and teacher corrective feedback have on second language (L2) writing performance, these studies have conflated two different variables—feedback source and perceptions of feedback source. Using a quasi-experimental design, two groups of second language English writers—perceived teacher feedback group (N = 88) and perceived automated feedback group (N = 73)—received feedback on four essays written over an 18-week academic semester. While both groups received AWE and a marked rubric as feedback, the perceived teacher feedback group was under the impression that the AWE was given by their teacher. Results showed the perceived automated feedback group outperformed the perceived teacher feedback group on essay 2 and essay 3; however, the reverse was found for essay 4. Furthermore, the perceived automated feedback group showed statistically significant differences in writing performance; they improved from essay 2 to essay 3 but regressed from essay 3 to essay 4. Perceptions of the feedback source may have caused these results, suggesting that it should be worthwhile to inform L2 student writers of the feedback source. Before providing feedback to L2 student writers, teachers can inform students on how their writing performance can benefit from AWE and afterwards the teachers can meet students’ needs by following up with the delivery of teacher feedback.
This is a preview of subscription content, log in via an institution to check access.
References
Bitchener, J., & Knoch, U. (2010). Raising the linguistic accuracy level of advanced L2 writers with written corrective feedback. Journal of Second Language Writing, 19(4), 207–217. https://doi.org/10.1016/j.jslw.2010.10.002
Burstein, J., Elliot, N., & Molloy, H. (2016). Informing automated writing evaluations using the lens of genre: Two studies. CALICO Journal, 33(1), 117–141. https://doi.org/10.1558/cj.v33il.26374
Chander, J. (2003). The efficacy of various kinds of error feedback for improvement in the accuracy and fluency of L2 student writing. Journal of Second Language Writing, 12(3), 267–296. https://doi.org/10.1016/S1060-3743(03)00038-9
Chen, C.-F. E., & Cheng, W.-Y. E. (2008). Beyond the design of automated writing evaluation: Pedagogical practices and perceived learning. Language Learning & Technology, 12(2), 94–112. http://llt.msu.edu/vol12num2/chencheng/
Chen, H. H. J., Cheng, H. W. S., & Yang, T. Y. C. (2017). Comparing grammar feedback provided by teachers with an automated writing evaluation system. English Teaching & Learning, 41(4), 99–131. https://doi.org/10.6330/ETL.2017.41.4.04
Chern, C.-L. (2002). English language teaching in Taiwan today. Asia Pacific Journal of Education, 22(2), 97–105. https://doi.org/10.1080/0218879020220209
Chittooran, M. M. (2015). Reading and writing for critical reflective thinking. New Directions for Teaching and Learning, 143, 79–95. https://doi.org/10.1002/tl.20137
Chou, C. P., & Caching, G. (2012). Course, curriculum, and textbooks. Taiwan education at the crossroad (pp. 89–98). Palgrave Macmillan.
Cohen, A. D., & Cavalcanti, M. C. (1990). Feedback on compositions: Teacher and student verbal reports. In B. Kroll (Ed.), Second Language writing: Research insights for the classroom (pp. 155–177). Cambridge University Press.
Cotos, E., & Pendar, N. (2016). Classification into rhetorical functions for AWE feedback. CALICO Journal, 33(1), 92–116. https://doi.org/10.1558/cj.v33il.27047
El Ebyary, K., & Windeatt, S. (2010). The impact of computer-based feedback on students’ written work. International Journal of English Studies, 10(2), 121–142.
Ellis, R., Sheen, Y., Murakami, M., & Takashima, H. (2008). The effects of focused and unfocused written corrective feedback in an English as a foreign language context. System, 36(3), 353–371. https://doi.org/10.6018/ijes/2010/2/119231
Ferris, D. (1999). The case for grammar correction in L2 writing classes: A response to Truscott. Journal of Second Language Writing, 8(1), 1–11. https://doi.org/10.1016/S1060-3743(99)80110-6
Ferris, D. (2004). The “grammar correction” debate in L2 writing: Where are we, and where do we go from here? (and what do we do in the mean time…?). Journal of Second Language Writing, 13(1), 49–62. https://doi.org/10.1016/j.jslw.2004.04.005
Ferris, D. (2010). Second language writing research and written corrective feedback in SLA. Studies in Second Language Acquisition, 32(2), 181–210. https://doi.org/10.1017/S0272263109990490
Ferris, D. R. (2014). Responding to student writing: Teachers’ philosophies and practices. Assessing Writing, 19, 6–23. https://doi.org/10.1016/j.asw.2013.09.004
Groza, A., & Szabo, R. (2015, November). Enacting textual entailment and ontologies for automated essay grading in chemical domain. In 2015 16th IEEE International Symposium on Computational Intelligence and Informatics (CINTI) (pp. 221–226). IEEE.
Gupta, S. D., Abdullah, F., Li, G., & Yang, X. (2019). Peer assessment in writing: A critical review of previous studies. Journal of Advances in Linguistics, 10, 1478–1487.
Han, Y., & Hyland, F. (2015). Exploring learner engagement with written corrective feedback in a Chinese tertiary EFL classroom. Journal of Second Language Writing, 30, 31–44. https://doi.org/10.1016/j.jslw.2015.08.002
Han, Y., & Hyland, F. (2019). Academic emotions in written corrective feedback situations. Journal of English for Academic Purposes, 38, 1–13. https://doi.org/10.1016/j.jeap.2018.12.003
Heift, T., & Schulze, M. (2007). Errors and intelligence in computer-assisted language learning: Parsers and pedagogues. Routledge.
Hockly, N. (2018). Automated writing evaluation. ELT Journal, 73(1), 82–88. https://doi.org/10.1093/elt/ccy044
Hyland, K., & Hyland, F. (2019). Contexts and issues in feedback on L2 writing: An introduction. In K. Hyland & F. Hyland (Eds.), Feedback in second language writing: Contexts and issues (2nd ed., pp. 1–20). Cambridge University Press.
Jiang, L., Yu, S., & Wang, C. (2020). Second language writing instructors’ feedback practice in response to automated writing evaluation: A sociocultural perspective. System, 93, 1–12. https://doi.org/10.1016/j.system.2020.102302
Kao, C.-W., & Reynolds, B. L. (2020). High school teacher feedback on word choice errors. Language Learning & Technology, 24(3), 19–29. http://hdl.handle.net/10125/44732.
Karimi, M., & Fotovatnia, Z. (2012). The effects of focused vs. unfocused written teacher correction on the grammatical accuracy of Iranian EFL undergraduates. Asian EFL Journal, 62, 42–59.
Kepner, C. G. (1991). An experiment in the relationship of types of written feedback to the development of second language writing skills. Modern Language Journal, 75, 305–313.
Lee, G., & Schallert, D. L. (2008). Constructing trust between teacher and students through feedback and revision cycles in an EFL writing classroom. Written Communication, 25(4), 506–537. https://doi.org/10.1177/0741088308322301
Lee, I. (2014). Revisiting teacher feedback in EFL writing from sociocultural perspectives. TESOL Quarterly, 48(1), 201–213. https://doi.org/10.1002/tesq.153
Li, J., Link, S., & Hegelheimer, V. (2015). Rethinking the role of automated writing evaluation (AWE) feedback in ESL writing instruction. Journal of Second Language Writing, 27, 1–18. https://doi.org/10.1016/j.jslw.2014.10.004
Link, S., Mehrzad, M., & Rahimi, M. (2020). Impact of automated writing evaluation on teacher feedback, student revision, and writing improvement. Computer Assisted Language Learning. https://doi.org/10.1080/09588221.2020.1743323
Liu, S., & Kunna, A. (2016). Investigating the application of automated writing evaluation to Chinese undergraduate English majors: A case study of WriteToLearn. CALICO Journal, 33(1), 71–91. https://doi.org/10.1558/cj.v33i1.26380
Loewen, S., Li, S., Fei, F., Thompson, A., Nakatsukasa, K., Ahn, S., & Chen, X. (2009). Second language learners’ beliefs about grammar instruction and error correction. The Modern Language Journal, 93(1), 91–104. https://doi.org/10.1111/j.1540-4781.2009.00830.x
Manap, M. R., Ramli, N. F., & Kassim, A. A. M. (2019). Web 2.0 automated essay scoring application and human ESL essay assessment: A comparison study. European Journal of English Language Teaching, 5(1), 146–162. https://doi.org/10.5281/zenodo.3461784
Nemati, M., Alavi, S. M., Mohebbi, H., & Masjedlou, A. P. (2017). Speaking out on behalf of the voiceless learners: Written corrective feedback for English language learners in Iran. Issues in Educational Research, 27(4), 822–847.
PaperRater. (2020). PaperRater. Retrieved from, https://www.paperrater.com
Paulus, T. (1999). The effects of peer and teacher feedback on student writing. Journal of Second Language Writing, 8, 265–289.
Phakiti, A. (2014). Experimental research methods in language learning. Bloomsbury Academic.
Polio, C., & Lee, J. (2019). Experimental studies in L2 classrooms. In J. Schwieter & A. Benati (Eds.), The Cambridge handbook of language learning (pp. 137–165). Cambridge University Press. https://doi.org/10.1017/9781108333603.007
Quixal, M., & Meurers, D. (2016). How can writing tasks be characterized in a way serving pedagogical goals and automatic analysis needs? CALICO Journal, 33(1), 19–48. https://doi.org/10.1558/cj.v33i1.26543
Reynolds, B. L., & Kao, C.-W. (2021). The effects of digital game-based instruction, teacher instruction, and direct focused written corrective feedback on the grammatical accuracy of English articles. Computer Assisted Language Learning, 34(4), 462–482. https://doi.org/10.1080/09588221.2019.1617747.
Roscoe, R., Wilson, J., Johnson, A., & Mayra, C. (2017). Presentation, expectations, and experience: Sources of student perceptions of automated writing evaluation. Computers in Human Behavior, 70, 207–221. https://doi.org/10.1016/j.chb.2016.12.076
Saricaoglu, A. (2019). The impact of automated feedback on L2 learners’ written causal explanations. ReCALL, 31(2), 189–203. https://doi.org/10.1017/S095834401800006X
Shermis, M. D., & Hammer, B. (2013). Handbook of automated essay evaluation: Current applications and new directions. Routledge.
Shintani, N., & Ellis, R. (2015). Does language analytical ability mediate the effect of written feedback on grammatical accuracy in second language writing? System, 49, 110–119. https://doi.org/10.1016/j.system.2015.01.006
Shintani, N., Ellis, R., & Suzuki, W. (2014). Effects of written feedback and revision on learners’ accuracy in using two English grammatical structures. Language Learning, 64(1), 103–131. https://doi.org/10.1111/lang.12029
Thong, N. V. (2017). Automated essay assessment: An evaluation on PaperRater’s reliability from practice. Creative Practices in Language Learning and Teaching, 5(1), 1–18.
Truscott, J. (1996). The case against grammar correction in L2 writing classes. Language Learning, 46(2), 327–369. https://doi.org/10.1111/j.1467-1770.1996.tb01238.x
Truscott, J. (1999). The case for “The case against grammar correction in L2 writing classes”: A response to Ferris. Journal of Second Language Writing, 8(1), 111–122. https://doi.org/10.1016/S1060-3743(99)80110-6
Truscott, J. (2004). Evidence and conjecture on the effects of correction: A response to Chandler. Journal of Second Language Writing, 13, 337–343. https://doi.org/10.1016/j.jslw.2004.05.002
Truscott, J., & Hsu, Y. P. (2008). Error correction, revision, and learning. Journal of Second Language Writing, 17(4), 292–305. https://doi.org/10.1016/j.jslw.2008.05.003
Van Beuningen, C. G., De Jong, N. H., & Kuiken, F. (2012). Evidence on the effectiveness of comprehensive error correction in second language writing. Language Learning, 62(1), 1–41. https://doi.org/10.1111/j.1467-9922.2011.00674.x
Williams, J., Brown, K., & Hood, S. (2012). Academic encounters: Life in society 3 (2nd ed.). Cambridge University Press.
Wilson, J., & Czik, A. (2016). Automated essay evaluation software in English language arts classrooms: Effects on teacher feedback, student motivation, and writing quality. Computers & Education, 100, 94–109. https://doi.org/10.1016/j.compedu.2016.05.004
Wu, J. R. W., & Wu, R. Y. F. (2010). Relating the GEPT reading comprehension tests to the CEFR. In W. Martyniuk (Ed.), Aligning tests with the CEFR, studies in language testing 33 (pp. 204–224). Cambridge University Press.
Yu, S., Jiang, L., & Zhou, N. (2020). Investigating what feedback practices contribute to students’ writing motivation and engagement in Chinese EFL context: A large scale study. Assessing Writing, 44, 1–15. https://doi.org/10.1016/j.asw.2020.100451
Yu, S., & Lee, I. (2016). Peer feedback in second language writing (2005–2014). Language Teaching, 49(4), 461–493. https://doi.org/10.1017/S0261444816000161
Zamel, V. (1985). Responding to student writing. TESOL Quarterly, 19(1), 79–102. https://doi.org/10.2307/3586773
Zhang, Z. (2020). Engaging with automated writing evaluation (AWE) feedback on L2 writing: Student perceptions and revisions. Assessing Writing, 43, 1–14. https://doi.org/10.1016/j.asw.2019.100439
Funding
This work was supported by the University of Macau, Grant Number MYRG2018-00008-FED and the Ministry of Science and Technology, Taiwan, Grant Number MOST 110-2511-H-263-001.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Reynolds, B.L., Kao, CW. & Huang, Yy. Investigating the Effects of Perceived Feedback Source on Second Language Writing Performance: A Quasi-Experimental Study. Asia-Pacific Edu Res 30, 585–595 (2021). https://doi.org/10.1007/s40299-021-00597-3
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s40299-021-00597-3