Skip to main content

Advertisement

SpringerLink
Log in

The development of a situational judgement test of personal attributes for quality teaching in rural and remote Australia

  • Published:
The Australian Educational Researcher Aims and scope Submit manuscript

Abstract

Education authorities in Australia are calling for valid tools to help assess prospective teachers’ non-academic attributes, with a particular need for identifying those attributes necessary for effective teaching in specific contexts. With the New South Wales (NSW) Department of Education, we aimed to develop a scenario-based tool to help assess the non-academic attributes necessary for beginning teachers working in rural and remote settings. Using a situational judgement test (SJT) methodology, we worked with experienced teachers (n = 19) to develop scenario-based items, which were then reviewed by school principals (n = 13). The teachers also adapted items previously developed and piloted in the UK. Next, prospective NSW teachers (n = 99) tried 32 new and adapted items targeting four clusters of attributes: empathy and communication, resilience and adaptability, organisation and planning, and culture and context. Item quality analyses revealed 22 acceptable items. We conclude by suggesting SJTs for promoting non-academic growth of prospective and beginning teachers.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

Notes

  1. Please refer to Patterson et al.’s (2013) work for the formula as it is beyond the scope of this article to elaborate on the calculations.

  2. While inconclusive, a larger sample of participants may help determine the validity of the scenario-based items as a whole in relation to a range of related constructs. Since SJT research, overall, has struggled with conventional methods for assessing validity (Sorrel et al. 2016), additional methods in addition to use with a larger sample will be explored in future research.

  3. The low reliability may be due in part to the low sample size and higher number of primary trained participants (since the items were specific to secondary school settings). In addition, 10 items were revealed as being of limited quality (see Table 4). By excluding lower quality items, the reliability (as indicated by Cronbach’s alpha) increased from 0.16 to 0.42. Previous SJT studies have averaged an internal consistency of 0.46 (see Sorrel et al. 2016 for a discussion).

References

  • Austin, J. (2010). Remote ready. Remote Health Recruitment, Northern Territory Government. Charles Darwin University, Australia. Retrieved from http://www.remoterecruitment.nt.gov.au/docs/remote_ready.pdf.

  • Australian Bureau of Statistics. (2017). Census 2016 QuickStats: Aboriginal and/or Torres Strait Islander people. Belconnen, ACT: Commonwealth of Australia. Retrieved from http://www.abs.gov.au/.

  • Australian Institute for Teaching and School Leadership (AITSL). (2011). Accreditation of initial teacher education programs in Australia: Standards and procedures. Melbourne, VIC: Ministerial Council for Education, Early Childhood Development and Youth Affairs.

    Google Scholar 

  • Australian Institute for Teaching and School Leadership (AITSL). (2015). Action now: Selection of entrants into initial teacher education—guidelines. Melbourne, VIC: Australian Institute for Teaching and School Leadership.

    Google Scholar 

  • Bahr, N., & Mellor, S. (2016). Australian education review: Building quality in teaching and teacher education. Camberwell, VIC: Australian Council for Educational Research.

    Google Scholar 

  • Baxter, J., Gray, M., & Hayes, A. (2011). Families in regional, rural and remote Australia (Fact Sheet). Australia Institute of Families Studies, Melbourne, VIC: Commonwealth of Australia. Retrieved from https://aifs.gov.au/publications/families-regional-rural-and-remote-australia.

  • Beltman, S., Mansfield, C., & Price, A. (2011). Thriving not just surviving: A review of research on teacher resilience. Educational Research Review, 6(3), 185–207. https://doi.org/10.1016/j.edurev.2011.09.001.

    Article  Google Scholar 

  • Buehl, M. M., & Fives, H. (2009). Exploring teachers’ beliefs about teaching knowledge: Where does it come from? Does it change? The Journal of Experimental Education, 77(4), 367–407.

    Article  Google Scholar 

  • Caldwell, B., & Sutton, D. (2010). Review of teacher education and induction. First report—full report. Education Queensland http://education.qld.gov.au/students/higher-education/resources/review-teachereducation-school-induction-first-full-report.pdf.

  • Cox, C. B., Barron, L. G., Davis, W., & de la Garza, B. (2017). Using situational judgment tests (SJTs) in training. Personnel Review, 46(1), 36–45. https://doi.org/10.1108/PR-05-2015-0137.

    Article  Google Scholar 

  • Day, C., & Gu, Q. (2014). Resilient teachers, resilient schools. New York, NY: Routledge Taylor Francis.

    Google Scholar 

  • Frenzel, A. C., Becker-Kurz, B., Pekrun, R., & Goetz, T. (2015). Teaching this class drives me nuts!—Examining the person and context specificity of teacher emotions. PLoS ONE, 10(6), e0129630. https://doi.org/10.1371/journal.pone.0129630.

    Article  Google Scholar 

  • Goldhaber, D., Grout, C., & Huntington-Klein, N. (2014). Screen twice, cut once: Assessing the predictive validity of teacher selection tools. (CEDR Working Paper No. 9). Seattle, WA: Center for Education Data and Research. http://www.cedr.us/papers/working/CEDR%20WP%202014-9.1.pdf.

  • Gore, J., Barron, R. J., Holmes, K., & Smith, M. (2016). Who says we are not attracting the best and brightest? Teacher selection and the aspirations of Australian school students. The Australian Educational Researcher, 43(5), 527–549. https://doi.org/10.1007/s13384-016-0221-8.

    Article  Google Scholar 

  • Gosling, S. D., Rentfrow, P. J., & Swann, W. B., Jr. (2003). A very brief measure of the big-five personality domains. Journal of Research in Personality, 37, 504–528. https://doi.org/10.1016/S0092-6566(03)00046-1.

    Article  Google Scholar 

  • Hamre, B. K., Pianta, R. C., Downer, J. T., DeCoster, J., Mashburn, A. J., Jones, S. M., et al. (2013). Teaching through interactions: Testing a developmental framework of teacher effectiveness in over 4000 classrooms. The Elementary School Journal, 113(4), 461–487.

    Article  Google Scholar 

  • Holland, J. L. (1997). Making vocational choices: A theory of vocational personalities and work environments (3rd ed.). Odessa, FL: Psychological Assessment Resources.

    Google Scholar 

  • Huberman, M. (1989). The professional life cycle of teachers. Teachers College Record, 91, 31–57.

    Google Scholar 

  • Ingersoll, R. M., & Strong, M. (2011). The impact of induction and mentoring programs for beginning teachers: A critical review of the research. Review of Educational Research, 81, 201–233. https://doi.org/10.3102/0034654311403323.

    Article  Google Scholar 

  • King, R. B., & McInerney, D. M. (2016). Culture and motivation. In K. R. Wentzel & D. B. Miele (Eds.), Handbook of motivation at school (pp. 275–310). New York: Routledge.

    Google Scholar 

  • Klassen, R. M., Durksen, T. L., Kim, L., Patterson, F., Rowett, E., Warwick, J., et al. (2017a). Developing a proof-of-concept selection test for entry into primary teacher education programs. International Journal of Assessment Tools in Education, 4(2), 96–114. https://doi.org/10.21449/ijate.275772.

    Article  Google Scholar 

  • Klassen, R. M., Durksen, T. L., Patterson, F., & Rowett, E. (2017b). Filtering functions of assessment for selection into initial teacher education programs. In J. Clandinin & J. Husu (Eds.), The Sage handbook of research on teacher education (Vol. 1, pp. 893–909). New York: Sage Publications Ltd.

    Chapter  Google Scholar 

  • Klassen, R. M., Durksen, T. L., Rowett, E., & Patterson, F. (2014). Applicant reactions to a situational judgment test used for selection into initial teacher training. International Journal of Educational Psychology, 3, 104–125. Retrieved from http://www.hipatiapress.com/hpjournals/index.php/ijep.

  • Klassen, R. M., & Tze, V. M. C. (2014). Teachers’ self-efficacy, personality, and teaching effectiveness: A meta-analysis. Educational Research Review, 12, 59–76.

    Article  Google Scholar 

  • Klassen, R. M., Tze, V. M. C., Betts, S. M., & Gordon, K. A. (2011). Teacher efficacy research 1998–2009: Signs of progress or unfulfilled promise? Educational Psychology Review, 23, 21–43. https://doi.org/10.1007/s10648-010-9141-8.

    Article  Google Scholar 

  • Klassen, R. M., Yerdelen, S., & Durksen, T. L. (2013). Measuring teacher engagement: The development of the engaged teacher scale (ETS). Frontline Learning Research, 1, 33–52. Retrieved from http://journals.sfu.ca/flr/index.php/journal/article/view/44/37.

  • Kutsyuruba, B., Godden, L., & Tregunna, L. (2014). Curbing early-career teacher attrition: A pan-Canadian document analysis of teacher induction and mentorship programs. Canadian Journal of Educational Administration and Policy, 161, 1–42.

    Google Scholar 

  • Lyons, T. (2006). Science, ICT and mathematics education in rural and regional Australia, SiMERR National Research Centre [Report]. http://apo.org.au/node/2732.

  • Martin, A. J., & Dowson, M. (2009). Interpersonal relationships, motivation, engagement, and achievement: Yields for theory, current issues, and practice. Review of Educational Research, 79, 327–365.

    Article  Google Scholar 

  • Mayer, D., Allard, A., Bates, R., Dixon, M., Doeke, B., & Kline, J., et al. (2015). Studying the effectiveness of teacher education (Final Report). Geelong, VIC: Deakin University. http://www.setearc.com.au/publication-reports/.

  • MCEETYA. (2001). National framework for rural and remote education in Australia. Journal of Research in Rural Education, 17(2), 112–119.

    Google Scholar 

  • McNeilage, A. (2014, January 21). Adrian Piccoli says teacher selection criteria should be more strict. The Sydney Morning Herald. http://www.smh.com.au/national/education/adrian-piccoli-says-teacher-selection-criteria-should-be-more-strict-20140120-314vn.html.

  • Meng, L., & Muñoz, M. (2016). Teachers’ perceptions of effective teaching: A comparative study of elementary school teachers from China and the USA. Educational Assessment, Evaluation and Accountability. https://doi.org/10.1007/s11092-015-9230-9.

    Article  Google Scholar 

  • Motowidlo, S. J., & Beier, M. E. (2010). Differentiating specific job knowledge from implicit trait policies in procedural knowledge measured by a situational judgment test. Journal of Applied Psychology, 95(2), 321.

    Article  Google Scholar 

  • NSW Council of Deans of Education. (2016, July). Position paper: Selection methods into initial teacher education programs [Draft v2]. NSWCDE and BOSTES Joint Forum: Assuring Graduate Quality through the New Accreditation Standards. Received at University of Technology, Sydney.

  • NSW Council of Deans of Education. (2017, January). Assessing and confirming suitability for teaching: A selection framework for NSW teacher education providers [Brief Report]. Sydney, NSW.

  • NSW Department of Education and Communities (NSWDEC). (2013, November). Rural and remote education: A blueprint for action. NSW Department of Education and Communities. www.dec.nsw.gov.au/.

  • Patterson, F., Ashworth, V., & Good, D. (2013). Situational judgement tests: A guide for applicants to the UK foundation programme. London: Medical School Council.

    Google Scholar 

  • Patterson, F., Zibarras, L., & Ashworth, V. (2015). Situational judgment tests in medical education and training: Research, theory and practice: AMEE Guide No.100. Medical Teacher, 00(00), 1–15. https://doi.org/10.3109/0142159X.2015.1072619.

    Article  Google Scholar 

  • Reid, J.-A., Green, B., Cooper, M., Hastings, W., Lock, G., & White, S. (2010). Regenerating rural social space? Teacher education for rural—Regional sustainability. Australian Journal of Education, 54(3), 262–276.

    Article  Google Scholar 

  • Rice, S., Richardson, P. W., & Watt, H. M. G. (2017). Hard-to-staff Australian schools: How can we ensure that all students have access to quality teachers? In T. Bentley and G. C. Savage (Eds.), Educating Australia: Challenges for the decade ahead (Chapter 17). Melbourne, VIC: Melbourne University Publishing (Kindle Edition).

  • Ryan, A. M., & Ployhart, R. E. (2014). A century of selection. Annual Review of Psychology, 65, 20.1–20.25.

    Article  Google Scholar 

  • Sautelle, E., Bowles, T., Hattie, J., & Arifin, D. (2015). Personality, resilience, self-regulation and cognitive ability relevant to teacher selection. Australian Journal of Teacher Education, 40(40), 1–19. http://doi.org/10.14221/ajte.2015v40n4.4.

  • Schultheiss, O., & Brunstein, J. (2010). Implicit motives. UK: Oxford University Press.

    Book  Google Scholar 

  • Sorrel, M. A., Olea, J., Abad, F. J., de la Torre, J., Aguado, D., & Lievens, F. (2016). Validity and reliability of situational judgement test scores. Organizational Research Methods, 19(3), 506–532. https://doi.org/10.1177/1094428116630065.

    Article  Google Scholar 

  • Stemig, M. S., Sackett, P. R., & Lievens, F. (2015). Effects of organizationally endorsed coaching on performance and validity of situational judgment tests. International Journal of Selection and Assessment, 23(2), 174–181.

    Article  Google Scholar 

  • Stemler, S. E., Aggarwal, V., & Nithyanand, S. (2016). Knowing what NOT to do is a critical job skill: Evidence from 10 different scoring methods. International Journal of Selection and Assessment, 24(3), 229–245.

    Article  Google Scholar 

  • Stenberg, K., Karlsson, L., Pitkaniemi, H., & Maaranen, K. (2014). Beginning student teachers’ teacher identities based on their practical theories. European Journal of Teacher Education, 37(2), 204–219.

    Article  Google Scholar 

  • TEMAG: Teacher Education Ministerial Advisory Group. (2014). Action now: Classroom ready teachers. Canberra: Australian Government Department of Education. http://www.studentsfirst.gov.au.

  • Tripp, D. (1994). Teachers’ lives, critical incidents, and professional practice. International Journal of Qualitative Studies in Education, 7(1), 65–76. https://doi.org/10.1080/0951839940070105.

    Article  Google Scholar 

  • Tschannen-Moran, M., & Woolfolk Hoy, A. (2001). Teacher efficacy: Capturing an elusive construct. Teaching and Teacher Education, 17, 783–805.

    Article  Google Scholar 

  • Whelpley, C. E. (2014). How to score situational judgment tests: A theoretical approach and empirical test [Unpublished Doctoral Dissertation]. Virginia Commonwealth University, USA.

  • Whetzel, D. L., & McDaniel, M. A. (2016). Are Situational Judgment Tests better assessments of personality than traditional personality tests in high-stakes testing? In U. Kumar (Ed.), The Wiley Handbook of Personality Assessment (pp. 205–214). West Sussex: Wiley.

    Chapter  Google Scholar 

Download references

Acknowledgements

We gratefully acknowledge the assistance provided by Lisa Kim (University of York, UK) and the consistent dedication and commitment shown by our team of collaborators at the New South Wales (NSW) Department of Education. In particular, we are grateful for the help provided by Mark Anderson, Ian Quintos, John Wilson, and Veronica Willmott of the Human Resources Directorate. We also express our utmost thanks to the NSW teachers and principals who shared their insights and expertise and to the pilot study participants for their time and feedback.

Author information

Authors and Affiliations

  1. School of Education, Faculty of Arts and Social Sciences, UNSW Sydney, 131 John Goodsell Bldg, Sydney, NSW, 2052, Australia

    Tracy L. Durksen

  2. University of York, York, UK

    Robert M. Klassen

Authors
  1. Tracy L. Durksen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Robert M. Klassen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tracy L. Durksen.

Appendix: Example of a situational judgement test item for teachers

Appendix: Example of a situational judgement test item for teachers

You are teaching a lesson and have asked the students to individually complete an exercise that requires them to write down their responses. You have explained the exercise to the students and answered all of the questions that they have asked. As the students begin writing, one student, Ruby, starts to throw paper around and is clearly distracting the students sitting nearby. You know from previous incidents that Ruby often becomes frustrated when she does not understand how to complete activities, and that she often displays her frustration by being disruptive.


Choose the three most appropriate actions to take in this situation (alternatively, Rank the items in the most appropriate order)

  • Send Ruby out the class if she continues to be disruptive

  • Ask Ruby if she understands what the activity requires her to do

  • Check in 5 min to see if Ruby has made progress with the exercise

  • Tell Ruby that you are disappointed in her behaviour

  • Ask Ruby’s classmate to discreetly provide help

  • Stop the exercise and discuss the classroom behaviour plan with the whole class

  • etc. (eight total response options)

Note This is an example only, and is adapted from an item in a SJT for primary teachers (Klassen et al. 2017, p. 905).

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Durksen, T.L., Klassen, R.M. The development of a situational judgement test of personal attributes for quality teaching in rural and remote Australia. Aust. Educ. Res. 45, 255–276 (2018). https://doi.org/10.1007/s13384-017-0248-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13384-017-0248-5

Keywords

Search

Navigation