Abstract
In this study, researchers designed and implemented a 1-year professional development (PD) program that focused on supporting teachers in evaluating and selecting digital learning contents. Participants in this investigation included 109 teachers who consented to the study amongst a total of 171 teachers from five school districts across central Ohio. In addition to their participation in the PD program, they completed surveys, interviews, and self-reflections in this mixed-method study. The results revealed that teachers’ perceived TPACK increased over time throughout the PD program, suggesting that training teachers to evaluate digital contents can be an effective PD model to improve teachers’ capacity in learning technology integration. The PD program was especially effective for teachers with less prior experience in technology integration or related training. Mathematics teachers, in comparison to teachers from other disciplines, began with low TPACK; however, these initial differences gradually diminished over the course of the PD program. In terms of their motivation in digital content evaluation, teachers’ expectancy for success increased significantly while their task values remained medium high. The qualitative analyses provided additional insights and revealed design suggestions for success in future PDs.
Similar content being viewed by others
References
Abbitt, J. T. (2011). Measuring technological pedagogical content knowledge in preservice teacher education: A review of current methods and instruments. Journal of Research on Technology in Education, 43(4), 281–300.
Adelman, N., Donnelly, M. B., Dove, T., Tiffany-Morales, J., Wayne, A., & Zucker, A. (2002). The integrated studies of educational technology: Professional development and teachers’ use of technology. Arlington, VA: SRI International.
Anderson, L. W., & Krathwohl, D. R. (Eds.). (2001). A taxonomy for learning, teaching, and assessing: A revision of Bloom’s taxonomy of educational objectives. New York: Longman.
Angeli, C., & Valanides, N. (2013). Using educational data mining methods to assess field-dependent and field-independent learners’ complex problem solving. Educational Technology Research and Development, 61(3), 521–548.
Bandura, A. (1977). Self-efficacy: toward a unifying theory of behavioral change. Psychological Review, 84(2), 191–215.
Bloom, B. S., Engelhart, M. D., Furst, E. J., Hill, W. H., & Krathwohl, D. R. (1956). Taxonomy of educational objectives, handbook I: Cognitive domain. New York: Longman Publishers.
Bonk, C., & Graham, C. (Eds.). (2005). The handbook of blended learning: Global perspectives, local designs. New York: Pfeiffer.
Brinkerhoff, J. (2006). Effects of a long-duration, professional development academy on technology skills, computer self-efficacy, and technology integration beliefs and practices. Journal of Research on Technology in Education, 39(1), 22–43. doi:10.1080/15391523.2006.10782471.
Brophy, J. (2008). Developing students’ appreciation for what is taught in school. Educational Psychologist, 43(3), 132–141.
Buchanan, B. (1974). Building organizational commitment: The socialization of managers in work organizations. Administrative science quarterly, 19(4), 533–546.
Chai, C. S., Koh, J. H. L., & Tsai, C. C. (2010). Facilitating preservice teachers’ development of technological, pedagogical, and content knowledge (TPACK). Journal of Educational Technology & Society, 13(4), 63–73.
Chow, A., Eccles, J. S., & Salmela-Aro, K. (2012). Task value profiles across subjects and aspirations to physical and IT-related sciences in the United States and Finland. Developmental Psychology, 48(6), 1612–1628.
Cohen, J. (1988). Statistical power analysis for the behavioral sciences. Hillsdale: LEA.
Cosmah, M., & Saine, P. (2013). Targeting digital technologies in common core standards: A framework for professional development. New England Reading Association Journal, 48(2), 81–86.
Creswell, J. W., & Clark, V. L. P. (2007). Designing and conducting mixed methods research. Thousand Oaks, CA: Sage.
Creswell, J. W., Plano Clark, V. L., Guttmann, M. L., & Hanson, E. E. (2003). Advanced mixed methods research design. In A. Tashakkori & C. Teddlie (Eds.), Handbook of mixed methods in social and behavioral research (pp. 209–240). Thousand Oaks, CA: Sage.
Dean, D., & Kuhn, D. (2003). Metacognition and critical thinking. Retrieved from http://eric.ed.gov/?id=ED477930.
Deanna, K. (1999). A developmental model of critical thinking. Educational Researcher, 28(2), 16–46.
DeBacker, T.K., Miller, R.B., Walker, C.O., & Man Sell, R., (2004, September). Perceptions of classroom climate, student motivation, and achievement: Changes and interrelationships across an academic year. Paper presented at the Ninth International conference on Motivation. Lisbon, Portugal.
DeSantis, J. (2013). Exploring the effects of professional development for the interactive whiteboard on teachers’ technology self-efficacy. Journal of Information Technology Education: Research, 12, 343–362.
Dewey, J. (1933). How we think: A restatement of the relation of reflective thinking to the educative process. Lexington, MA: Heath.
Doering, A., Veletsianos, G., Scharber, C., & Miller, C. (2009). Using the technological, pedagogical, and content knowledge framework to design online learning environments and professional development. Journal of Educational Computing Research, 41(3), 319–346.
Doherty, I. (2011). Evaluating the impact of educational technology professional development upon adoption of Web 2.0 tools in teaching. Australasian Journal of Educational Technology, 27(3), 381–396.
Duran, M., Brunvand, S., Ellsworth, J., & Şendağ, S. (2011). Impact of research-based professional development: Investigation of inservice teacher learning and practice in Wiki integration. Journal of Research on Technology in Education, 44(4), 313–334.
Eccles, J. (1983). Expectancies, values and academic behaviors. In J. T. Spence (Ed.), Achievement and achievement motives (pp. 75–146). San Francisco: Freeman.
Eccles, J. S., O’Neill, S. A., & Wigfield, A. (2005). Ability self-perceptions and subjective task values in adolescents and children. In K. A. Moore & L. H. Lippman (Eds.), What do children need to flourish? Conceptualizing and measuring indicators of positive development (pp. 237–249). New York: Springer.
Eccles, J. S., & Wigfield, A. (1995). In the mind of the achiever: The structure of adolescents’ academic achievement related-beliefs and self-perceptions. Personality and Social Psychology Bulletin, 21, 215–225.
Eccles, J. S., & Wigfield, A. (2002). Motivational beliefs, values, and goals. Annual Review of Psychology, 53(1), 109–132.
Ennis, R. H. (1987). A taxonomy of critical thinking dispositions and abilities. In J. B. Baron & R. J. Sternberg (Eds.), Teaching thinking skills (pp. 9–26). New York: Freeman.
Ertmer, P. A. (2005). Teacher pedagogical beliefs: The final frontier in our quest for technology integration? Educational Technology Research and Development, 53(4), 25–39.
Ertmer, P. A., & Ottenbreit-Leftwich, A. T. (2010). Teacher technology change: How knowledge, confidence, beliefs, and culture intersect. Journal of research on Technology in Education, 42(3), 255–284.
Ertmer, P. A., Ottenbreit-Leftwich, A. T., Sadik, O., Sendurur, E., & Sendurur, P. (2012). Teacher beliefs and technology integration practices: A critical relationship. Computers & Education, 59(2), 423–435.
Ertmer, P. A., Paul, A., Molly, L., Eva, R., & Denise, W. (1999). Examining teachers’ beliefs about the role of technology in the elementary classroom. Journal of Research on Computing in Education, 32(1), 54–72.
Facione, N. E. (1990). The Delphi report. Committee on pre-college philosophy. American Philosophical Association.
Fiorella, L., Vogel-Walcutt, J. J., & Schatz, S. (2012). Applying the modality principle to real-time feedback and the acquisition of higher-order cognitive skills. Educational Technology Research and Development, 60(2), 223–238.
Fleck, J. (2012). Blended learning and learning communities: opportunities and challenges. Journal of Management Development, 31(4), 398–411.
Fraenkel, J., Wallen, N., & Hyun, H. H. (2012). How to design and evaluate research in education (8th ed.). Boston: McGraw Hill.
Gallant, D.J., Hill, E., Luthy, N., & Shiverdecker, T. (2015). Using a modified Delphi survey and focus group interviews to determine K-12 teachers’ and principals’ information needs to select digital content. Paper presented at the annual conference of American Evaluation Association, Chicago, IL.
Garet, M. S., Porter, A. C., Desimone, L., Birman, B. F., & Yoon, K. S. (2001). What makes professional development effective? Results from a national sample of teachers. American Educational Research Journal, 38(4), 915–945.
Goodson, I. F., & Mangan, J. M. (1995). Subject cultures and the introduction of classroom computers. British Educational Research Journal, 21(5), 613–628.
Greene, J. C. (2007). Mixed methods in social inquiry. San Francisco, CA: Jossey-Bass.
Hartsell, T., Herron, S., Fang, H., & Rathod, A. (2009). Effectiveness of professional development in teaching mathematics and technology applications. Journal of Educational Technology Development and Exchange, 2(1), 53–64.
Hew, K. F., & Brush, T. (2007). Integrating technology into K-12 teaching and learning: Current knowledge gaps and recommendations for future research. Educational Technology Research and Development, 55(3), 223–252.
Higgins, T. E., & Spitulnik, M. W. (2008). Supporting teachers’ use of technology in science instruction through professional development: a literature review. Journal of Science Education and Technology, 17(5), 511–521.
Holbein, M. F., & Jackson, K. (1999). Study groups and electronic portfolios: A professional development school inservice project. Journal of Technology and teacher Education, 7(3), 205–218.
Holland, P. E. (2001). Professional development in technology: Catalyst for school reform. Journal of Technology in Teacher Education, 9(2), 245–267.
Howard, S. K., Chan, A., Mozejko, A., & Caputi, P. (2015). Technology practices: Confirmatory factor analysis and exploration of teachers’ technology integration in subject areas. Computers & Education, 90, 24–35.
Howard, S. K., & Maton, K. (2011). Theorising knowledge practices: A missing piece of the educational technology puzzle. Research in Learning Technology, 19(3), 191–206.
Inan, F. A., & Lowther, D. L. (2010). Factors affecting technology integration in K-12 classrooms: A path model. Educational Technology Research and Development, 58(2), 137–154.
Jimoyiannis, A. (2010). Designing and implementing an integrated technological pedagogical science knowledge framework for science teachers professional development. Computers & Education, 55(3), 1259–1269.
John, P. D., & La Velle, L. B. (2004). Devices and desires: Subject subcultures, pedagogical identity and the challenge of information and communications technology. Technology, Pedagogy and Education, 13(3), 307–326.
Keller, J. B., Bonk, C. J., & Hew, K. (2005). The TICKIT to teacher learning: Designing professional development according to situative principles. Journal of Educational Computing Research, 32(4), 329–340.
Kereluik, K., Casperson, G., & Akcaoglu, M. (2010). Coding pre-service teacher lesson plans for TPACK. In D. Gibson & B. Dodge (Eds.), Proceedings of Society for Information Technology and Teacher Education International Conference 2010 (pp. 3889–3891). Chesapeake, VA: AACE.
Kleickmann, T., Tröbst, S., Jonen, A., Vehmeyer, J., & Möller, K. (2016). The effects of expert scaffolding in elementary science professional development on teachers’ beliefs and motivations, instructional practices, and student achievement. Journal of Educational Psychology, 108(1), 21–42. doi:10.1037/edu0000041.
Koehler, M., & Mishra, P. (2009). What is technological pedagogical content knowledge (TPACK)? Contemporary Issues in Technology and Teacher Education, 9(1), 60–70.
Koehler, M. J., Mishra, P., Kereluik, K., Shin, T. S., & Graham, C. R. (2014). The technological pedagogical content knowledge framework. In J. M. Spector, M. D. Merrill, J. Elen, & M. J. Bishop (Eds.), Handbook of research on educational communications and technology (pp. 101–111). New York, NY: Springer.
Koehler, M. J., Mishra, P., & Yahya, K. (2007). Tracing the development of teacher knowledge in a design seminar: Integrating content, pedagogy and technology. Computers & Education, 49(3), 740–762.
Kopcha, T. J. (2012). Teachers’ perceptions of the barriers to technology integration and practices with technology under situated professional development. Computers & Education, 59(4), 1109–1121.
Labin, S. N., Duffy, J. L., Meyers, D. C., Wandersman, A., & Lesesne, C. A. (2012). A research synthesis of the evaluation capacity building literature. American Journal of Evaluation, 33, 307–338.
Lawless, K. A., & Pellegrino, J. W. (2007). Professional development in integrating technology into teaching and learning: Knowns, unknowns, and ways to pursue better questions and answers. Review of Educational Research, 77(4), 575–614.
Martin, W., Strother, S., Beglau, M., Bates, L., Reitzes, T., & McMillan Culp, K. (2010). Connecting instructional technology professional development to teacher and student outcomes. Journal of Research on Technology in Education, 43(1), 53–74.
Matzen, N. J., & Edmunds, J. A. (2007). Technology as a catalyst for change: The role of professional development. Journal of Research on Technology in Education, 39(4), 417–430.
Mayer, R. E. (2002). Rote versus meaningful learning. Theory Into Practice, 41, 226–232.
Meece, J. L., Wigfield, A., & Eccles, J. S. (1990). Predictors of math anxiety and its influence on young adolescents’ course enrollment intentions and performance in mathematics. Journal of Educational Psychology, 82(1), 60–70.
Midgley, C., & Urdan, T. (2001). Academic self-handicapping and achievement goals: A further examination. Contemporary Educational Psychology, 26(1), 61–75.
Milbrath, Y. C. L., & Kinzie, M. B. (2000). Computer technology training for prospective teachers: Computer attitudes and perceived self-efficacy. Journal of Technology and Teacher Education, 8(4), 373–396.
Mishra, P., & Koehler, M. (2006). Technological pedagogical content knowledge: A framework for teacher knowledge. The Teachers College Record, 108(6), 1017–1054.
Mouza, C. (2006). Linking professional development to teacher learning and practice: A multi-case study analysis of urban teachers. Journal of Educational Computing Research, 34(4), 405–440.
Mouza, C. (2009). Does research-based professional development make a difference? A longitudinal investigation of teacher learning in technology integration. The Teachers College Record, 111(5), 1195–1241.
Mouza, C. (2011). Promoting urban teachers’ understanding of technology, content, and pedagogy in the context of case development. Journal of Research on Technology in Education, 44(1), 1–29.
Mouza, C., Karchmer-Klein, R., Nandakumar, R., Ozden, S. Y., & Hu, L. (2014). Investigating the impact of an integrated approach to the development of preservice teachers’ technological pedagogical content knowledge (TPACK). Computers & Education, 71, 206–221.
Mumford, M. D., Mobley, M. I., Reiter-Palmon, R., Uhlman, C. E., & Doares, L. M. (1991). Process analytic models of creative capacities. Creativity Research Journal, 4(2), 91–122.
Niess, M. L., van Zee, E. H., & Gillow-Wiles, H. (2010). Knowledge growth in teaching mathematics/science with spreadsheets: Moving PCK to TPACK through online professional development. Journal of Digital Learning in Teacher Education, 27(2), 42–52.
Owston, R., Wideman, H., Murphy, J., & Lupshenyuk, D. (2008). Blended teacher professional development: A synthesis of three program evaluations. The Internet and Higher Education, 11(3), 201–210.
Piaget, J. (1985). The equilibration of cognitive structures. Chicago: University of Chicago Press.
Pintrich, P. R. (2000). Multiple goals, multiple pathways: The role of goal orientation in learning and achievement. Journal of Educational Psychology, 92(3), 544–555.
Polly, D. (2011). Examining teachers’ enactment of technological pedagogical and content knowledge (TPACK) in their mathematics teaching after technology integration professional development. Journal of Computers in Mathematics and Science Teaching, 30(1), 37–59.
Rienties, B., Brouwer, N., & Lygo-Baker, S. (2013). The effects of online professional development on higher education teachers’ beliefs and intentions towards learning facilitation and technology. Teaching and Teacher Education, 29, 122–131.
Schieb, L. J., & Karabenick, S. A. (2011). Teacher Motivation and Professional Development: A Guide to Resources. Math and Science Partnership—Motivation Assessment Program, University of Michigan, Ann Arbor, MI.
Schmidt, D. A., Baran, E., Thompson, A. D., Mishra, P., Koehler, M. J., & Shin, T. S. (2009). Technological pedagogical content knowledge (TPACK) the development and validation of an assessment instrument for preservice teachers. Journal of Research on Technology in Education, 42(2), 123–149.
Scott, P., & Mouza, C. (2007). The impact of professional development on teacher learning, practice and leadership skills: A study on the integration of technology in the teaching of writing. Journal of Educational Computing Research, 37(3), 229–266.
Simpkins, S. D., Fredricks, J. A., & Eccles, J. S. (2012). Charting the Eccles’ expectancy-value model from mothers’ beliefs in childhood to youth’s activities in adolescence. Developmental Psychology, 48, 1019–1032.
Sinnott, J. D. (1989). A model for solution of ill-structured problems: Implications for everyday and abstract problem solving. In J. D. Sinnott (Ed.), Everyday problem solving: Theory and applications (pp. 72–99). New York: Praeger.
Skoretz, Y., & Childress, R. (2013). An evaluation of a school-based professional development program on teachers’ efficacy for technology integration: Findings from an initial study. Journal of Technology and Teacher Education, 21(4), 461–484.
Smith, S. M. (2001). The four sources of influence on computer self-efficacy. Delta Pi Epsilon Journal, 43(1), 27–39.
Sparks, (2002). Designing powerful professional development for teachers and principals. Oxford, OH: National Staff Development Council.
Staker, H., Chan, E., Clayton, M., Hernandez, A., Horn, M.B., & Mackey, K. (2011). The rise of K–12 blended learning: Profiles of emerging models. Innosight Institute report. Retrieved from http://www.innosightinstitute.org/innosight/wp-content/uploads/2011/01/The-Rise-of-K-12-Blended-Learning.pdf.
Stipek, D. J., & Ryan, R. H. (1997). Economically disadvantaged preschoolers: Ready to learn but further to go. Developmental Psychology, 33(4), 711–723.
Suharwoto, G. (2006). Developing and implementing a technology pedagogical content knowledge (TPCK) for teaching mathematics with technology. In C. Crawford, D. Willis, R. Carlsen, I. Gibson, K. McFerrin, J. Price, & R. Weber (Eds.), Proceedings of society for information technology and teacher education international conference 2006 (pp. 3824–3828). Chesapeake, VA: AACE.
Tabachnick, B. G., & Fidell, L. S. (2013). Using multivariate statistics (6th ed.). Boston, MA: Pearson.
Tamim, R. M., Bernard, R. M., Borokhovski, E., Abrami, P. C., & Schmid, R. F. (2011). What forty years of research says about the impact of technology on learning a second-order meta-analysis and validation study. Review of Educational Research, 81(1), 4–28.
Trautmann, N. M., & MaKinster, J. G. (2010). Flexibly adaptive professional development in support of teaching science with geospatial technology. Journal of Science Teacher Education, 21(3), 351–370.
Uslu, O., & Bümen, N. T. (2012). Effects of the professional development program on Turkish teachers: Technology integration along with attitude towards ICT in education. The Turkish Online Journal of Educational Technology, 11(3), 115–127.
Van Braak, J., Tondeur, J., & Valcke, M. (2004). Explaining different types of computer use among primary school teachers. European Journal of Psychology of Education, 19(4), 407–422.
Wachira, P., & Keengwe, J. (2011). Technology integration barriers: Urban school mathematics teachers perspectives. Journal of Science Education and Technology, 20(1), 17–25.
Watson, D. M. (2001). Pedagogy before technology: Re-thinking the relationship between ICT and teaching. Education and Information Technologies, 6(4), 251–266.
Wells, J. (2007). Key design factors in durable instructional technology professional development. Journal of Technology and Teacher Education, 15(1), 101–122.
Wigfield, A. (1994). Expectancy-value theory of achievement motivation: A developmental perspective. Educational Psychology Review, 6(1), 49–78.
Wigfield, A., & Eccles, J. S. (1992). The development of achievement task values: A theoretical analysis. Developmental Review, 12(3), 265–310.
Wigfield, A., & Eccles, J. S. (2000). Expectancy—Value theory of achievement motivation. Contemporary Educational Psychology, 25(1), 68–81.
Wiley, D., Bliss, T. J., & McEwen, M. (2014). Open educational resources: A review of the literature. In J. M. Spector, M. D. Merrill, J. Elen, & M. J. Bishop (Eds.), Handbook of research on educational communications and technology (pp. 781–789). New York: Springer.
Xie, K., DeBacker, T. K., & Ferguson, C. (2006). Extending the traditional classroom through online discussion: The role of student motivation. Journal of Educational Computing Research, 34(1), 67–89.
Xie, K., Durrington, V. A., & Yen, L. L. (2011). Relationship between students’ motivation and their participation in asynchronous online discussions. Journal of Online Learning and Teaching, 7(1), 17–29.
Acknowledgements
The study reported in this paper is based upon work in the EDCITE: Evaluating Digital Content for Instructional and Teaching Excellence project supported by the Straight A Fund from the Ohio Department of Education. The conclusions and recommendations expressed in this article do not necessarily reflect the views of the Ohio Department of Education.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Ethical approval
All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee.
Appendices
Appendix 1
See Table 6.
Appendix 2
See Table 7.
Appendix 3
Self-reflection protocol
1. Please describe your professional development experiences using specific instances:
-
a.
What valuable things did you find in the course of the professional development? In what ways did the things contribute to your learning and participation?
-
b.
What technologies were employed to support the work of the project? How were the technologies used and integrated with your work?
-
c.
If any, what types of group collaboration were employed? How did the technologies support the group collaboration?
-
d.
What were the issues and difficulties you encountered when you participated in the project? In what ways did you or the project team address the issues?
2. Please describe how you implemented the evaluation of digital curricula:
-
a.
What were your strategies applied into the evaluation? (What was your approach to ensure the quality of your evaluation?)
-
b.
What issues did you encounter in the course of the evaluation? What changes did you make to deal with the issues?
-
c.
Which of the four criteria (i.e., content, pedagogy, technology, and standards alignment) was the easiest to evaluate? Why?
-
d.
Which of the four criteria (i.e., content, pedagogy, technology, and standards alignment) was the most difficult to evaluate? How did you tackle the issue?
-
e.
How did you work with content specialists?
3. Please describe how you implemented or will implement digital curriculum in your class and in your school.
-
a.
If any, what changes did you make in you your classroom teaching on the basis of your project experience?
-
b.
How would you characterize leadership and their colleagues toward the use of digital curriculum? If any, what was the use or plan of digital curriculum in your school?
-
c.
How did you communicate with your school leadership concerning digital curriculum implementation?
-
d.
What was your role in adopting digital curriculum into your school? Were there any changes or expectations after the project participation?
-
e.
How did you connect to and share your lessons with other teachers in your school?
Appendix 4
Interview protocol
Use of digital content/teaching beliefs
-
1.
Are you using technology or digital contents in your teaching?
-
a.
What kinds?
-
a.
-
2.
What kind of adjustments do you make in your teaching when you incorporate technology and digital content?
-
3.
How do students react when you using digital content?
-
a.
What do you do and your students do in the class?
-
a.
-
4.
Are there any differences in managing your classroom when using digital content?
-
a.
Are there any ways you are looking to improve in your implementation of technology/digital content in the classroom?
-
a.
-
5.
What barriers prevent you from using technology/digital content in your classroom?
-
6.
Are you currently looking for additional ways to use technology? How or for what?
Leadership/collaboration
-
1.
Are you sharing your use of digital contents with other teachers in your school more so now that you have participated in this EDCITE project?
-
2.
Are you now as a result of your participation planning/leading up any initiatives in your school regarding technology/digital content?
-
3.
How could you describe your roles and responsibility in your school now that you have this experience and knowledge through participating in this project?
EDCITE PD Evaluation Competency
-
1.
Are you satisfied with the PD provided?
-
2.
Can you describe difficulties/frustrations (or benefits/gains) with the face-to-face PD or online modules?
-
3.
Can you describe difficulties/frustrations (or benefits/gains) with evaluating digital contents in EDCITE project?
-
4.
How did the PD help change your evaluation skills?
-
5.
How was your experience using the Rubric provided? Was it helpful?
-
6.
How did the PD help change your evaluation skills?
-
7.
Do you feel more confident now in evaluating digital content and using tech in the classroom?
-
8.
Is there anything you would share with novice teachers who are evaluating digital content for the first time?
-
9.
Do you have any suggestion for the EDCITE team going forward? Or ideas for the programs PD next year?
Rights and permissions
About this article
Cite this article
Xie, K., Kim, M.K., Cheng, SL. et al. Teacher professional development through digital content evaluation. Education Tech Research Dev 65, 1067–1103 (2017). https://doi.org/10.1007/s11423-017-9519-0
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11423-017-9519-0