Abstract
The Reproducibility Crisis is a phenomenon that has gained considerable attention in the psychological sciences. Scholars in these fields have found that many high profile findings are either difficult to reproduce or could not be replicated. These findings have ultimately encouraged researchers to adopt pre-registered results, replication in study design and open data. As an emerging field, NeuroIS has an opportunity to learn from this crisis and adopt new practices based on the lessons learned in the psychological sciences. We explored the current state of NeuroIS research from the perspective of reproducibility by conducting a survey of the extant NeuroIS literature. We conclude by suggesting two practices that the NeuroIS community can undertake to help address the replication problem.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Discrepancies between the findings of this paper and Riedl et al. [16] can be attributed to differences in how authors interpreted studies as complete or empirical, and the subjective judgement employed by us when identifying a primary research method for each study.
References
Hempel, C. G. (1968). Maximal specificity and lawlikeness in probabilistic explanation. Philosophy of Science, 35(2), 116–133.
Open Science Collaboration. (2015). Estimating the reproducibility of psychological science. Science, 349(6251), aac4716.
Baker, M. (2016). Is There a Reproducibility Crisis? Nature, 533.
Francis, G. (2012). Too good to be true: Publication bias in two prominent studies from experimental psychology. Psychonomic Bulletin & Review, 19(2), 151–156.
Francis, G. (2012). Publication bias and the failure of replication in experimental psychology. Psychonomic Bulletin & Review, 19(6), 975–991.
Gonzales, J. E., & Cunningham, C. A. (2015). The promise of pre-registration in psychological research. https://www.apa.org/science/about/psa/2015/08/pre-registration.
Foster, E. D., & Deardorff, A. (2017). Open science framework (OSF). Journal of the Medical Library Association, 105(2), 203.
Luck, S. J., & Gaspelin, N. (2017). How to get statistically significant effects in any ERP experiment (and why you shouldn’t). Psychophysiology, 54, 146–157.
Poldrack, R. A., Baker, C. I., Durnez, J., Gorgolewski, K. J., Matthews, P. M., Munafo, M. R., et al. (2017). Scanning the horizon: Towards transparent and reproducible neuroimaging research. Nature Reviews, 18.
Van Essen, D. C., Ugurbil, K., Auerbach, E., Barch, D., Behrens, T. E. J., Bucholz, R., et al. (2012). The Human Connectome Project: A data acquisition perspective. Neuroimage, 62(4), 2222–2231.
Taylor, J. R., Williams, N., Cusack, R., Auer, T., Shafto, M. A., Dixon, M., et al. (2017). The Cambridge Centre for Ageing and Neuroscience (CamCAN) data repository: Structural and functional MRI, MEG, and cognitive data from a cross-sectional adult lifespan sample. NeuroImage, 144, 262–269.
Mueller, S. G., Weiner, M. W., Thal, L. J., Petersen, R. C., Jack, C. R., Jagust, W., et al. (2005). Ways toward an early diagnosis in Alzheimer’s disease: The Alzheimer’s Disease Neuroimaging Initiative (ADNI). Alzheimer’s and Dementia, 1(1), 55–66.
Dimoka, A., Davis, F. D., Gupta, A., Pavlou, P. A., Banker, R. D., Dennis, A. R., et al. (2012). On the use of neurophysiological tools in IS research: Developing a research agenda for NeuroIS. MIS Quarterly, 36(3), 679–702.
vom Brocke, J., & Liang, T.-P. (2014). Guidelines for neuroscience studies in information systems research. Journal of Management Information Systems.
Dennis, A. R., & Valacich, J. S. (2014). A replication manifesto. AIS Transactions on Replication Research, 1, 1–4.
Riedl, R., Fischer, T., & Léger, P.-M. (2017). A decade of NeuroIS research: Status quo, challenges, and future directions. In Proceedings of the 38th International Conference on Information Systems, Seoul.
Dimoka, A. (2010). What does the brain tell us about trust and distrust? Evidence from a functional neuroimaging study. MIS Quarterly, 34(2), 373–396.
De Guinea, A. O., & Webster, J. (2013). An investigation of information systems use patterns: Technological events as triggers, the effect of time, and consequences for performance. MIS Quarterly, 38(4), 1165–1188.
Goggins, S. P., Schmidt, M., Guajardo, J., & Moore, J. (2010). Assessing multiple perspectives in three dimensional virtual worlds: Eye tracking and all views qualitative analysis (AVQA). In 2010 43rd Hawaii International Conference on System Sciences (pp. 1–10). IEEE.
Gregor, S., Lin, A. C., Gedeon, T., Riaz, A., & Zhu, D. (2014). Neuroscience and a nomological network for the understanding and assessment of emotions in information systems research. Journal of Management Information Systems, 30(4), 13–48.
Perrin, J. L., Paillé, D., & Baccino, T. (2014). Reading tilted: Does the use of tablets impact performance? An oculometric study. Computers in Human Behavior, 39, 339–345.
Clayton, R. B., Leshner, G., & Almond, A. (2015). The extended iSelf: The impact of iPhone separation on cognition, emotion, and physiology. Journal of Computer-Mediated Communication, 20(2), 119–135.
Cole, M. J., Hendahewa, C., Belkin, N. J., & Shah, C. (2015). User activity patterns during information search. ACM Transactions on Information Systems (TOIS), 33(1), 1–39.
Jenkins, J. L., Anderson, B. B., Vance, A., Kirwan, C. B., & Eargle, D. (2016). More harm than good? How messages that interrupt can make us vulnerable. Information Systems Research, 27(4), 880–896.
Luan, J., Yao, Z., Zhao, F., & Liu, H. (2016). Search product and experience product online reviews: An eye-tracking study on consumers’ review search behavior. Computers in Human Behavior, 65, 420–430.
Bahr, G. S., & Ford, R. A. (2011). How and why pop-ups don’t work: Pop-up prompted eye movements, user affect and decision making. Computers in Human Behavior, 27(2), 776–783.
Nunamaker, J. F., Derrick, D. C., Elkins, A. C., Burgoon, J. K., & Patton, M. W. (2011). Embodied conversational agent-based kiosk for automated interviewing. Journal of Management Information Systems, 28(1), 17–48.
Astor, P. J., Adam, M. T., Jerčić, P., Schaaff, K., & Weinhardt, C. (2013). Integrating biosignals into information systems: A NeuroIS tool for improving emotion regulation. Journal of Management Information Systems, 30(3), 247–278.
De Guinea, A. O., Titah, R., & Léger, P. M. (2013). Measure for measure: A two study multi-trait multi-method investigation of construct validity in IS research. Computers in Human Behavior, 29(3), 833–844.
Jay, C., Brown, A., & Harper, S. (2013). Predicting whether users view dynamic content on the world wide web. ACM Transactions on Computer-Human Interaction (TOCHI), 20(2), 1–33.
Dogusoy-Taylan, B., & Cagiltay, K. (2014). Cognitive analysis of experts’ and novices’ concept mapping processes: An eye tracking study. Computers in Human Behavior, 36, 82–93.
Molina, A. I., Redondo, M. A., Lacave, C., & Ortega, M. (2014). Assessing the effectiveness of new devices for accessing learning materials: An empirical analysis based on eye tracking and learner subjective perception. Computers in Human Behavior, 31, 475–490.
Wang, C. C., & Hsu, M. C. (2014). An exploratory study using inexpensive electroencephalography (EEG) to understand flow experience in computer-based instruction. Information & Management, 51(7), 912–923.
Galluch, P. S., Grover, V., & Thatcher, J. B. (2015). Interrupting the workplace: Examining stressors in an information technology context. Journal of the Association for Information Systems, 16(1), 1–47.
Huang, Y. F., Kuo, F. Y., Luu, P., Tucker, D., & Hsieh, P. J. (2015). Hedonic evaluation can be automatically performed: An electroencephalography study of website impression across two cultures. Computers in Human Behavior, 49, 138–146.
Hu, Q., West, R., & Smarandescu, L. (2015). The role of self-control in information security violations: Insights from a cognitive neuroscience perspective. Journal of Management Information Systems, 31(4), 6–48.
Adam, M. T., Krämer, J., & Weinhardt, C. (2012). Excitement up! Price down! Measuring emotions in Dutch auctions. International Journal of Electronic Commerce, 17(2), 7–40.
Riedl, R., Mohr, P. N., Kenning, P. H., Davis, F. D., & Heekeren, H. R. (2014). Trusting humans and avatars: A brain imaging study based on evolution theory. Journal of Management Information Systems, 30(4), 83–114.
Labonté-LeMoyne, É., Santhanam, R., Léger, P. M., Courtemanche, F., Fredette, M., & Sénécal, S. (2015). The delayed effect of treadmill desk usage on recall and attention. Computers in Human Behavior, 46, 1–5.
Association for Information Systems: Senior Scholars’ Basket of Journals. https://aisnet.org/page/SeniorScholarBasket.
van der Aalst, W., Bichler, M., & Heinzl, A. (2016). Open research in business and information systems engineering. Business & Information Systems Engineering, 58(6), 375–379.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Conrad, C., Bailey, L. (2020). What Can NeuroIS Learn from the Replication Crisis in Psychological Science?. In: Davis, F., Riedl, R., vom Brocke, J., Léger, PM., Randolph, A., Fischer, T. (eds) Information Systems and Neuroscience. Lecture Notes in Information Systems and Organisation, vol 32. Springer, Cham. https://doi.org/10.1007/978-3-030-28144-1_14
Download citation
DOI: https://doi.org/10.1007/978-3-030-28144-1_14
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-28143-4
Online ISBN: 978-3-030-28144-1
eBook Packages: Business and ManagementBusiness and Management (R0)