Abstract
This paper provides an overview of eRisk 2019, the third edition of this lab under the CLEF conference. The main purpose of eRisk is to explore issues of evaluation methodology, effectiveness metrics and other processes related to early risk detection. Early detection technologies can be employed in different areas, particularly those related to health and safety. This edition of eRisk had three tasks. Two of them shared the same format and focused on early detecting signs of depression (T1) or self-harm (T2). The third task focused on an innovative challenge related to automatically filling a depression questionnaire based on user interactions in social media.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
However, following the extraction method suggested by Coppersmith and colleagues [2], the post discussing the diagnosis was removed from the collection.
- 2.
More information about the server can be found on the lab website http://early.irlab.org/server.html.
- 3.
Observe that Sadeque et al. (see [6], p. 497) computed the latency for all users such that \(g_u=1\). We argue that latency should be computed only for the true positives. The false negatives (\(g_u=1\), \(d_u=0\)) are not detected by the system and, therefore, they would not generate an alert.
- 4.
Again, we adopt Sadeque et al.’s proposal but we estimate latency only over the true positives.
- 5.
In the eRisk 2017 collection this led to setting p to 0.0078.
- 6.
Slightly less than 25% because a couple of questions have more than four possible answers.
References
Beck, A.T., Ward, C.H., Mendelson, M., Mock, J., Erbaugh, J.: An inventory for measuring depression. JAMA Psychiatry 4(6), 561–571 (1961). https://doi.org/10.1001/archpsyc.1961.01710120031004
Coppersmith, G., Dredze, M., Harman, C.: Quantifying mental health signals in Twitter. In: ACL Workshop on Computational Linguistics and Clinical Psychology (2014)
Losada, D.E., Crestani, F.: A test collection for research on depression and language use. In: Fuhr, N., et al. (eds.) CLEF 2016. LNCS, vol. 9822, pp. 28–39. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-44564-9_3
Losada, D.E., Crestani, F., Parapar, J.: eRISK 2017: CLEF lab on early risk prediction on the internet: experimental foundations. In: Jones, G.J.F., et al. (eds.) CLEF 2017. LNCS, vol. 10456, pp. 346–360. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-65813-1_30
Losada, D.E., Crestani, F., Parapar, J.: Overview of eRisk: early risk prediction on the internet. In: Bellot, P., et al. (eds.) CLEF 2018. LNCS, vol. 11018, pp. 343–361. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-98932-7_30
Sadeque, F., Xu, D., Bethard, S.: Measuring the latency of depression detection in social media. In: WSDM, pp. 495–503. ACM (2018)
Trotzek, M., Koitka, S., Friedrich, C.M.: Utilizing neural networks and linguistic metadata for early detection of depression indications in text sequences. CoRR abs/1804.07000 (2018)
Acknowledgements
We thank the support obtained from the Swiss National Science Foundation (SNSF) under the project “Early risk prediction on the Internet: an evaluation corpus”, 2015.
We also thank the financial support obtained from the (i) “Ministerio de Ciencia, Innovación y Universidades” of the Government of Spain (research grants RTI2018-093336-B-C21 and RTI2018-093336-B-C22), (ii) “Consellería de Educación, Universidade e Formación Profesional”, Xunta de Galicia (grants ED431C 2018/29, ED431G/08 and ED431G/01 – “Centro singular de investigación de Galicia” –). All grants were co-funded by the European Regional Development Fund (ERDF/FEDER program).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Losada, D.E., Crestani, F., Parapar, J. (2019). Overview of eRisk 2019 Early Risk Prediction on the Internet. In: Crestani, F., et al. Experimental IR Meets Multilinguality, Multimodality, and Interaction. CLEF 2019. Lecture Notes in Computer Science(), vol 11696. Springer, Cham. https://doi.org/10.1007/978-3-030-28577-7_27
Download citation
DOI: https://doi.org/10.1007/978-3-030-28577-7_27
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-28576-0
Online ISBN: 978-3-030-28577-7
eBook Packages: Computer ScienceComputer Science (R0)