Abstract
In recent years, product and project ideas are often sourced from public competitions, where anyone can enter their own solutions to an open-ended question. While copious ideas can be gathered in this way, it becomes difficult to find the most promising results among all entries. This paper explores the potential of automating the detection of interesting ideas and studies the effect of various features of ideas on the prediction task. A BERT-based model is built to rank ideas by their predicted interestingness, using text embeddings from idea descriptions and the concreteness, novelty as well as the uniqueness of ideas. The model is trained on a dataset of OpenIDEO idea competitions. The results show that language models can be used to speed up finding promising ideas, but care must be taken in choosing a suitable dataset.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Adamopoulos, P., Tuzhilin, A.: On unexpectedness in recommender systems: Or how to better expect the unexpected. ACM Trans. Intell. Syst. Technol. (TIST) 5(4), 1–32 (2014)
Ahmed, F., Fuge, M.: Capturing winning ideas in online design communities. In: Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing, ACM, New York, NY, USA (2017). https://doi.org/10.1145/2998181.2998249
Ahmed, F., Fuge, M.: Ranking ideas for diversity and quality. J. Mech. Des. 140(1) (2018). https://doi.org/10.1115/1.4038070
Blohm, I., Riedl, C., Füller, J., Leimeister, J.M.: Rate or trade? identifying winning ideas in open idea sourcing. Inf. Syst. Res. 27(1), 27–48 (2016). https://doi.org/10.1287/isre.2015.0605
Breunig, M.M., Kriegel, H.P., Ng, R.T., Sander, J.: Lof. In: Dunham, M. (ed.) Proceedings of the 2000 ACM SIGMOD International Conference on Management of Data, pp. 93–104. ACM Conferences, ACM, New York, NY (2000). https://doi.org/10.1145/342009.335388
Fuge, M., Tee, K., Agogino, A., Maton, N.: Analysis of collaborative design networks: a case study of openideo. J. Comput. Inf. Sci. Eng. 14(2) (2014). https://doi.org/10.1115/1.4026510
Jatowt, A., Hung, I.-C., Färber, M., Campos, R., Yoshikawa, M.: Exploding TV sets and disappointing laptops: suggesting interesting content in news archives based on surprise estimation. In: Hiemstra, D., Moens, M.-F., Mothe, J., Perego, R., Potthast, M., Sebastiani, F. (eds.) ECIR 2021. LNCS, vol. 12656, pp. 254–269. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-72113-8_17
Kuznetsov, S.O., Makhalova, T.: On interestingness measures of formal concepts. Inf. Sci. 442–443, 202–219 (2018). https://doi.org/10.1016/j.ins.2018.02.032
Loshchilov, I., Hutter, F.: Fixing weight decay regularization in adam. CoRR abs/1711.05101 (2017)
Naveed, N., Gottron, T., Kunegis, J., Alhadi, A.C.: Bad news travel fast: a content-based analysis of interestingness on twitter. In: Proceedings of the 3rd International Web Science Conference, pp. 1–7 (2011)
Paetzold, G., Specia, L.: Inferring psycholinguistic properties of words. In: Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Association for Computational Linguistics, Stroudsburg, PA, USA (2016). https://doi.org/10.18653/v1/n16-1050
Reimers, N., Gurevych, I.: Sentence-BERT: sentence embeddings using siamese BERT-networks. arXiv (2019). https://doi.org/10.48550/arXiv.1908.10084
Ribeiro, M.T., Singh, S., Guestrin, C.: Why should i trust you?”: Explaining the predictions of any classifier. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA, 13–17 August 2016, pp. 1135–1144 (2016)
Sanh, V., Debut, L., Chaumond, J., Wolf, T.: Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter. CoRR abs/1910.01108 (2019)
Sun, Y., Wong, A.K.C., Kamel, M.S.: Classification of imbalanced data: a review. Int. J. Pattern Recogn. Artif. Intell. 23(04), 687–719 (2009)
Terwiesch, C., Xu, Y.: Innovation contests, open innovation, and multiagent problem solving. Manage. Sci. 54(9), 1529–1543 (2008). https://doi.org/10.1287/mnsc.1080.0884
Dasgupta, T., Dey, L.: Automatic scoring for innovativeness of textual ideas (2016)
van der Burgt, M.: Calibrating low-default portfolios, using the cumulative accuracy profile. J. Risk Model Validation 1(4), 17–33 (2008)
Wahl, J., Füller, J., Hutter, K.: What’s the problem? how crowdsourcing and text-mining may contribute to the understanding of unprecedented problems such as covid-19. R &D Manage. 52(2), 427–446 (2022). https://doi.org/10.1111/radm.12526
Baba, Y., Li, J., Kashima, H.: Crowdea: multi-view idea prioritization with crowds. In: Proceedings of the AAAI Conference on Human Computation and Crowdsourcing, vol. 8, pp. 23–32 (2020). https://ojs.aaai.org/index.php/hcomp/article/view/7460
Zhang, Y., Siriaraya, P., Kawai, Y., Jatowt, A.: Predicting time and location of future crimes with recommendation methods. Knowledge-Based Systems 210, 106503 (2020). https://doi.org/10.1016/j.knosys.2020.106503
Acknowledgment
The dataset consisting of 21 OpenIDEO competitions used in the paper has been provided by the research group for Innovation & Entrepreneurship (https://www.uibk.ac.at/smt/innovation-entrepreneurship/) at the Department of Strategic Management, Marketing and Tourism of the University of Innsbruck. Parts of the experiments have been conducted while Bela Pfahl was employed as a student research assistant in the group.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Pfahl, B., Jatowt, A. (2023). Towards Detecting Interesting Ideas Expressed in Text. In: Kamps, J., et al. Advances in Information Retrieval. ECIR 2023. Lecture Notes in Computer Science, vol 13981. Springer, Cham. https://doi.org/10.1007/978-3-031-28238-6_45
Download citation
DOI: https://doi.org/10.1007/978-3-031-28238-6_45
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-28237-9
Online ISBN: 978-3-031-28238-6
eBook Packages: Computer ScienceComputer Science (R0)