Abstract
Unsupervised textual style transfer presupposes that style is a coherent and consistent concept and that style transfer approaches will generalise consistently across different domains of style. This paper explores whether this presupposition is appropriate for different types of style. We explore this question by comparing the performance and latent representations of a variety of neural encoder-decoder style-transfer architecture when applied to sentiment transfer and formality transfer. Our findings indicate that the relationship between style and content shifts between these different domains of style: for sentiment, style and content are closely entangled; however, for formality, they are less entangled. Our findings suggest that for different types of styles different approaches to modeling style for style-transfer are necessary.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
References
Bahdanau, D., Cho, K.H., Bengio, Y.: Neural machine translation by jointly learning to align and translate. In: 3rd International Conference on Learning Representations, ICLR 2015 (2015)
Chung, J., Gulcehre, C., Cho, K., Bengio, Y.: Empirical evaluation of gated recurrent neural networks on sequence modeling. In: NIPS 2014 Workshop on Deep Learning, December 2014 (2014)
Conneau, A., Kruszewski, G., Lample, G., Barrault, L., Baroni, M.: What you can cram into a single vector: Probing sentence embeddings for linguistic properties. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 2126–2136 (2018)
Fu, Z., Tan, X., Peng, N., Zhao, D., Yan, R.: Style transfer in text: exploration and evaluation. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 32 (2018)
Goodfellow, I., et al.: Generative adversarial nets. In: Ghahramani, Z., Welling, M., Cortes, C., Lawrence, N.D., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems, vol. 27, pp. 2672–2680. Curran Associates, Inc. (2014). http://papers.nips.cc/paper/5423-generative-adversarial-nets.pdf
Hu, Z., Yang, Z., Liang, X., Salakhutdinov, R., Xing, E.P.: Controllable text generation. CoRR abs/1703.00955 (2017). http://arxiv.org/abs/1703.00955
Jafaritazehjani, S., Lecorvé, G., Lolive, D., Kelleher, J.D.: Style versus content: a distinction without a (learnable) difference? In: Proceedings of the 28th International Conference on Computational Linguistics, COLING 2020, Association for Computational Linguistics (2020)
John, V., Mou, L., Bahuleyan, H., Vechtomova, O.: Disentangled representation learning for non-parallel text style transfer. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 424–434 (2019)
Kim, Y.: Convolutional neural networks for sentence classification. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1746–1751 (2014)
Kusner, M., Sun, Y., Kolkin, N., Weinberger, K.: From word embeddings to document distances. In: Bach, F., Blei, D. (eds.) Proceedings of the 32nd International Conference on Machine Learning. Proceedings of Machine Learning Research, PMLR, Lille, France, 07–09 Jul 2015, vol. 37, pp. 957–966 (2015). http://proceedings.mlr.press/v37/kusnerb15.html
Leeftink, W., Spanakis, G.: Towards controlled transformation of sentiment in sentences. CoRR abs/1808.04365 (2019). http://arxiv.org/abs/1808.04365
Li, J., Jia, R., He, H., Liang, P.: Delete, retrieve, generate: a simple approach to sentiment and style transfer. In: 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL HLT 2018, pp. 1865–1874. Association for Computational Linguistics (ACL) (2018)
Ma, S., Sun, X.: A semantic relevance based neural network for text summarization and text simplification. Comput. Linguist. 1(1) (2017)
Pennington, J., Socher, R., Manning, C.D.: Glove: global vectors for word representation. In: EMNLP (2014)
Prabhumoye, S., Tsvetkov, Y., Salakhutdinov, R., Black, A.W.: Style transfer through back-translation. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 866–876. Association for Computational Linguistics (2018). http://aclweb.org/anthology/P18-1080
Ramos, J., et al.: Using TF-IDF to determine word relevance in document queries. Citeseer
Rao, S., Tetreault, J.R.: Dear sir or madam, may i introduce the GYAFC dataset: Corpus, benchmarks and metrics for formality style transfer. In: NAACL-HLT (2018)
Romanov, A., Rumshisky, A., Rogers, A., Donahue, D.: Adversarial decomposition of text representation. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pp. 815–825 (2019)
Rubner, Y., Tomasi, C., Guibas, L.J.: The earth mover’s distance as a metric for image retrieval. Int. J. Comput. Vis. 40(2), 99–121 (2000)
Shen, T., Lei, T., Barzilay, R., Jaakkola, T.: Style transfer from non-parallel text by cross-alignment. In: Guyon, I., et al. (eds.) Advances in Neural Information Processing Systems, vol. 30, pp. 6830–6841. Curran Associates, Inc. (2017). http://papers.nips.cc/paper/7259-style-transfer-from-non-parallel-text-by-cross-alignment.pdf
Shivakumar, P.G., Georgiou, P.: Confusion2Vec: towards enriching vector space word representations with representational ambiguities. PeerJ Comput. Sci. 5, e195 (2019)
Singh, A., Palod, R.: Sentiment transfer using seq2seq adversarial autoencoders. CoRR abs/1804.04003 (2018). http://arxiv.org/abs/1804.04003
Sutskever, I., Vinyals, O., Le, Q.V.: Sequence to sequence learning with neural networks. In: Proceedings of the Conference in Neural Information Processing Systems (NIPS), pp. 3104–3112 (2014)
Tikhonov, A., Yamshchikov, I.P.: What is wrong with style transfer for texts? CoRR abs/1808.04365 (2018). http://arxiv.org/abs/1808.04365
Yamshchikov, I., Shibaev, V., Khlebnikov, N., Tikhonov, A.: Style-transfer and paraphrase: looking for a sensible semantic similarity metric. arXiv preprint arXiv:2004.05001 (2020)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Jafaritazehjani, S., Lecorvé, G., Lolive, D., Kelleher, J.D. (2021). Style as Sentiment Versus Style as Formality: The Same or Different?. In: Farkaš, I., Masulli, P., Otte, S., Wermter, S. (eds) Artificial Neural Networks and Machine Learning – ICANN 2021. ICANN 2021. Lecture Notes in Computer Science(), vol 12895. Springer, Cham. https://doi.org/10.1007/978-3-030-86383-8_39
Download citation
DOI: https://doi.org/10.1007/978-3-030-86383-8_39
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-86382-1
Online ISBN: 978-3-030-86383-8
eBook Packages: Computer ScienceComputer Science (R0)