Stance Detection Using Transformer Architectures and Temporal Convolutional Networks

Abstract

Stance detection can be defined as the task of automatically detecting the relation between or the relative perspective of two pieces of text- a claim or headline and the corresponding article body. Stance detection is an integral part of the pipeline used for automatic fake news detection which is an open research problem in Natural Language Processing. The past year has seen a lot of developments in the field of NLP and the application of transfer learning to it. Bidirectional language models with recurrence and various transformer models have been consistently improving the state-of-the-art results on various NLP tasks. In this research work, we specifically focus on the application of embeddings from BERT and XLNet to solve the problem of stance detection. We extract the weights from the last hidden layer of the base models in both cases and use them as embeddings to train task-specific recurrent models. We also present a novel approach to tackle stance detection wherein we apply Temporal Convolutional Networks to solve the problem. Temporal Convolutional Networks are being seen as an ideal replacement for LSTM/GRUs for sequence modelling tasks. In this work, we implement models to investigate if they can be used for NLP tasks as well. We present our results with an exhaustive comparative analysis of multiple architectures trained on the Fake News Challenge (FNC) dataset.

Kushal Jain and Fenil Doshi have made equal contributions to the work and Lakshmi Kurup was our supervisor.