Seminars

View all Seminars  |  Download ICal for this event

Inducing constraints in paraphrase generation and consistency in paraphrase detection

Series: Ph.D. (Engg.) Colloquium- ONLINE

Speaker: Ashutosh Kumar, Ph.D (Engg.) student Dept. of C.S.A I.I.Sc.

Date/Time: Jun 29 09:00:00

Location: Microsoft Teams - ON-LINE

Faculty Advisor: Prof. Partha Pratim Talukdar

Abstract:
Deep learning models typically require a large volume of data. Manual curation of datasets is time-consuming and limited by imagination. As a result, natural language generation (NLG) has been employed to automate the process. However, NLG models are prone to producing degenerate, uninteresting, and often hallucinated outputs [1]. Constrained generation aims to overcome these shortcomings by appending additional information to the generation process. Training data thus generated can help improve the robustness of deep learning models. Therefore, the key research question of the thesis is: “How can we constrain generation models, especially in NLP, to produce meaningful outputs and utilize them for building better classification models?” In the first part, we present two approaches for constraining NLG models via the task of paraphrase generation. Paraphrase generation involves the generation of text that conveys the same meaning as a reference text. Our proposal is the following two strategies: DiPS (Diversity in Paraphrases using Submodularity): The first approach deals with constraining paraphrase generation to ensure diversity, i.e., ensuring that generated text(s) are sufficiently lexically different from each other without compromising on relevance (fidelity). We propose a decoding algorithm for obtaining such diverse texts. We provide a novel formulation of the problem in terms of monotone submodular function maximization, specifically targeted toward the task of paraphrase generation. We demonstrate the effectiveness of our method for data augmentation on multiple tasks such as intent classification and paraphrase recognition. SGCP (Syntax Guided Controlled Paraphraser): The second approach deals with constraining paraphrase generation to ensure syntacticality, i.e., ensuring that the generated text is syntactically coherent with an exemplar sentence. We propose Syntax Guided Controlled Paraphraser (SGCP), an end-to-end framework for syntactic paraphrase generation without compromising relevance (fidelity). The framework uses a sequence-to-sequence model with a Tree-LSTM-based gating mechanism to selectively choose syntactic representations during text generation. This approach performs significantly better than prior works that utilize only limited syntactic information in the exemplar. The second part of the research question pertains to ensuring that the generated output is meaningful. ​​Towards this, we present an approach for paraphrase detection to ascertain that the generated output is semantically coherent with the reference text. Paraphrase Detection is the task of detecting whether or not the two input natural language statements are paraphrases of each other. Fine-tuning pre-trained models such as BERT and RoBERTa on paraphrase datasets have become the go-to approaches for such tasks. However, tasks like paraphrase detection are symmetric - they require the output to be invariant of the order of the inputs. In fine-tuned models for classification, inconsistency is often observed in the predicted labels or confidence scores. We validate this shortcoming and apply a consistency loss function to alleviate inconsistency in symmetric classification. Our results show an improved consistency in predictions for three paraphrase detection datasets without a significant drop in the accuracy scores. While this work addresses the research question via paraphrase generation and detection, the approaches presented here apply broadly to NLP-based deep learning models that require imposing constraints and ensuring consistency. The work on paraphrase generation can be extended to impose new kinds of constraints on generation (for example, sentiment coherence), and the work on paraphrase detection can be applied to ensure consistency in other symmetric classification tasks that use deep learning models (for example, sarcasm interpretation). References: [1] Ehud Reiter. Hallucination in neural nlg. https://ehudreiter.com/2018/11/12/hallucination-in-neural-nlg/, 2018. Microsoft teams link: https://teams.microsoft.com/l/meetup-join/19%3ameeting_ZGM0NjE1ZTYtNWJhMi00ZjhmLTkxOTMtNjBhMmY1ZDYxMGI3%40thread.v2/0?context=%7b%22Tid%22%3a%226f15cd97-f6a7-41e3-b2c5-ad4193976476%22%2c%22Oid%22%3a%2258c2a810-9762-463d-90d0-20832b3d1592%22%7d

Speaker Bio:

Host Faculty: