Paper Group NANR 143
A Kernel Random Matrix-Based Approach for Sparse PCA. Bag-of-Words Transfer: Non-Contextual Techniques for Multi-Task Learning. Predicting Sentiment of Polish Language Short Texts. Improving Named Entity Linking Corpora Quality. The Impact of Word Representations on Sequential Neural MWE Identification. Fast Domain Adaptation of Semantic Parsers vi …
A Kernel Random Matrix-Based Approach for Sparse PCA
| Title | A Kernel Random Matrix-Based Approach for Sparse PCA |
| Authors | Mohamed El Amine Seddik, Mohamed Tamaazousti, Romain Couillet |
| Abstract | In this paper, we present a random matrix approach to recover sparse principal components from n p-dimensional vectors. Specifically, considering the large dimensional setting where n, p → ∞ with p/n → c ∈ (0, ∞) and under Gaussian vector observations, we study kernel random matrices of the type f (Ĉ), where f is a three-times continuously differentiable function applied entry-wise to the sample covariance matrix Ĉ of the data. Then, assuming that the principal components are sparse, we show that taking f in such a way that f’(0) = f’'(0) = 0 allows for powerful recovery of the principal components, thereby generalizing previous ideas involving more specific f functions such as the soft-thresholding function. |
| Tasks | |
| Published | 2019-05-01 |
| URL | https://openreview.net/forum?id=rkgBHoCqYX |
| https://openreview.net/pdf?id=rkgBHoCqYX | |
| PWC | https://paperswithcode.com/paper/a-kernel-random-matrix-based-approach-for |
| Repo | |
| Framework | |
Bag-of-Words Transfer: Non-Contextual Techniques for Multi-Task Learning
| Title | Bag-of-Words Transfer: Non-Contextual Techniques for Multi-Task Learning |
| Authors | Seth Ebner, Felicity Wang, Benjamin Van Durme |
| Abstract | Many architectures for multi-task learning (MTL) have been proposed to take advantage of transfer among tasks, often involving complex models and training procedures. In this paper, we ask if the sentence-level representations learned in previous approaches provide significant benefit beyond that provided by simply improving word-based representations. To investigate this question, we consider three techniques that ignore sequence information: a syntactically-oblivious pooling encoder, pre-trained non-contextual word embeddings, and unigram generative regularization. Compared to a state-of-the-art MTL approach to textual inference, the simple techniques we use yield similar performance on a universe of task combinations while reducing training time and model size. |
| Tasks | Multi-Task Learning, Word Embeddings |
| Published | 2019-11-01 |
| URL | https://www.aclweb.org/anthology/D19-6105/ |
| https://www.aclweb.org/anthology/D19-6105 | |
| PWC | https://paperswithcode.com/paper/bag-of-words-transfer-non-contextual |
| Repo | |
| Framework | |
Predicting Sentiment of Polish Language Short Texts
| Title | Predicting Sentiment of Polish Language Short Texts |
| Authors | Aleks Wawer, er, Julita Sobiczewska |
| Abstract | The goal of this paper is to use all available Polish language data sets to seek the best possible performance in supervised sentiment analysis of short texts. We use text collections with labelled sentiment such as tweets, movie reviews and a sentiment treebank, in three comparison modes. In the first, we examine the performance of models trained and tested on the same text collection using standard cross-validation (in-domain). In the second we train models on all available data except the given test collection, which we use for testing (one vs rest cross-domain). In the third, we train a model on one data set and apply it to another one (one vs one cross-domain). We compare wide range of methods including machine learning on bag-of-words representation, bidirectional recurrent neural networks as well as the most recent pre-trained architectures ELMO and BERT. We formulate conclusions as to cross-domain and in-domain performance of each method. Unsurprisingly, BERT turned out to be a strong performer, especially in the cross-domain setting. What is surprising however, is solid performance of the relatively simple multinomial Naive Bayes classifier, which performed equally well as BERT on several data sets. |
| Tasks | Sentiment Analysis |
| Published | 2019-09-01 |
| URL | https://www.aclweb.org/anthology/R19-1151/ |
| https://www.aclweb.org/anthology/R19-1151 | |
| PWC | https://paperswithcode.com/paper/predicting-sentiment-of-polish-language-short |
| Repo | |
| Framework | |
Improving Named Entity Linking Corpora Quality
| Title | Improving Named Entity Linking Corpora Quality |
| Authors | Albert Weichselbraun, Adrian M.P. Brasoveanu, Philipp Kuntschik, Lyndon J.B. Nixon |
| Abstract | Gold standard corpora and competitive evaluations play a key role in benchmarking named entity linking (NEL) performance and driving the development of more sophisticated NEL systems. The quality of the used corpora and the used evaluation metrics are crucial in this process. We, therefore, assess the quality of three popular evaluation corpora, identifying four major issues which affect these gold standards: (i) the use of different annotation styles, (ii) incorrect and missing annotations, (iii) Knowledge Base evolution, (iv) and differences in annotating co-occurrences. This paper addresses these issues by formalizing NEL annotations and corpus versioning which allows standardizing corpus creation, supports corpus evolution, and paves the way for the use of lenses to automatically transform between different corpus configurations. In addition, the use of clearly defined scoring rules and evaluation metrics ensures a better comparability of evaluation results. |
| Tasks | Entity Linking |
| Published | 2019-09-01 |
| URL | https://www.aclweb.org/anthology/R19-1152/ |
| https://www.aclweb.org/anthology/R19-1152 | |
| PWC | https://paperswithcode.com/paper/improving-named-entity-linking-corpora |
| Repo | |
| Framework | |
The Impact of Word Representations on Sequential Neural MWE Identification
| Title | The Impact of Word Representations on Sequential Neural MWE Identification |
| Authors | Nicolas Zampieri, Carlos Ramisch, Geraldine Damnati |
| Abstract | Recent initiatives such as the PARSEME shared task allowed the rapid development of MWE identification systems. Many of those are based on recent NLP advances, using neural sequence models that take continuous word representations as input. We study two related questions in neural MWE identification: (a) the use of lemmas and/or surface forms as input features, and (b) the use of word-based or character-based embeddings to represent them. Our experiments on Basque, French, and Polish show that character-based representations yield systematically better results than word-based ones. In some cases, character-based representations of surface forms can be used as a proxy for lemmas, depending on the morphological complexity of the language. |
| Tasks | |
| Published | 2019-08-01 |
| URL | https://www.aclweb.org/anthology/W19-5121/ |
| https://www.aclweb.org/anthology/W19-5121 | |
| PWC | https://paperswithcode.com/paper/the-impact-of-word-representations-on |
| Repo | |
| Framework | |
Fast Domain Adaptation of Semantic Parsers via Paraphrase Attention
| Title | Fast Domain Adaptation of Semantic Parsers via Paraphrase Attention |
| Authors | Avik Ray, Yilin Shen, Hongxia Jin |
| Abstract | Semantic parsers are used to convert user{'}s natural language commands to executable logical form in intelligent personal agents. Labeled datasets required to train such parsers are expensive to collect, and are never comprehensive. As a result, for effective post-deployment domain adaptation and personalization, semantic parsers are continuously retrained to learn new user vocabulary and paraphrase variety. However, state-of-the art attention based neural parsers are slow to retrain which inhibits real time domain adaptation. Secondly, these parsers do not leverage numerous paraphrases already present in the training dataset. Designing parsers which can simultaneously maintain high accuracy and fast retraining time is challenging. In this paper, we present novel paraphrase attention based sequence-to-sequence/tree parsers which support fast near real time retraining. In addition, our parsers often boost accuracy by jointly modeling the semantic dependencies of paraphrases. We evaluate our model on benchmark datasets to demonstrate upto 9X speedup in retraining time compared to existing parsers, as well as achieving state-of-the-art accuracy. |
| Tasks | Domain Adaptation |
| Published | 2019-11-01 |
| URL | https://www.aclweb.org/anthology/D19-6111/ |
| https://www.aclweb.org/anthology/D19-6111 | |
| PWC | https://paperswithcode.com/paper/fast-domain-adaptation-of-semantic-parsers |
| Repo | |
| Framework | |
An Investigation of Deep Learning Systems for Suicide Risk Assessment
| Title | An Investigation of Deep Learning Systems for Suicide Risk Assessment |
| Authors | Michelle Morales, Prajjalita Dey, Thomas Theisen, Danny Belitz, Natalia Chernova |
| Abstract | This work presents the systems explored as part of the CLPsych 2019 Shared Task. More specifically, this work explores the promise of deep learning systems for suicide risk assessment. |
| Tasks | |
| Published | 2019-06-01 |
| URL | https://www.aclweb.org/anthology/W19-3023/ |
| https://www.aclweb.org/anthology/W19-3023 | |
| PWC | https://paperswithcode.com/paper/an-investigation-of-deep-learning-systems-for |
| Repo | |
| Framework | |
Learning Bilingual Sentiment-Specific Word Embeddings without Cross-lingual Supervision
| Title | Learning Bilingual Sentiment-Specific Word Embeddings without Cross-lingual Supervision |
| Authors | Yanlin Feng, Xiaojun Wan |
| Abstract | Word embeddings learned in two languages can be mapped to a common space to produce Bilingual Word Embeddings (BWE). Unsupervised BWE methods learn such a mapping without any parallel data. However, these methods are mainly evaluated on tasks of word translation or word similarity. We show that these methods fail to capture the sentiment information and do not perform well enough on cross-lingual sentiment analysis. In this work, we propose UBiSE (Unsupervised Bilingual Sentiment Embeddings), which learns sentiment-specific word representations for two languages in a common space without any cross-lingual supervision. Our method only requires a sentiment corpus in the source language and pretrained monolingual word embeddings of both languages. We evaluate our method on three language pairs for cross-lingual sentiment analysis. Experimental results show that our method outperforms previous unsupervised BWE methods and even supervised BWE methods. Our method succeeds for a distant language pair English-Basque. |
| Tasks | Sentiment Analysis, Word Embeddings |
| Published | 2019-06-01 |
| URL | https://www.aclweb.org/anthology/N19-1040/ |
| https://www.aclweb.org/anthology/N19-1040 | |
| PWC | https://paperswithcode.com/paper/learning-bilingual-sentiment-specific-word |
| Repo | |
| Framework | |
``Caption’’ as a Coherence Relation: Evidence and Implications
| Title | ``Caption’’ as a Coherence Relation: Evidence and Implications | |
| Authors | Malihe Alikhani, Matthew Stone |
| Abstract | We study verbs in image{–}text corpora, contrasting \textit{caption} corpora, where texts are explicitly written to characterize image content, with \textit{depiction} corpora, where texts and images may stand in more general relations. Captions show a distinctively limited distribution of verbs, with strong preferences for specific tense, aspect, lexical aspect, and semantic field. These limitations, which appear in data elicited by a range of methods, restrict the utility of caption corpora to inform image retrieval, multimodal document generation, and perceptually-grounded semantic models. We suggest that these limitations reflect the discourse constraints in play when subjects write texts to accompany imagery, so we argue that future development of image{–}text corpora should work to increase the diversity of event descriptions, while looking explicitly at the different ways text and imagery can be coherently related. |
| Tasks | Image Retrieval |
| Published | 2019-06-01 |
| URL | https://www.aclweb.org/anthology/W19-1806/ |
| https://www.aclweb.org/anthology/W19-1806 | |
| PWC | https://paperswithcode.com/paper/caption-as-a-coherence-relation-evidence-and |
| Repo | |
| Framework | |
Neural Text Simplification of Clinical Letters with a Domain Specific Phrase Table
| Title | Neural Text Simplification of Clinical Letters with a Domain Specific Phrase Table |
| Authors | Matthew Shardlow, Raheel Nawaz |
| Abstract | Clinical letters are infamously impenetrable for the lay patient. This work uses neural text simplification methods to automatically improve the understandability of clinical letters for patients. We take existing neural text simplification software and augment it with a new phrase table that links complex medical terminology to simpler vocabulary by mining SNOMED-CT. In an evaluation task using crowdsourcing, we show that the results of our new system are ranked easier to understand (average rank 1.93) than using the original system (2.34) without our phrase table. We also show improvement against baselines including the original text (2.79) and using the phrase table without the neural text simplification software (2.94). Our methods can easily be transferred outside of the clinical domain by using domain-appropriate resources to provide effective neural text simplification for any domain without the need for costly annotation. |
| Tasks | Text Simplification |
| Published | 2019-07-01 |
| URL | https://www.aclweb.org/anthology/P19-1037/ |
| https://www.aclweb.org/anthology/P19-1037 | |
| PWC | https://paperswithcode.com/paper/neural-text-simplification-of-clinical |
| Repo | |
| Framework | |
Multimodal, Multilingual Grapheme-to-Phoneme Conversion for Low-Resource Languages
| Title | Multimodal, Multilingual Grapheme-to-Phoneme Conversion for Low-Resource Languages |
| Authors | James Route, Steven Hillis, Isak Czeresnia Etinger, Han Zhang, Alan W Black |
| Abstract | Grapheme-to-phoneme conversion (g2p) is the task of predicting the pronunciation of words from their orthographic representation. His- torically, g2p systems were transition- or rule- based, making generalization beyond a mono- lingual (high resource) domain impractical. Recently, neural architectures have enabled multilingual systems to generalize widely; however, all systems to date have been trained only on spelling-pronunciation pairs. We hy- pothesize that the sequences of IPA characters used to represent pronunciation do not capture its full nuance, especially when cleaned to fa- cilitate machine learning. We leverage audio data as an auxiliary modality in a multi-task training process to learn a more optimal inter- mediate representation of source graphemes; this is the first multimodal model proposed for multilingual g2p. Our approach is highly ef- fective: on our in-domain test set, our mul- timodal model reduces phoneme error rate to 2.46{%}, a more than 65{%} decrease compared to our implementation of a unimodal spelling- pronunciation model{—}which itself achieves state-of-the-art results on the Wiktionary test set. The advantages of the multimodal model generalize to wholly unseen languages, reduc- ing phoneme error rate on our out-of-domain test set to 6.39{%} from the unimodal 8.21{%}, a more than 20{%} relative decrease. Further- more, our training and test sets are composed primarily of low-resource languages, demon- strating that our multimodal approach remains useful when training data are constrained. |
| Tasks | |
| Published | 2019-11-01 |
| URL | https://www.aclweb.org/anthology/D19-6121/ |
| https://www.aclweb.org/anthology/D19-6121 | |
| PWC | https://paperswithcode.com/paper/multimodal-multilingual-grapheme-to-phoneme |
| Repo | |
| Framework | |
Argumentative Evidences Classification and Argument Scheme Detection Using Tree Kernels
| Title | Argumentative Evidences Classification and Argument Scheme Detection Using Tree Kernels |
| Authors | Davide Liga |
| Abstract | The purpose of this study is to deploy a novel methodology for classifying different argumentative support (supporting evidences) in arguments, without considering the context. The proposed methodology is based on the idea that the use of Tree Kernel algorithms can be a good way to discriminate between different types of argumentative stances without the need of highly engineered features. This can be useful in different Argumentation Mining sub-tasks. This work provides an example of classifier built using a Tree Kernel method, which can discriminate between different kinds of argumentative support with a high accuracy. The ability to distinguish different kinds of support is, in fact, a key step toward Argument Scheme classification. |
| Tasks | |
| Published | 2019-08-01 |
| URL | https://www.aclweb.org/anthology/W19-4511/ |
| https://www.aclweb.org/anthology/W19-4511 | |
| PWC | https://paperswithcode.com/paper/argumentative-evidences-classification-and |
| Repo | |
| Framework | |
Reinforcement-based denoising of distantly supervised NER with partial annotation
| Title | Reinforcement-based denoising of distantly supervised NER with partial annotation |
| Authors | Farhad Nooralahzadeh, Jan Tore L{\o}nning, Lilja {\O}vrelid |
| Abstract | Existing named entity recognition (NER) systems rely on large amounts of human-labeled data for supervision. However, obtaining large-scale annotated data is challenging particularly in specific domains like health-care, e-commerce and so on. Given the availability of domain specific knowledge resources, (e.g., ontologies, dictionaries), distant supervision is a solution to generate automatically labeled training data to reduce human effort. The outcome of distant supervision for NER, however, is often noisy. False positive and false negative instances are the main issues that reduce performance on this kind of auto-generated data. In this paper, we explore distant supervision in a supervised setup. We adopt a technique of partial annotation to address false negative cases and implement a reinforcement learning strategy with a neural network policy to identify false positive instances. Our results establish a new state-of-the-art on four benchmark datasets taken from different domains and different languages. We then go on to show that our model reduces the amount of manually annotated data required to perform NER in a new domain. |
| Tasks | Denoising, Named Entity Recognition |
| Published | 2019-11-01 |
| URL | https://www.aclweb.org/anthology/D19-6125/ |
| https://www.aclweb.org/anthology/D19-6125 | |
| PWC | https://paperswithcode.com/paper/reinforcement-based-denoising-of-distantly |
| Repo | |
| Framework | |
Neural Topic Model with Reinforcement Learning
| Title | Neural Topic Model with Reinforcement Learning |
| Authors | Lin Gui, Jia Leng, Gabriele Pergola, Yu Zhou, Ruifeng Xu, Yulan He |
| Abstract | In recent years, advances in neural variational inference have achieved many successes in text processing. Examples include neural topic models which are typically built upon variational autoencoder (VAE) with an objective of minimising the error of reconstructing original documents based on the learned latent topic vectors. However, minimising reconstruction errors does not necessarily lead to high quality topics. In this paper, we borrow the idea of reinforcement learning and incorporate topic coherence measures as reward signals to guide the learning of a VAE-based topic model. Furthermore, our proposed model is able to automatically separating background words dynamically from topic words, thus eliminating the pre-processing step of filtering infrequent and/or top frequent words, typically required for learning traditional topic models. Experimental results on the 20 Newsgroups and the NIPS datasets show superior performance both on perplexity and topic coherence measure compared to state-of-the-art neural topic models. |
| Tasks | Topic Models |
| Published | 2019-11-01 |
| URL | https://www.aclweb.org/anthology/D19-1350/ |
| https://www.aclweb.org/anthology/D19-1350 | |
| PWC | https://paperswithcode.com/paper/neural-topic-model-with-reinforcement |
| Repo | |
| Framework | |
Proceedings of the Workshop MultiLing 2019: Summarization Across Languages, Genres and Sources
| Title | Proceedings of the Workshop MultiLing 2019: Summarization Across Languages, Genres and Sources |
| Authors | |
| Abstract | |
| Tasks | |
| Published | 2019-09-01 |
| URL | https://www.aclweb.org/anthology/W19-8900/ |
| https://www.aclweb.org/anthology/W19-8900 | |
| PWC | https://paperswithcode.com/paper/proceedings-of-the-workshop-multiling-2019 |
| Repo | |
| Framework | |
