April 2, 2020

3173 words 15 mins read

Paper Group ANR 196

TNT-KID: Transformer-based Neural Tagger for Keyword Identification. Detecting New Word Meanings: A Comparison of Word Embedding Models in Spanish. Generación automática de frases literarias en español. Multimodal Story Generation on Plural Images. Deep learning predicts total knee replacement from magnetic resonance images. An Improved EEG Acquisi …

TNT-KID: Transformer-based Neural Tagger for Keyword Identification


Title	TNT-KID: Transformer-based Neural Tagger for Keyword Identification
Authors	Matej Martinc, Blaž Škrlj, Senja Pollak
Abstract	With growing amounts of available textual data, development of algorithms capable of automatic analysis, categorization and summarization of these data has become a necessity. In this research we present a novel algorithm for keyword identification, i.e., an extraction of one or multi-word phrases representing key aspects of a given document, called Transformer-based Neural Tagger for Keyword IDentification (TNT-KID). By adapting the transformer architecture for a specific task at hand and leveraging language model pretraining on a small domain specific corpus, the model is capable of overcoming deficiencies of both supervised and unsupervised state-of-the-art approaches to keyword extraction by offering competitive and robust performance on a variety of different datasets while requiring only a fraction of manually labeled data required by the best performing systems. This study also offers thorough error analysis with valuable insights into inner workings of the model and an ablation study measuring the influence of specific components of the keyword identification workflow on the overall performance.
Tasks	Keyword Extraction, Language Modelling
Published	2020-03-20
URL	https://arxiv.org/abs/2003.09166v1
PDF	https://arxiv.org/pdf/2003.09166v1.pdf
PWC	https://paperswithcode.com/paper/tnt-kid-transformer-based-neural-tagger-for
Repo
Framework

Detecting New Word Meanings: A Comparison of Word Embedding Models in Spanish


Title	Detecting New Word Meanings: A Comparison of Word Embedding Models in Spanish
Authors	Andrés Torres-Rivera, Juan-Manuel Torres-Moreno
Abstract	Semantic neologisms (SN) are defined as words that acquire a new word meaning while maintaining their form. Given the nature of this kind of neologisms, the task of identifying these new word meanings is currently performed manually by specialists at observatories of neology. To detect SN in a semi-automatic way, we developed a system that implements a combination of the following strategies: topic modeling, keyword extraction, and word sense disambiguation. The role of topic modeling is to detect the themes that are treated in the input text. Themes within a text give clues about the particular meaning of the words that are used, for example: viral has one meaning in the context of computer science (CS) and another when talking about health. To extract keywords, we used TextRank with POS tag filtering. With this method, we can obtain relevant words that are already part of the Spanish lexicon. We use a deep learning model to determine if a given keyword could have a new meaning. Embeddings that are different from all the known meanings (or topics) indicate that a word might be a valid SN candidate. In this study, we examine the following word embedding models: Word2Vec, Sense2Vec, and FastText. The models were trained with equivalent parameters using Wikipedia in Spanish as corpora. Then we used a list of words and their concordances (obtained from our database of neologisms) to show the different embeddings that each model yields. Finally, we present a comparison of these outcomes with the concordances of each word to show how we can determine if a word could be a valid candidate for SN.
Tasks	Keyword Extraction, Word Sense Disambiguation
Published	2020-01-12
URL	https://arxiv.org/abs/2001.05285v1
PDF	https://arxiv.org/pdf/2001.05285v1.pdf
PWC	https://paperswithcode.com/paper/detecting-new-word-meanings-a-comparison-of
Repo
Framework

Generación automática de frases literarias en español


Title	Generación automática de frases literarias en español
Authors	Luis-Gil Moreno-Jiménez, Juan-Manuel Torres-Moreno, Roseli S. Wedemann
Abstract	In this work we present a state of the art in the area of Computational Creativity (CC). In particular, we address the automatic generation of literary sentences in Spanish. We propose three models of text generation based mainly on statistical algorithms and shallow parsing analysis. We also present some rather encouraging preliminary results.
Tasks	Text Generation
Published	2020-01-17
URL	https://arxiv.org/abs/2001.11381v1
PDF	https://arxiv.org/pdf/2001.11381v1.pdf
PWC	https://paperswithcode.com/paper/generacion-automatica-de-frases-literarias-en
Repo
Framework

Multimodal Story Generation on Plural Images


Title	Multimodal Story Generation on Plural Images
Authors	Jing Jiang
Abstract	Traditionally, text generation models take in a sequence of text as input, and iteratively generate the next most probable word using pre-trained parameters. In this work, we propose the architecture to use images instead of text as the input of the text generation model, called StoryGen. In the architecture, we design a Relational Text Data Generator algorithm that relates different features from multiple images. The output samples from the model demonstrate the ability to generate meaningful paragraphs of text containing the extracted features from the input images.
Tasks	Text Generation
Published	2020-01-16
URL	https://arxiv.org/abs/2001.10980v1
PDF	https://arxiv.org/pdf/2001.10980v1.pdf
PWC	https://paperswithcode.com/paper/multimodal-story-generation-on-plural-images
Repo
Framework

Deep learning predicts total knee replacement from magnetic resonance images


Title	Deep learning predicts total knee replacement from magnetic resonance images
Authors	Aniket A. Tolpadi, Jinhee J. Lee, Valentina Pedoia, Sharmila Majumdar
Abstract	Knee Osteoarthritis (OA) is a common musculoskeletal disorder in the United States. When diagnosed at early stages, lifestyle interventions such as exercise and weight loss can slow OA progression, but at later stages, only an invasive option is available: total knee replacement (TKR). Though a generally successful procedure, only 2/3 of patients who undergo the procedure report their knees feeling ‘‘normal’’ post-operation, and complications can arise that require revision. This necessitates a model to identify a population at higher risk of TKR, particularly at less advanced stages of OA, such that appropriate treatments can be implemented that slow OA progression and delay TKR. Here, we present a deep learning pipeline that leverages MRI images and clinical and demographic information to predict TKR with AUC $0.834 \pm 0.036$ (p < 0.05). Most notably, the pipeline predicts TKR with AUC $0.943 \pm 0.057$ (p < 0.05) for patients without OA. Furthermore, we develop occlusion maps for case-control pairs in test data and compare regions used by the model in both, thereby identifying TKR imaging biomarkers. As such, this work takes strides towards a pipeline with clinical utility, and the biomarkers identified further our understanding of OA progression and eventual TKR onset.
Tasks
Published	2020-02-24
URL	https://arxiv.org/abs/2002.10591v1
PDF	https://arxiv.org/pdf/2002.10591v1.pdf
PWC	https://paperswithcode.com/paper/deep-learning-predicts-total-knee-replacement
Repo
Framework

An Improved EEG Acquisition Protocol Facilitates Localized Neural Activation


Title	An Improved EEG Acquisition Protocol Facilitates Localized Neural Activation
Authors	Jerrin Thomas Panachakel, Nandagopal Netrakanti Vinayak, Maanvi Nunna, A. G. Ramakrishnan, Kanishka Sharma
Abstract	This work proposes improvements in the electroencephalogram (EEG) recording protocols for motor imagery through the introduction of actual motor movement and/or somatosensory cues. The results obtained demonstrate the advantage of requiring the subjects to perform motor actions following the trials of imagery. By introducing motor actions in the protocol, the subjects are able to perform actual motor planning, rather than just visualizing the motor movement, thus greatly improving the ease with which the motor movements can be imagined. This study also probes the added advantage of administering somatosensory cues in the subject, as opposed to the conventional auditory/visual cues. These changes in the protocol show promise in terms of the aptness of the spatial filters obtained on the data, on application of the well-known common spatial pattern (CSP) algorithms. The regions highlighted by the spatial filters are more localized and consistent across the subjects when the protocol is augmented with somatosensory stimuli. Hence, we suggest that this may prove to be a better EEG acquisition protocol for detecting brain activation in response to intended motor commands in (clinically) paralyzed/locked-in patients.
Tasks	EEG
Published	2020-03-13
URL	https://arxiv.org/abs/2003.10212v1
PDF	https://arxiv.org/pdf/2003.10212v1.pdf
PWC	https://paperswithcode.com/paper/an-improved-eeg-acquisition-protocol
Repo
Framework

A Comprehensive Survey of Multilingual Neural Machine Translation


Title	A Comprehensive Survey of Multilingual Neural Machine Translation
Authors	Raj Dabre, Chenhui Chu, Anoop Kunchukuttan
Abstract	We present a survey on multilingual neural machine translation (MNMT), which has gained a lot of traction in the recent years. MNMT has been useful in improving translation quality as a result of translation knowledge transfer (transfer learning). MNMT is more promising and interesting than its statistical machine translation counterpart because end-to-end modeling and distributed representations open new avenues for research on machine translation. Many approaches have been proposed in order to exploit multilingual parallel corpora for improving translation quality. However, the lack of a comprehensive survey makes it difficult to determine which approaches are promising and hence deserve further exploration. In this paper, we present an in-depth survey of existing literature on MNMT. We first categorize various approaches based on their central use-case and then further categorize them based on resource scenarios, underlying modeling principles, core-issues and challenges. Wherever possible we address the strengths and weaknesses of several techniques by comparing them with each other. We also discuss the future directions that MNMT research might take. This paper is aimed towards both, beginners and experts in NMT. We hope this paper will serve as a starting point as well as a source of new ideas for researchers and engineers interested in MNMT.
Tasks	Machine Translation, Transfer Learning
Published	2020-01-04
URL	https://arxiv.org/abs/2001.01115v2
PDF	https://arxiv.org/pdf/2001.01115v2.pdf
PWC	https://paperswithcode.com/paper/a-comprehensive-survey-of-multilingual-neural
Repo
Framework

Morphological Word Segmentation on Agglutinative Languages for Neural Machine Translation


Title	Morphological Word Segmentation on Agglutinative Languages for Neural Machine Translation
Authors	Yirong Pan, Xiao Li, Yating Yang, Rui Dong
Abstract	Neural machine translation (NMT) has achieved impressive performance on machine translation task in recent years. However, in consideration of efficiency, a limited-size vocabulary that only contains the top-N highest frequency words are employed for model training, which leads to many rare and unknown words. It is rather difficult when translating from the low-resource and morphologically-rich agglutinative languages, which have complex morphology and large vocabulary. In this paper, we propose a morphological word segmentation method on the source-side for NMT that incorporates morphology knowledge to preserve the linguistic and semantic information in the word structure while reducing the vocabulary size at training time. It can be utilized as a preprocessing tool to segment the words in agglutinative languages for other natural language processing (NLP) tasks. Experimental results show that our morphologically motivated word segmentation method is better suitable for the NMT model, which achieves significant improvements on Turkish-English and Uyghur-Chinese machine translation tasks on account of reducing data sparseness and language complexity.
Tasks	Machine Translation
Published	2020-01-02
URL	https://arxiv.org/abs/2001.01589v1
PDF	https://arxiv.org/pdf/2001.01589v1.pdf
PWC	https://paperswithcode.com/paper/morphological-word-segmentation-on
Repo
Framework

Guidelines for enhancing data locality in selected machine learning algorithms


Title	Guidelines for enhancing data locality in selected machine learning algorithms
Authors	Imen Chakroun, Tom Vander Aa, Thomas J. Ashby
Abstract	To deal with the complexity of the new bigger and more complex generation of data, machine learning (ML) techniques are probably the first and foremost used. For ML algorithms to produce results in a reasonable amount of time, they need to be implemented efficiently. In this paper, we analyze one of the means to increase the performances of machine learning algorithms which is exploiting data locality. Data locality and access patterns are often at the heart of performance issues in computing systems due to the use of certain hardware techniques to improve performance. Altering the access patterns to increase locality can dramatically increase performance of a given algorithm. Besides, repeated data access can be seen as redundancy in data movement. Similarly, there can also be redundancy in the repetition of calculations. This work also identifies some of the opportunities for avoiding these redundancies by directly reusing computation results. We start by motivating why and how a more efficient implementation can be achieved by exploiting reuse in the memory hierarchy of modern instruction set processors. Next we document the possibilities of such reuse in some selected machine learning algorithms.
Tasks
Published	2020-01-09
URL	https://arxiv.org/abs/2001.03000v1
PDF	https://arxiv.org/pdf/2001.03000v1.pdf
PWC	https://paperswithcode.com/paper/guidelines-for-enhancing-data-locality-in
Repo
Framework

Gradient Boosting on Decision Trees for Mortality Prediction in Transcatheter Aortic Valve Implantation


Title	Gradient Boosting on Decision Trees for Mortality Prediction in Transcatheter Aortic Valve Implantation
Authors	Marco Mamprin, Jo M. Zelis, Pim A. L. Tonino, Svitlana Zinger, Peter H. N. de With
Abstract	Current prognostic risk scores in cardiac surgery are based on statistics and do not yet benefit from machine learning. Statistical predictors are not robust enough to correctly identify patients who would benefit from Transcatheter Aortic Valve Implantation (TAVI). This research aims to create a machine learning model to predict one-year mortality of a patient after TAVI. We adopt a modern gradient boosting on decision trees algorithm, specifically designed for categorical features. In combination with a recent technique for model interpretations, we developed a feature analysis and selection stage, enabling to identify the most important features for the prediction. We base our prediction model on the most relevant features, after interpreting and discussing the feature analysis results with clinical experts. We validated our model on 270 TAVI cases, reaching an AUC of 0.83. Our approach outperforms several widespread prognostic risk scores, such as logistic EuroSCORE II, the STS risk score and the TAVI2-score, which are broadly adopted by cardiologists worldwide.
Tasks	Mortality Prediction
Published	2020-01-08
URL	https://arxiv.org/abs/2001.02431v1
PDF	https://arxiv.org/pdf/2001.02431v1.pdf
PWC	https://paperswithcode.com/paper/gradient-boosting-on-decision-trees-for
Repo
Framework

The State of Service Robots: Current Bottlenecks in Object Perception and Manipulation


Title	The State of Service Robots: Current Bottlenecks in Object Perception and Manipulation
Authors	S. Hamidreza Kasaei, Jorik Melsen, Floris van Beers, Christiaan Steenkist, Klemen Voncina
Abstract	Service robots are appearing more and more in our daily life. The development of service robots combines multiple fields of research, from object perception to object manipulation. The state-of-the-art continues to improve to make a proper coupling between object perception and manipulation. This coupling is necessary for service robots not only to perform various tasks in a reasonable amount of time but also to adapt to new environments through time and interact with non-expert human users safely. Nowadays, robots are able to recognize various objects, and quickly plan a collision-free trajectory to grasp a target object. While there are many successes, the robot should be painstakingly coded in advance to perform a set of predefined tasks. Besides, in most of the cases, there is a reliance on large amounts of training data. Therefore, the knowledge of such robots is fixed after the training phase, and any changes in the environment require complicated, time-consuming, and expensive robot re-programming by human experts. Therefore, these approaches are still too rigid for real-life applications in unstructured environments, where a significant portion of the environment is unknown and cannot be directly sensed or controlled. In this paper, we review advances in service robots from object perception to complex object manipulation and shed a light on the current challenges and bottlenecks.
Tasks
Published	2020-03-18
URL	https://arxiv.org/abs/2003.08151v1
PDF	https://arxiv.org/pdf/2003.08151v1.pdf
PWC	https://paperswithcode.com/paper/the-state-of-service-robots-current
Repo
Framework

Recurrent Attention Model with Log-Polar Mapping is Robust against Adversarial Attacks


Title	Recurrent Attention Model with Log-Polar Mapping is Robust against Adversarial Attacks
Authors	Taro Kiritani, Koji Ono
Abstract	Convolutional neural networks are vulnerable to small $\ell^p$ adversarial attacks, while the human visual system is not. Inspired by neural networks in the eye and the brain, we developed a novel artificial neural network model that recurrently collects data with a log-polar field of view that is controlled by attention. We demonstrate the effectiveness of this design as a defense against SPSA and PGD adversarial attacks. It also has beneficial properties observed in the animal visual system, such as reflex-like pathways for low-latency inference, fixed amount of computation independent of image size, and rotation and scale invariance. The code for experiments is available at https://gitlab.com/exwzd-public/kiritani_ono_2020.
Tasks
Published	2020-02-13
URL	https://arxiv.org/abs/2002.05388v1
PDF	https://arxiv.org/pdf/2002.05388v1.pdf
PWC	https://paperswithcode.com/paper/recurrent-attention-model-with-log-polar
Repo
Framework

The SPECIAL-K Personal Data Processing Transparency and Compliance Platform


Title	The SPECIAL-K Personal Data Processing Transparency and Compliance Platform
Authors	Sabrina Kirrane, Javier D. Fernández, Piero Bonatti, Uros Milosevic, Axel Polleres, Rigo Wenning
Abstract	The European General Data Protection Regulation (GDPR) brings new challenges for companies, who must provide transparency with respect to personal data processing and sharing within and between organisations. Additionally companies need to demonstrate that their systems and business processes comply with usage constraints specified by data subjects. This paper first presents the Linked Data ontologies and vocabularies developed within the SPECIAL EU H2020 project, which can be used to represent data usage policies and data processing and sharing events, including the consent provided by the data subject and subsequent changes to or revocation of said consent. Following on from this, we propose a concrete transparency and compliance architecture, referred to as SPECIAL-K, that can automatically verify that data processing and sharing complies with the relevant usage control policies. Our evaluation, based on a new transparency and compliance benchmark, shows the efficiency and scalability of the system with increasing number of events and users, covering a wide range of real-world streaming and batch processing scenarios.
Tasks
Published	2020-01-26
URL	https://arxiv.org/abs/2001.09461v1
PDF	https://arxiv.org/pdf/2001.09461v1.pdf
PWC	https://paperswithcode.com/paper/the-special-k-personal-data-processing
Repo
Framework

Generative Adversarial Zero-Shot Relational Learning for Knowledge Graphs


Title	Generative Adversarial Zero-Shot Relational Learning for Knowledge Graphs
Authors	Pengda Qin, Xin Wang, Wenhu Chen, Chunyun Zhang, Weiran Xu, William Yang Wang
Abstract	Large-scale knowledge graphs (KGs) are shown to become more important in current information systems. To expand the coverage of KGs, previous studies on knowledge graph completion need to collect adequate training instances for newly-added relations. In this paper, we consider a novel formulation, zero-shot learning, to free this cumbersome curation. For newly-added relations, we attempt to learn their semantic features from their text descriptions and hence recognize the facts of unseen relations with no examples being seen. For this purpose, we leverage Generative Adversarial Networks (GANs) to establish the connection between text and knowledge graph domain: The generator learns to generate the reasonable relation embeddings merely with noisy text descriptions. Under this setting, zero-shot learning is naturally converted to a traditional supervised classification task. Empirically, our method is model-agnostic that could be potentially applied to any version of KG embeddings, and consistently yields performance improvements on NELL and Wiki dataset.
Tasks	Knowledge Graph Completion, Knowledge Graphs, Relational Reasoning, Zero-Shot Learning
Published	2020-01-08
URL	https://arxiv.org/abs/2001.02332v1
PDF	https://arxiv.org/pdf/2001.02332v1.pdf
PWC	https://paperswithcode.com/paper/generative-adversarial-zero-shot-relational
Repo
Framework

Trust Your Model: Iterative Label Improvement and Robust Training by Confidence Based Filtering and Dataset Partitioning


Title	Trust Your Model: Iterative Label Improvement and Robust Training by Confidence Based Filtering and Dataset Partitioning
Authors	Christian Haase-Schütz, Rainer Stal, Heinz Hertlein, Bernhard Sick
Abstract	State-of-the-art, high capacity deep neural networks not only require large amounts of labelled training data, they are also highly susceptible to label errors in this data, typically resulting in large efforts and costs and therefore limiting the applicability of deep learning. To alleviate this issue, we propose a novel meta training and labelling scheme that is able to use inexpensive unlabelled data by taking advantage of the generalization power of deep neural networks. We show experimentally that by solely relying on one network architecture and our proposed scheme of iterative training and prediction steps, both label quality and resulting model accuracy can be improved significantly. Our method achieves state-of-the-art results, while being architecture agnostic and therefore broadly applicable. Compared to other methods dealing with erroneous labels, our approach does neither require another network to be trained, nor does it necessarily need an additional, highly accurate reference label set. Instead of removing samples from a labelled set, our technique uses additional sensor data without the need for manual labelling.
Tasks
Published	2020-02-07
URL	https://arxiv.org/abs/2002.02705v2
PDF	https://arxiv.org/pdf/2002.02705v2.pdf
PWC	https://paperswithcode.com/paper/trust-your-model-iterative-label-improvement
Repo
Framework