October 20, 2019

3368 words 16 mins read

Paper Group ANR 61

Metadata Enrichment of Multi-Disciplinary Digital Library: A Semantic-based Approach. COREclust: a new package for a robust and scalable analysis of complex data. Informed Group-Sparse Representation for Singing Voice Separation. Pangloss: Fast Entity Linking in Noisy Text Environments. Improving Temporal Relation Extraction with a Globally Acquire …

Metadata Enrichment of Multi-Disciplinary Digital Library: A Semantic-based Approach


Title	Metadata Enrichment of Multi-Disciplinary Digital Library: A Semantic-based Approach
Authors	Hussein T. Al-Natsheh, Lucie Martinet, Fabrice Muhlenbach, Fabien Rico, Djamel A. Zighed
Abstract	In the scientific digital libraries, some papers from different research communities can be described by community-dependent keywords even if they share a semantically similar topic. Articles that are not tagged with enough keyword variations are poorly indexed in any information retrieval system which limits potentially fruitful exchanges between scientific disciplines. In this paper, we introduce a novel experimentally designed pipeline for multi-label semantic-based tagging developed for open-access metadata digital libraries. The approach starts by learning from a standard scientific categorization and a sample of topic tagged articles to find semantically relevant articles and enrich its metadata accordingly. Our proposed pipeline aims to enable researchers reaching articles from various disciplines that tend to use different terminologies. It allows retrieving semantically relevant articles given a limited known variation of search terms. In addition to achieving an accuracy that is higher than an expanded query based method using a topic synonym set extracted from a semantic network, our experiments also show a higher computational scalability versus other comparable techniques. We created a new benchmark extracted from the open-access metadata of a scientific digital library and published it along with the experiment code to allow further research in the topic.
Tasks	Information Retrieval
Published	2018-06-21
URL	http://arxiv.org/abs/1806.08202v1
PDF	http://arxiv.org/pdf/1806.08202v1.pdf
PWC	https://paperswithcode.com/paper/metadata-enrichment-of-multi-disciplinary
Repo
Framework

COREclust: a new package for a robust and scalable analysis of complex data


Title	COREclust: a new package for a robust and scalable analysis of complex data
Authors	Camille Champion, Anne-Claire Brunet, Jean-Michel Loubes, Laurent Risser
Abstract	In this paper, we present a new R package COREclust dedicated to the detection of representative variables in high dimensional spaces with a potentially limited number of observations. Variable sets detection is based on an original graph clustering strategy denoted CORE-clustering algorithm that detects CORE-clusters, i.e. variable sets having a user defined size range and in which each variable is very similar to at least another variable. Representative variables are then robustely estimate as the CORE-cluster centers. This strategy is entirely coded in C++ and wrapped by R using the Rcpp package. A particular effort has been dedicated to keep its algorithmic cost reasonable so that it can be used on large datasets. After motivating our work, we will explain the CORE-clustering algorithm as well as a greedy extension of this algorithm. We will then present how to use it and results obtained on synthetic and real data.
Tasks	Graph Clustering
Published	2018-05-25
URL	http://arxiv.org/abs/1805.10211v1
PDF	http://arxiv.org/pdf/1805.10211v1.pdf
PWC	https://paperswithcode.com/paper/coreclust-a-new-package-for-a-robust-and
Repo
Framework

Informed Group-Sparse Representation for Singing Voice Separation


Title	Informed Group-Sparse Representation for Singing Voice Separation
Authors	Tak-Shing T. Chan, Yi-Hsuan Yang
Abstract	Singing voice separation attempts to separate the vocal and instrumental parts of a music recording, which is a fundamental problem in music information retrieval. Recent work on singing voice separation has shown that the low-rank representation and informed separation approaches are both able to improve separation quality. However, low-rank optimizations are computationally inefficient due to the use of singular value decompositions. Therefore, in this paper, we propose a new linear-time algorithm called informed group-sparse representation, and use it to separate the vocals from music using pitch annotations as side information. Experimental results on the iKala dataset confirm the efficacy of our approach, suggesting that the music accompaniment follows a group-sparse structure given a pre-trained instrumental dictionary. We also show how our work can be easily extended to accommodate multiple dictionaries using the DSD100 dataset.
Tasks	Information Retrieval, Music Information Retrieval
Published	2018-01-09
URL	http://arxiv.org/abs/1801.03815v1
PDF	http://arxiv.org/pdf/1801.03815v1.pdf
PWC	https://paperswithcode.com/paper/informed-group-sparse-representation-for
Repo
Framework

Pangloss: Fast Entity Linking in Noisy Text Environments


Title	Pangloss: Fast Entity Linking in Noisy Text Environments
Authors	Michael Conover, Matthew Hayes, Scott Blackburn, Pete Skomoroch, Sam Shah
Abstract	Entity linking is the task of mapping potentially ambiguous terms in text to their constituent entities in a knowledge base like Wikipedia. This is useful for organizing content, extracting structured data from textual documents, and in machine learning relevance applications like semantic search, knowledge graph construction, and question answering. Traditionally, this work has focused on text that has been well-formed, like news articles, but in common real world datasets such as messaging, resumes, or short-form social media, non-grammatical, loosely-structured text adds a new dimension to this problem. This paper presents Pangloss, a production system for entity disambiguation on noisy text. Pangloss combines a probabilistic linear-time key phrase identification algorithm with a semantic similarity engine based on context-dependent document embeddings to achieve better than state-of-the-art results (>5% in F1) compared to other research or commercially available systems. In addition, Pangloss leverages a local embedded database with a tiered architecture to house its statistics and metadata, which allows rapid disambiguation in streaming contexts and on-device disambiguation in low-memory environments such as mobile phones.
Tasks	Entity Disambiguation, Entity Linking, graph construction, Question Answering, Semantic Similarity, Semantic Textual Similarity
Published	2018-07-16
URL	http://arxiv.org/abs/1807.06036v1
PDF	http://arxiv.org/pdf/1807.06036v1.pdf
PWC	https://paperswithcode.com/paper/pangloss-fast-entity-linking-in-noisy-text
Repo
Framework

Improving Temporal Relation Extraction with a Globally Acquired Statistical Resource


Title	Improving Temporal Relation Extraction with a Globally Acquired Statistical Resource
Authors	Qiang Ning, Hao Wu, Haoruo Peng, Dan Roth
Abstract	Extracting temporal relations (before, after, overlapping, etc.) is a key aspect of understanding events described in natural language. We argue that this task would gain from the availability of a resource that provides prior knowledge in the form of the temporal order that events usually follow. This paper develops such a resource – a probabilistic knowledge base acquired in the news domain – by extracting temporal relations between events from the New York Times (NYT) articles over a 20-year span (1987–2007). We show that existing temporal extraction systems can be improved via this resource. As a byproduct, we also show that interesting statistics can be retrieved from this resource, which can potentially benefit other time-aware tasks. The proposed system and resource are both publicly available.
Tasks	Relation Extraction
Published	2018-04-17
URL	http://arxiv.org/abs/1804.06020v1
PDF	http://arxiv.org/pdf/1804.06020v1.pdf
PWC	https://paperswithcode.com/paper/improving-temporal-relation-extraction-with-a
Repo
Framework

Grow and Prune Compact, Fast, and Accurate LSTMs


Title	Grow and Prune Compact, Fast, and Accurate LSTMs
Authors	Xiaoliang Dai, Hongxu Yin, Niraj K. Jha
Abstract	Long short-term memory (LSTM) has been widely used for sequential data modeling. Researchers have increased LSTM depth by stacking LSTM cells to improve performance. This incurs model redundancy, increases run-time delay, and makes the LSTMs more prone to overfitting. To address these problems, we propose a hidden-layer LSTM (H-LSTM) that adds hidden layers to LSTM’s original one level non-linear control gates. H-LSTM increases accuracy while employing fewer external stacked layers, thus reducing the number of parameters and run-time latency significantly. We employ grow-and-prune (GP) training to iteratively adjust the hidden layers through gradient-based growth and magnitude-based pruning of connections. This learns both the weights and the compact architecture of H-LSTM control gates. We have GP-trained H-LSTMs for image captioning and speech recognition applications. For the NeuralTalk architecture on the MSCOCO dataset, our three models reduce the number of parameters by 38.7x [floating-point operations (FLOPs) by 45.5x], run-time latency by 4.5x, and improve the CIDEr score by 2.6. For the DeepSpeech2 architecture on the AN4 dataset, our two models reduce the number of parameters by 19.4x (FLOPs by 23.5x), run-time latency by 15.7%, and the word error rate from 12.9% to 8.7%. Thus, GP-trained H-LSTMs can be seen to be compact, fast, and accurate.
Tasks	Image Captioning, Speech Recognition
Published	2018-05-30
URL	http://arxiv.org/abs/1805.11797v2
PDF	http://arxiv.org/pdf/1805.11797v2.pdf
PWC	https://paperswithcode.com/paper/grow-and-prune-compact-fast-and-accurate
Repo
Framework

Satyam: Democratizing Groundtruth for Machine Vision


Title	Satyam: Democratizing Groundtruth for Machine Vision
Authors	Hang Qiu, Krishna Chintalapudi, Ramesh Govindan
Abstract	The democratization of machine learning (ML) has led to ML-based machine vision systems for autonomous driving, traffic monitoring, and video surveillance. However, true democratization cannot be achieved without greatly simplifying the process of collecting groundtruth for training and testing these systems. This groundtruth collection is necessary to ensure good performance under varying conditions. In this paper, we present the design and evaluation of Satyam, a first-of-its-kind system that enables a layperson to launch groundtruth collection tasks for machine vision with minimal effort. Satyam leverages a crowdtasking platform, Amazon Mechanical Turk, and automates several challenging aspects of groundtruth collection: creating and launching of custom web-UI tasks for obtaining the desired groundtruth, controlling result quality in the face of spammers and untrained workers, adapting prices to match task complexity, filtering spammers and workers with poor performance, and processing worker payments. We validate Satyam using several popular benchmark vision datasets, and demonstrate that groundtruth obtained by Satyam is comparable to that obtained from trained experts and provides matching ML performance when used for training.
Tasks	Autonomous Driving
Published	2018-11-08
URL	http://arxiv.org/abs/1811.03621v1
PDF	http://arxiv.org/pdf/1811.03621v1.pdf
PWC	https://paperswithcode.com/paper/satyam-democratizing-groundtruth-for-machine
Repo
Framework

Visual Dynamics: Stochastic Future Generation via Layered Cross Convolutional Networks


Title	Visual Dynamics: Stochastic Future Generation via Layered Cross Convolutional Networks
Authors	Tianfan Xue, Jiajun Wu, Katherine L. Bouman, William T. Freeman
Abstract	We study the problem of synthesizing a number of likely future frames from a single input image. In contrast to traditional methods that have tackled this problem in a deterministic or non-parametric way, we propose to model future frames in a probabilistic manner. Our probabilistic model makes it possible for us to sample and synthesize many possible future frames from a single input image. To synthesize realistic movement of objects, we propose a novel network structure, namely a Cross Convolutional Network; this network encodes image and motion information as feature maps and convolutional kernels, respectively. In experiments, our model performs well on synthetic data, such as 2D shapes and animated game sprites, and on real-world video frames. We present analyses of the learned network representations, showing it is implicitly learning a compact encoding of object appearance and motion. We also demonstrate a few of its applications, including visual analogy-making and video extrapolation.
Tasks
Published	2018-07-24
URL	https://arxiv.org/abs/1807.09245v3
PDF	https://arxiv.org/pdf/1807.09245v3.pdf
PWC	https://paperswithcode.com/paper/visual-dynamics-stochastic-future-generation
Repo
Framework

The Trace Criterion for Kernel Bandwidth Selection for Support Vector Data Description


Title	The Trace Criterion for Kernel Bandwidth Selection for Support Vector Data Description
Authors	Arin Chaudhuri, Carol Sadek, Deovrat Kakde, Wenhao Hu, Hansi Jiang, Seunghyun Kong, Yuewei Liao, Sergiy Peredriy, Haoyu Wang
Abstract	Support vector data description (SVDD) is a popular anomaly detection technique. The SVDD classifier partitions the whole data space into an inlier region, which consists of the region near the training data, and an outlier region, which consists of points away from the training data. The computation of the SVDD classifier requires a kernel function, for which the Gaussian kernel is a common choice. The Gaussian kernel has a bandwidth parameter, and it is important to set the value of this parameter correctly for good results. A small bandwidth leads to overfitting such that the resulting SVDD classifier overestimates the number of anomalies, whereas a large bandwidth leads to underfitting and an inability to detect many anomalies. In this paper, we present a new unsupervised method for selecting the Gaussian kernel bandwidth. Our method exploits a low-rank representation of the kernel matrix to suggest a kernel bandwidth value. Our new technique is competitive with the current state of the art for low-dimensional data and performs extremely well for many classes of high-dimensional data. Because the mathematical formulation of SVDD is identical with the mathematical formulation of one-class support vector machines (OCSVM) when the Gaussian kernel is used, our method is equally applicable to Gaussian kernel bandwidth tuning for OCSVM.
Tasks	Anomaly Detection
Published	2018-11-15
URL	https://arxiv.org/abs/1811.06838v3
PDF	https://arxiv.org/pdf/1811.06838v3.pdf
PWC	https://paperswithcode.com/paper/the-trace-criterion-for-kernel-bandwidth
Repo
Framework

Ultra-Fine Entity Typing


Title	Ultra-Fine Entity Typing
Authors	Eunsol Choi, Omer Levy, Yejin Choi, Luke Zettlemoyer
Abstract	We introduce a new entity typing task: given a sentence with an entity mention, the goal is to predict a set of free-form phrases (e.g. skyscraper, songwriter, or criminal) that describe appropriate types for the target entity. This formulation allows us to use a new type of distant supervision at large scale: head words, which indicate the type of the noun phrases they appear in. We show that these ultra-fine types can be crowd-sourced, and introduce new evaluation sets that are much more diverse and fine-grained than existing benchmarks. We present a model that can predict open types, and is trained using a multitask objective that pools our new head-word supervision with prior supervision from entity linking. Experimental results demonstrate that our model is effective in predicting entity types at varying granularity; it achieves state of the art performance on an existing fine-grained entity typing benchmark, and sets baselines for our newly-introduced datasets. Our data and model can be downloaded from: http://nlp.cs.washington.edu/entity_type
Tasks	Entity Linking, Entity Typing
Published	2018-07-13
URL	http://arxiv.org/abs/1807.04905v1
PDF	http://arxiv.org/pdf/1807.04905v1.pdf
PWC	https://paperswithcode.com/paper/ultra-fine-entity-typing
Repo
Framework

Prediction of the Optimal Threshold Value in DF Relay Selection Schemes Based on Artificial Neural Networks


Title	Prediction of the Optimal Threshold Value in DF Relay Selection Schemes Based on Artificial Neural Networks
Authors	Ferdi Kara, Hakan Kaya, Okan Erkaymaz, Ertan Ozturk
Abstract	In wireless communications, the cooperative communication (CC) technology promises performance gains compared to traditional Single-Input Single Output (SISO) techniques. Therefore, the CC technique is one of the nominees for 5G networks. In the Decode-and-Forward (DF) relaying scheme which is one of the CC techniques, determination of the threshold value at the relay has a key role for the system performance and power usage. In this paper, we propose prediction of the optimal threshold values for the best relay selection scheme in cooperative communications, based on Artificial Neural Networks (ANNs) for the first time in literature. The average link qualities and number of relays have been used as inputs in the prediction of optimal threshold values using Artificial Neural Networks (ANNs): Multi-Layer Perceptron (MLP) and Radial Basis Function (RBF) networks. The MLP network has better performance from the RBF network on the prediction of optimal threshold value when the same number of neurons is used at the hidden layer for both networks. Besides, the optimal threshold values obtained using ANNs are verified by the optimal threshold values obtained numerically using the closed form expression derived for the system. The results show that the optimal threshold values obtained by ANNs on the best relay selection scheme provide a minimum Bit-Error-Rate (BER) because of the reduction of the probability that error propagation may occur. Also, for the same BER performance goal, prediction of optimal threshold values provides 2dB less power usage, which is great gain in terms of green communicationBER performance goal, prediction of optimal threshold values provides 2dB less power usage, which is great gain in terms of green communication.
Tasks
Published	2018-01-18
URL	http://arxiv.org/abs/1801.05984v1
PDF	http://arxiv.org/pdf/1801.05984v1.pdf
PWC	https://paperswithcode.com/paper/prediction-of-the-optimal-threshold-value-in
Repo
Framework

Multi-task learning to improve natural language understanding


Title	Multi-task learning to improve natural language understanding
Authors	Stefan Constantin, Jan Niehues, Alex Waibel
Abstract	Recently advancements in sequence-to-sequence neural network architectures have led to an improved natural language understanding. When building a neural network-based Natural Language Understanding component, one main challenge is to collect enough training data. The generation of a synthetic dataset is an inexpensive and quick way to collect data. Since this data often has less variety than real natural language, neural networks often have problems to generalize to unseen utterances during testing. In this work, we address this challenge by using multi-task learning. We train out-of-domain real data alongside in-domain synthetic data to improve natural language understanding. We evaluate this approach in the domain of airline travel information with two synthetic datasets. As out-of-domain real data, we test two datasets based on the subtitles of movies and series. By using an attention-based encoder-decoder model, we were able to improve the F1-score over strong baselines from 80.76 % to 84.98 % in the smaller synthetic dataset.
Tasks	Multi-Task Learning
Published	2018-12-17
URL	http://arxiv.org/abs/1812.06876v2
PDF	http://arxiv.org/pdf/1812.06876v2.pdf
PWC	https://paperswithcode.com/paper/multi-task-learning-to-improve-natural
Repo
Framework

Differential Expression Analysis of Dynamical Sequencing Count Data with a Gamma Markov Chain


Title	Differential Expression Analysis of Dynamical Sequencing Count Data with a Gamma Markov Chain
Authors	Ehsan Hajiramezanali, Siamak Zamani Dadaneh, Paul de Figueiredo, Sing-Hoi Sze, Mingyuan Zhou, Xiaoning Qian
Abstract	Next-generation sequencing (NGS) to profile temporal changes in living systems is gaining more attention for deriving better insights into the underlying biological mechanisms compared to traditional static sequencing experiments. Nonetheless, the majority of existing statistical tools for analyzing NGS data lack the capability of exploiting the richer information embedded in temporal data. Several recent tools have been developed to analyze such data but they typically impose strict model assumptions, such as smoothness on gene expression dynamic changes. To capture a broader range of gene expression dynamic patterns, we develop the gamma Markov negative binomial (GMNB) model that integrates a gamma Markov chain into a negative binomial distribution model, allowing flexible temporal variation in NGS count data. Using Bayes factors, GMNB enables more powerful temporal gene differential expression analysis across different phenotypes or treatment conditions. In addition, it naturally handles the heterogeneity of sequencing depth in different samples, removing the need for ad-hoc normalization. Efficient Gibbs sampling inference of the GMNB model parameters is achieved by exploiting novel data augmentation techniques. Extensive experiments on both simulated and real-world RNA-seq data show that GMNB outperforms existing methods in both receiver operating characteristic (ROC) and precision-recall (PR) curves of differential expression analysis results.
Tasks	Data Augmentation
Published	2018-03-07
URL	http://arxiv.org/abs/1803.02527v1
PDF	http://arxiv.org/pdf/1803.02527v1.pdf
PWC	https://paperswithcode.com/paper/differential-expression-analysis-of-dynamical
Repo
Framework

Sacrificing Accuracy for Reduced Computation: Cascaded Inference Based on Softmax Confidence


Title	Sacrificing Accuracy for Reduced Computation: Cascaded Inference Based on Softmax Confidence
Authors	Konstantin Berestizshevsky, Guy Even
Abstract	We study the tradeoff between computational effort and accuracy in a cascade of deep neural networks. During inference, early termination in the cascade is controlled by confidence levels derived directly from the softmax outputs of intermediate classifiers. The advantage of early termination is that classification is performed using less computation, thus adjusting the computational effort to the complexity of the input. Moreover, dynamic modification of confidence thresholds allow one to trade accuracy for computational effort without requiring retraining. Basing of early termination on softmax classifier outputs is justified by experimentation that demonstrates an almost linear relation between confidence levels in intermediate classifiers and accuracy. Our experimentation with architectures based on ResNet obtained the following results. (i) A speedup of 1.5 that sacrifices 1.4% accuracy with respect to the CIFAR-10 test set. (ii) A speedup of 1.19 that sacrifices 0.7% accuracy with respect to the CIFAR-100 test set. (iii) A speedup of 2.16 that sacrifices 1.4% accuracy with respect to the SVHN test set.
Tasks
Published	2018-05-28
URL	http://arxiv.org/abs/1805.10982v1
PDF	http://arxiv.org/pdf/1805.10982v1.pdf
PWC	https://paperswithcode.com/paper/sacrificing-accuracy-for-reduced-computation
Repo
Framework

Improved Calibration of Numerical Integration Error in Sigma-Point Filters


Title	Improved Calibration of Numerical Integration Error in Sigma-Point Filters
Authors	Jakub Prüher, Toni Karvonen, Chris J. Oates, Ondřej Straka, Simo Särkkä
Abstract	The sigma-point filters, such as the UKF, which exploit numerical quadrature to obtain an additional order of accuracy in the moment transformation step, are popular alternatives to the ubiquitous EKF. The classical quadrature rules used in the sigma-point filters are motivated via polynomial approximation of the integrand, however in the applied context these assumptions cannot always be justified. As a result, quadrature error can introduce bias into estimated moments, for which there is no compensatory mechanism in the classical sigma-point filters. This can lead in turn to estimates and predictions that are poorly calibrated. In this article, we investigate the Bayes-Sard quadrature method in the context of sigma-point filters, which enables uncertainty due to quadrature error to be formalised within a probabilistic model. Our first contribution is to derive the well-known classical quadratures as special cases of the Bayes-Sard quadrature method. Then a general-purpose moment transform is developed and utilised in the design of novel sigma-point filters, so that uncertainty due to quadrature error is explicitly quantified. Numerical experiments on a challenging tracking example with misspecified initial conditions show that the additional uncertainty quantification built into our method leads to better-calibrated state estimates with improved RMSE.
Tasks	Calibration
Published	2018-11-28
URL	https://arxiv.org/abs/1811.11474v2
PDF	https://arxiv.org/pdf/1811.11474v2.pdf
PWC	https://paperswithcode.com/paper/improved-calibration-of-numerical-integration
Repo
Framework