October 17, 2019

3066 words 15 mins read

Paper Group ANR 847

What’s Going On in Neural Constituency Parsers? An Analysis. A Tour of Reinforcement Learning: The View from Continuous Control. A latent topic model for mining heterogenous non-randomly missing electronic health records data. Learning Visual Question Answering by Bootstrapping Hard Attention. Domain Alignment with Triplets. A Neurobiological Evalu …

What’s Going On in Neural Constituency Parsers? An Analysis


Title	What’s Going On in Neural Constituency Parsers? An Analysis
Authors	David Gaddy, Mitchell Stern, Dan Klein
Abstract	A number of differences have emerged between modern and classic approaches to constituency parsing in recent years, with structural components like grammars and feature-rich lexicons becoming less central while recurrent neural network representations rise in popularity. The goal of this work is to analyze the extent to which information provided directly by the model structure in classical systems is still being captured by neural methods. To this end, we propose a high-performance neural model (92.08 F1 on PTB) that is representative of recent work and perform a series of investigative experiments. We find that our model implicitly learns to encode much of the same information that was explicitly provided by grammars and lexicons in the past, indicating that this scaffolding can largely be subsumed by powerful general-purpose neural machinery.
Tasks	Constituency Parsing
Published	2018-04-20
URL	http://arxiv.org/abs/1804.07853v1
PDF	http://arxiv.org/pdf/1804.07853v1.pdf
PWC	https://paperswithcode.com/paper/whats-going-on-in-neural-constituency-parsers
Repo
Framework

A Tour of Reinforcement Learning: The View from Continuous Control


Title	A Tour of Reinforcement Learning: The View from Continuous Control
Authors	Benjamin Recht
Abstract	This manuscript surveys reinforcement learning from the perspective of optimization and control with a focus on continuous control applications. It surveys the general formulation, terminology, and typical experimental implementations of reinforcement learning and reviews competing solution paradigms. In order to compare the relative merits of various techniques, this survey presents a case study of the Linear Quadratic Regulator (LQR) with unknown dynamics, perhaps the simplest and best-studied problem in optimal control. The manuscript describes how merging techniques from learning theory and control can provide non-asymptotic characterizations of LQR performance and shows that these characterizations tend to match experimental behavior. In turn, when revisiting more complex applications, many of the observed phenomena in LQR persist. In particular, theory and experiment demonstrate the role and importance of models and the cost of generality in reinforcement learning algorithms. This survey concludes with a discussion of some of the challenges in designing learning systems that safely and reliably interact with complex and uncertain environments and how tools from reinforcement learning and control might be combined to approach these challenges.
Tasks	Continuous Control
Published	2018-06-25
URL	http://arxiv.org/abs/1806.09460v2
PDF	http://arxiv.org/pdf/1806.09460v2.pdf
PWC	https://paperswithcode.com/paper/a-tour-of-reinforcement-learning-the-view
Repo
Framework

A latent topic model for mining heterogenous non-randomly missing electronic health records data


Title	A latent topic model for mining heterogenous non-randomly missing electronic health records data
Authors	Yue Li, Manolis Kellis
Abstract	Electronic health records (EHR) are rich heterogeneous collection of patient health information, whose broad adoption provides great opportunities for systematic health data mining. However, heterogeneous EHR data types and biased ascertainment impose computational challenges. Here, we present mixEHR, an unsupervised generative model integrating collaborative filtering and latent topic models, which jointly models the discrete distributions of data observation bias and actual data using latent disease-topic distributions. We apply mixEHR on 12.8 million phenotypic observations from the MIMIC dataset, and use it to reveal latent disease topics, interpret EHR results, impute missing data, and predict mortality in intensive care units. Using both simulation and real data, we show that mixEHR outperforms previous methods and reveals meaningful multi-disease insights.
Tasks	Topic Models
Published	2018-11-01
URL	http://arxiv.org/abs/1811.00464v1
PDF	http://arxiv.org/pdf/1811.00464v1.pdf
PWC	https://paperswithcode.com/paper/a-latent-topic-model-for-mining-heterogenous
Repo
Framework

Learning Visual Question Answering by Bootstrapping Hard Attention


Title	Learning Visual Question Answering by Bootstrapping Hard Attention
Authors	Mateusz Malinowski, Carl Doersch, Adam Santoro, Peter Battaglia
Abstract	Attention mechanisms in biological perception are thought to select subsets of perceptual information for more sophisticated processing which would be prohibitive to perform on all sensory inputs. In computer vision, however, there has been relatively little exploration of hard attention, where some information is selectively ignored, in spite of the success of soft attention, where information is re-weighted and aggregated, but never filtered out. Here, we introduce a new approach for hard attention and find it achieves very competitive performance on a recently-released visual question answering datasets, equalling and in some cases surpassing similar soft attention architectures while entirely ignoring some features. Even though the hard attention mechanism is thought to be non-differentiable, we found that the feature magnitudes correlate with semantic relevance, and provide a useful signal for our mechanism’s attentional selection criterion. Because hard attention selects important features of the input information, it can also be more efficient than analogous soft attention mechanisms. This is especially important for recent approaches that use non-local pairwise operations, whereby computational and memory costs are quadratic in the size of the set of features.
Tasks	Question Answering, Visual Question Answering
Published	2018-08-01
URL	http://arxiv.org/abs/1808.00300v1
PDF	http://arxiv.org/pdf/1808.00300v1.pdf
PWC	https://paperswithcode.com/paper/learning-visual-question-answering-by
Repo
Framework

Domain Alignment with Triplets


Title	Domain Alignment with Triplets
Authors	Weijian Deng, Liang Zheng, Jianbin Jiao
Abstract	Deep domain adaptation methods can reduce the distribution discrepancy by learning domain-invariant embedddings. However, these methods only focus on aligning the whole data distributions, without considering the class-level relations among source and target images. Thus, a target embeddings of a bird might be aligned to source embeddings of an airplane. This semantic misalignment can directly degrade the classifier performance on the target dataset. To alleviate this problem, we present a similarity constrained alignment (SCA) method for unsupervised domain adaptation. When aligning the distributions in the embedding space, SCA enforces a similarity-preserving constraint to maintain class-level relations among the source and target images, i.e., if a source image and a target image are of the same class label, their corresponding embeddings are supposed to be aligned nearby, and vise versa. In the absence of target labels, we assign pseudo labels for target images. Given labeled source images and pseudo-labeled target images, the similarity-preserving constraint can be implemented by minimizing the triplet loss. With the joint supervision of domain alignment loss and similarity-preserving constraint, we train a network to obtain domain-invariant embeddings with two critical characteristics, intra-class compactness and inter-class separability. Extensive experiments conducted on the two datasets well demonstrate the effectiveness of SCA.
Tasks	Domain Adaptation, Unsupervised Domain Adaptation
Published	2018-12-03
URL	http://arxiv.org/abs/1812.00893v2
PDF	http://arxiv.org/pdf/1812.00893v2.pdf
PWC	https://paperswithcode.com/paper/domain-alignment-with-triplets
Repo
Framework

A Neurobiological Evaluation Metric for Neural Network Model Search


Title	A Neurobiological Evaluation Metric for Neural Network Model Search
Authors	Nathaniel Blanchard, Jeffery Kinnison, Brandon RichardWebster, Pouya Bashivan, Walter J. Scheirer
Abstract	Neuroscience theory posits that the brain’s visual system coarsely identifies broad object categories via neural activation patterns, with similar objects producing similar neural responses. Artificial neural networks also have internal activation behavior in response to stimuli. We hypothesize that networks exhibiting brain-like activation behavior will demonstrate brain-like characteristics, e.g., stronger generalization capabilities. In this paper we introduce a human-model similarity (HMS) metric, which quantifies the similarity of human fMRI and network activation behavior. To calculate HMS, representational dissimilarity matrices (RDMs) are created as abstractions of activation behavior, measured by the correlations of activations to stimulus pairs. HMS is then the correlation between the fMRI RDM and the neural network RDM across all stimulus pairs. We test the metric on unsupervised predictive coding networks, which specifically model visual perception, and assess the metric for statistical significance over a large range of hyperparameters. Our experiments show that networks with increased human-model similarity are correlated with better performance on two computer vision tasks: next frame prediction and object matching accuracy. Further, HMS identifies networks with high performance on both tasks. An unexpected secondary finding is that the metric can be employed during training as an early-stopping mechanism.
Tasks
Published	2018-05-28
URL	http://arxiv.org/abs/1805.10726v4
PDF	http://arxiv.org/pdf/1805.10726v4.pdf
PWC	https://paperswithcode.com/paper/a-neurobiological-evaluation-metric-for
Repo
Framework

Importance Weighted Transfer of Samples in Reinforcement Learning


Title	Importance Weighted Transfer of Samples in Reinforcement Learning
Authors	Andrea Tirinzoni, Andrea Sessa, Matteo Pirotta, Marcello Restelli
Abstract	We consider the transfer of experience samples (i.e., tuples < s, a, s’, r >) in reinforcement learning (RL), collected from a set of source tasks to improve the learning process in a given target task. Most of the related approaches focus on selecting the most relevant source samples for solving the target task, but then all the transferred samples are used without considering anymore the discrepancies between the task models. In this paper, we propose a model-based technique that automatically estimates the relevance (importance weight) of each source sample for solving the target task. In the proposed approach, all the samples are transferred and used by a batch RL algorithm to solve the target task, but their contribution to the learning process is proportional to their importance weight. By extending the results for importance weighting provided in supervised learning literature, we develop a finite-sample analysis of the proposed batch RL algorithm. Furthermore, we empirically compare the proposed algorithm to state-of-the-art approaches, showing that it achieves better learning performance and is very robust to negative transfer, even when some source tasks are significantly different from the target task.
Tasks
Published	2018-05-28
URL	http://arxiv.org/abs/1805.10886v1
PDF	http://arxiv.org/pdf/1805.10886v1.pdf
PWC	https://paperswithcode.com/paper/importance-weighted-transfer-of-samples-in
Repo
Framework

Recurrent Deep Divergence-based Clustering for simultaneous feature learning and clustering of variable length time series


Title	Recurrent Deep Divergence-based Clustering for simultaneous feature learning and clustering of variable length time series
Authors	Daniel J. Trosten, Andreas S. Strauman, Michael Kampffmeyer, Robert Jenssen
Abstract	The task of clustering unlabeled time series and sequences entails a particular set of challenges, namely to adequately model temporal relations and variable sequence lengths. If these challenges are not properly handled, the resulting clusters might be of suboptimal quality. As a key solution, we present a joint clustering and feature learning framework for time series based on deep learning. For a given set of time series, we train a recurrent network to represent, or embed, each time series in a vector space such that a divergence-based clustering loss function can discover the underlying cluster structure in an end-to-end manner. Unlike previous approaches, our model inherently handles multivariate time series of variable lengths and does not require specification of a distance-measure in the input space. On a diverse set of benchmark datasets we illustrate that our proposed Recurrent Deep Divergence-based Clustering approach outperforms, or performs comparable to, previous approaches.
Tasks	Time Series
Published	2018-11-29
URL	http://arxiv.org/abs/1811.12050v2
PDF	http://arxiv.org/pdf/1811.12050v2.pdf
PWC	https://paperswithcode.com/paper/recurrent-deep-divergence-based-clustering
Repo
Framework

Sketch-a-Classifier: Sketch-based Photo Classifier Generation


Title	Sketch-a-Classifier: Sketch-based Photo Classifier Generation
Authors	Conghui Hu, Da Li, Yi-Zhe Song, Tao Xiang, Timothy M. Hospedales
Abstract	Contemporary deep learning techniques have made image recognition a reasonably reliable technology. However training effective photo classifiers typically takes numerous examples which limits image recognition’s scalability and applicability to scenarios where images may not be available. This has motivated investigation into zero-shot learning, which addresses the issue via knowledge transfer from other modalities such as text. In this paper we investigate an alternative approach of synthesizing image classifiers: almost directly from a user’s imagination, via free-hand sketch. This approach doesn’t require the category to be nameable or describable via attributes as per zero-shot learning. We achieve this via training a {model regression} network to map from {free-hand sketch} space to the space of photo classifiers. It turns out that this mapping can be learned in a category-agnostic way, allowing photo classifiers for new categories to be synthesized by user with no need for annotated training photos. {We also demonstrate that this modality of classifier generation can also be used to enhance the granularity of an existing photo classifier, or as a complement to name-based zero-shot learning.
Tasks	Transfer Learning, Zero-Shot Learning
Published	2018-04-30
URL	http://arxiv.org/abs/1804.11182v1
PDF	http://arxiv.org/pdf/1804.11182v1.pdf
PWC	https://paperswithcode.com/paper/sketch-a-classifier-sketch-based-photo
Repo
Framework

GONet: A Semi-Supervised Deep Learning Approach For Traversability Estimation


Title	GONet: A Semi-Supervised Deep Learning Approach For Traversability Estimation
Authors	Noriaki Hirose, Amir Sadeghian, Marynel Vázquez, Patrick Goebel, Silvio Savarese
Abstract	We present semi-supervised deep learning approaches for traversability estimation from fisheye images. Our method, GONet, and the proposed extensions leverage Generative Adversarial Networks (GANs) to effectively predict whether the area seen in the input image(s) is safe for a robot to traverse. These methods are trained with many positive images of traversable places, but just a small set of negative images depicting blocked and unsafe areas. This makes the proposed methods practical. Positive examples can be collected easily by simply operating a robot through traversable spaces, while obtaining negative examples is time consuming, costly, and potentially dangerous. Through extensive experiments and several demonstrations, we show that the proposed traversability estimation approaches are robust and can generalize to unseen scenarios. Further, we demonstrate that our methods are memory efficient and fast, allowing for real-time operation on a mobile robot with single or stereo fisheye cameras. As part of our contributions, we open-source two new datasets for traversability estimation. These datasets are composed of approximately 24h of videos from more than 25 indoor environments. Our methods outperform baseline approaches for traversability estimation on these new datasets.
Tasks
Published	2018-03-08
URL	http://arxiv.org/abs/1803.03254v1
PDF	http://arxiv.org/pdf/1803.03254v1.pdf
PWC	https://paperswithcode.com/paper/gonet-a-semi-supervised-deep-learning
Repo
Framework

Building a Unified Code-Switching ASR System for South African Languages


Title	Building a Unified Code-Switching ASR System for South African Languages
Authors	Emre Yılmaz, Astik Biswas, Ewald van der Westhuizen, Febe de Wet, Thomas Niesler
Abstract	We present our first efforts towards building a single multilingual automatic speech recognition (ASR) system that can process code-switching (CS) speech in five languages spoken within the same population. This contrasts with related prior work which focuses on the recognition of CS speech in bilingual scenarios. Recently, we have compiled a small five-language corpus of South African soap opera speech which contains examples of CS between 5 languages occurring in various contexts such as using English as the matrix language and switching to other indigenous languages. The ASR system presented in this work is trained on 4 corpora containing English-isiZulu, English-isiXhosa, English-Setswana and English-Sesotho CS speech. The interpolation of multiple language models trained on these language pairs enables the ASR system to hypothesize mixed word sequences from these 5 languages. We evaluate various state-of-the-art acoustic models trained on this 5-lingual training data and report ASR accuracy and language recognition performance on the development and test sets of the South African multilingual soap opera corpus.
Tasks	Speech Recognition
Published	2018-07-28
URL	http://arxiv.org/abs/1807.10949v1
PDF	http://arxiv.org/pdf/1807.10949v1.pdf
PWC	https://paperswithcode.com/paper/building-a-unified-code-switching-asr-system
Repo
Framework

Robustness against the channel effect in pathological voice detection


Title	Robustness against the channel effect in pathological voice detection
Authors	Yi-Te Hsu, Zining Zhu, Chi-Te Wang, Shih-Hau Fang, Frank Rudzicz, Yu Tsao
Abstract	Many people are suffering from voice disorders, which can adversely affect the quality of their lives. In response, some researchers have proposed algorithms for automatic assessment of these disorders, based on voice signals. However, these signals can be sensitive to the recording devices. Indeed, the channel effect is a pervasive problem in machine learning for healthcare. In this study, we propose a detection system for pathological voice, which is robust against the channel effect. This system is based on a bidirectional LSTM network. To increase the performance robustness against channel mismatch, we integrate domain adversarial training (DAT) to eliminate the differences between the devices. When we train on data recorded on a high-quality microphone and evaluate on smartphone data without labels, our robust detection system increases the PR-AUC from 0.8448 to 0.9455 (and 0.9522 with target sample labels). To the best of our knowledge, this is the first study applying unsupervised domain adaptation to pathological voice detection. Notably, our system does not need target device sample labels, which allows for generalization to many new devices.
Tasks	Domain Adaptation, Unsupervised Domain Adaptation
Published	2018-11-26
URL	http://arxiv.org/abs/1811.10376v2
PDF	http://arxiv.org/pdf/1811.10376v2.pdf
PWC	https://paperswithcode.com/paper/robustness-against-the-channel-effect-in
Repo
Framework

Bernoulli Embeddings for Graphs


Title	Bernoulli Embeddings for Graphs
Authors	Vinith Misra, Sumit Bhatia
Abstract	Just as semantic hashing can accelerate information retrieval, binary valued embeddings can significantly reduce latency in the retrieval of graphical data. We introduce a simple but effective model for learning such binary vectors for nodes in a graph. By imagining the embeddings as independent coin flips of varying bias, continuous optimization techniques can be applied to the approximate expected loss. Embeddings optimized in this fashion consistently outperform the quantization of both spectral graph embeddings and various learned real-valued embeddings, on both ranking and pre-ranking tasks for a variety of datasets.
Tasks	Information Retrieval, Quantization
Published	2018-03-25
URL	http://arxiv.org/abs/1803.09211v1
PDF	http://arxiv.org/pdf/1803.09211v1.pdf
PWC	https://paperswithcode.com/paper/bernoulli-embeddings-for-graphs
Repo
Framework

German Dialect Identification Using Classifier Ensembles


Title	German Dialect Identification Using Classifier Ensembles
Authors	Alina Maria Ciobanu, Shervin Malmasi, Liviu P. Dinu
Abstract	In this paper we present the GDI_classification entry to the second German Dialect Identification (GDI) shared task organized within the scope of the VarDial Evaluation Campaign 2018. We present a system based on SVM classifier ensembles trained on characters and words. The system was trained on a collection of speech transcripts of five Swiss-German dialects provided by the organizers. The transcripts included in the dataset contained speakers from Basel, Bern, Lucerne, and Zurich. Our entry in the challenge reached 62.03% F1-score and was ranked third out of eight teams.
Tasks
Published	2018-07-22
URL	http://arxiv.org/abs/1807.08230v1
PDF	http://arxiv.org/pdf/1807.08230v1.pdf
PWC	https://paperswithcode.com/paper/german-dialect-identification-using
Repo
Framework

Entropy-based closure for probabilistic learning on manifolds


Title	Entropy-based closure for probabilistic learning on manifolds
Authors	C. Soizea, R. Ghanem, C. Safta, X. Huan, Z. P. Vane, J. Oefelein, G. Lacaz, H. N. Najm, Q. Tang, X. Chen
Abstract	In a recent paper, the authors proposed a general methodology for probabilistic learning on manifolds. The method was used to generate numerical samples that are statistically consistent with an existing dataset construed as a realization from a non-Gaussian random vector. The manifold structure is learned using diffusion manifolds and the statistical sample generation is accomplished using a projected Ito stochastic differential equation. This probabilistic learning approach has been extended to polynomial chaos representation of databases on manifolds and to probabilistic nonconvex constrained optimization with a fixed budget of function evaluations. The methodology introduces an isotropic-diffusion kernel with hyperparameter {\epsilon}. Currently, {\epsilon} is more or less arbitrarily chosen. In this paper, we propose a selection criterion for identifying an optimal value of {\epsilon}, based on a maximum entropy argument. The result is a comprehensive, closed, probabilistic model for characterizing data sets with hidden constraints. This entropy argument ensures that out of all possible models, this is the one that is the most uncertain beyond any specified constraints, which is selected. Applications are presented for several databases.
Tasks
Published	2018-03-21
URL	http://arxiv.org/abs/1803.08161v2
PDF	http://arxiv.org/pdf/1803.08161v2.pdf
PWC	https://paperswithcode.com/paper/entropy-based-closure-for-probabilistic
Repo
Framework