February 1, 2020

2697 words 13 mins read

Paper Group AWR 365

A Tsetlin Machine with Multigranular Clauses. In Conclusion Not Repetition: Comprehensive Abstractive Summarization With Diversified Attention Based On Determinantal Point Processes. Tha3aroon at NSURL-2019 Task 8: Semantic Question Similarity in Arabic. Adversarial Augmentation for Enhancing Classification of Mammography Images. Persistent Homolog …

A Tsetlin Machine with Multigranular Clauses


Title	A Tsetlin Machine with Multigranular Clauses
Authors	Saeed Rahimi Gorji, Ole-Christoffer Granmo, Adrian Phoulady, Morten Goodwin
Abstract	The recently introduced Tsetlin Machine (TM) has provided competitive pattern recognition accuracy in several benchmarks, however, requires a 3-dimensional hyperparameter search. In this paper, we introduce the Multigranular Tsetlin Machine (MTM). The MTM eliminates the specificity hyperparameter, used by the TM to control the granularity of the conjunctive clauses that it produces for recognizing patterns. Instead of using a fixed global specificity, we encode varying specificity as part of the clauses, rendering the clauses multigranular. This makes it easier to configure the TM because the dimensionality of the hyperparameter search space is reduced to only two dimensions. Indeed, it turns out that there is significantly less hyperparameter tuning involved in applying the MTM to new problems. Further, we demonstrate empirically that the MTM provides similar performance to what is achieved with a finely specificity-optimized TM, by comparing their performance on both synthetic and real-world datasets.
Tasks
Published	2019-09-16
URL	https://arxiv.org/abs/1909.07310v1
PDF	https://arxiv.org/pdf/1909.07310v1.pdf
PWC	https://paperswithcode.com/paper/a-tsetlin-machine-with-multigranular-clauses
Repo	https://github.com/zdx3578/pyTsetlinMachine
Framework	none

In Conclusion Not Repetition: Comprehensive Abstractive Summarization With Diversified Attention Based On Determinantal Point Processes


Title	In Conclusion Not Repetition: Comprehensive Abstractive Summarization With Diversified Attention Based On Determinantal Point Processes
Authors	Lei Li, Wei Liu, Marina Litvak, Natalia Vanetik, Zuying Huang
Abstract	Various Seq2Seq learning models designed for machine translation were applied for abstractive summarization task recently. Despite these models provide high ROUGE scores, they are limited to generate comprehensive summaries with a high level of abstraction due to its degenerated attention distribution. We introduce Diverse Convolutional Seq2Seq Model(DivCNN Seq2Seq) using Determinantal Point Processes methods(Micro DPPs and Macro DPPs) to produce attention distribution considering both quality and diversity. Without breaking the end to end architecture, DivCNN Seq2Seq achieves a higher level of comprehensiveness compared to vanilla models and strong baselines. All the reproducible codes and datasets are available online.
Tasks	Abstractive Text Summarization, Machine Translation, Point Processes
Published	2019-09-24
URL	https://arxiv.org/abs/1909.10852v2
PDF	https://arxiv.org/pdf/1909.10852v2.pdf
PWC	https://paperswithcode.com/paper/in-conclusion-not-repetition-comprehensive
Repo	https://github.com/thinkwee/DPP_CNN_Summarization
Framework	pytorch

Tha3aroon at NSURL-2019 Task 8: Semantic Question Similarity in Arabic


Title	Tha3aroon at NSURL-2019 Task 8: Semantic Question Similarity in Arabic
Authors	Ali Fadel, Ibraheem Tuffaha, Mahmoud Al-Ayyoub
Abstract	In this paper, we describe our team’s effort on the semantic text question similarity task of NSURL 2019. Our top performing system utilizes several innovative data augmentation techniques to enlarge the training data. Then, it takes ELMo pre-trained contextual embeddings of the data and feeds them into an ON-LSTM network with self-attention. This results in sequence representation vectors that are used to predict the relation between the question pairs. The model is ranked in the 1st place with 96.499 F1-score (same as the second place F1-score) and the 2nd place with 94.848 F1-score (differs by 1.076 F1-score from the first place) on the public and private leaderboards, respectively.
Tasks	Data Augmentation, Question Similarity
Published	2019-12-28
URL	https://arxiv.org/abs/1912.12514v1
PDF	https://arxiv.org/pdf/1912.12514v1.pdf
PWC	https://paperswithcode.com/paper/tha3aroon-at-nsurl-2019-task-8-semantic
Repo	https://github.com/AliOsm/semantic-question-similarity
Framework	none

Adversarial Augmentation for Enhancing Classification of Mammography Images


Title	Adversarial Augmentation for Enhancing Classification of Mammography Images
Authors	Lukas Jendele, Ondrej Skopek, Anton S. Becker, Ender Konukoglu
Abstract	Supervised deep learning relies on the assumption that enough training data is available, which presents a problem for its application to several fields, like medical imaging. On the example of a binary image classification task (breast cancer recognition), we show that pretraining a generative model for meaningful image augmentation helps enhance the performance of the resulting classifier. By augmenting the data, performance on downstream classification tasks could be improved even with a relatively small training set. We show that this “adversarial augmentation” yields promising results compared to classical image augmentation on the example of breast cancer classification.
Tasks	Image Augmentation, Image Classification
Published	2019-02-20
URL	http://arxiv.org/abs/1902.07762v1
PDF	http://arxiv.org/pdf/1902.07762v1.pdf
PWC	https://paperswithcode.com/paper/adversarial-augmentation-for-enhancing
Repo	https://github.com/BreastGAN/augmentation
Framework	tf

Persistent Homology as Stopping-Criterion for Voronoi Interpolation


Title	Persistent Homology as Stopping-Criterion for Voronoi Interpolation
Authors	Luciano Melodia, Richard Lenz
Abstract	In this study the Voronoi interpolation is used to interpolate a set of points drawn from a topological space with higher homology groups on its filtration. The technique is based on Voronoi tesselation, which induces a natural dual map to the Delaunay triangulation. Advantage is taken from this fact calculating the persistent homology on it after each iteration to capture the changing topology of the data. The boundary points are identified as critical. The Bottleneck and Wasserstein distance serve as a measure of quality between the original point set and the interpolation. If the norm of two distances exceeds a heuristically determined threshold, the algorithm terminates. We give the theoretical basis for this approach and justify its validity with numerical experiments.
Tasks
Published	2019-11-08
URL	https://arxiv.org/abs/1911.02922v10
PDF	https://arxiv.org/pdf/1911.02922v10.pdf
PWC	https://paperswithcode.com/paper/persistent-homology-as-stopping-criterion-for
Repo	https://github.com/karhunenloeve/karhunenloeve.github.io
Framework	none

Efficient Computation of Hessian Matrices in TensorFlow


Title	Efficient Computation of Hessian Matrices in TensorFlow
Authors	Geir K. Nilsen, Antonella Z. Munthe-Kaas, Hans J. Skaug, Morten Brun
Abstract	The Hessian matrix has a number of important applications in a variety of different fields, such as optimzation, image processing and statistics. In this paper we focus on the practical aspects of efficiently computing Hessian matrices in the context of deep learning using the Python scripting language and the TensorFlow library.
Tasks
Published	2019-05-14
URL	https://arxiv.org/abs/1905.05559v1
PDF	https://arxiv.org/pdf/1905.05559v1.pdf
PWC	https://paperswithcode.com/paper/efficient-computation-of-hessian-matrices-in
Repo	https://github.com/gknilsen/pyhessian
Framework	tf

Category-Level Articulated Object Pose Estimation


Title	Category-Level Articulated Object Pose Estimation
Authors	Xiaolong Li, He Wang, Li Yi, Leonidas Guibas, A. Lynn Abbott, Shuran Song
Abstract	This paper addresses the task of category-level pose estimation for articulated objects from a single depth image. We present a novel category-level approach that correctly accommodates object instances not previously seen during training. A key aspect of the work is the new Articulation-Aware Normalized Coordinate Space Hierarchy (A-NCSH), which represents the different articulated objects for a given object category. This approach not only provides the canonical representation of each rigid part, but also normalizes the joint parameters and joint states. We developed a deep network based on PointNet++ that is capable of predicting an A-NCSH representation for unseen object instances from single depth input. The predicted A-NCSH representation is then used for global pose optimization using kinematic constraints. We demonstrate that constraints associated with joints in the kinematic chain lead to improved performance in estimating pose and relative scale for each part of the object. We also demonstrate that the approach can tolerate cases of severe occlusion in the observed data. Project webpage https://articulated-pose.github.io/
Tasks	Pose Estimation
Published	2019-12-26
URL	https://arxiv.org/abs/1912.11913v1
PDF	https://arxiv.org/pdf/1912.11913v1.pdf
PWC	https://paperswithcode.com/paper/category-level-articulated-object-pose
Repo	https://github.com/dragonlong/articulated-pose
Framework	tf

BayesOD: A Bayesian Approach for Uncertainty Estimation in Deep Object Detectors


Title	BayesOD: A Bayesian Approach for Uncertainty Estimation in Deep Object Detectors
Authors	Ali Harakeh, Michael Smart, Steven L. Waslander
Abstract	When incorporating deep neural networks into robotic systems, a major challenge is the lack of uncertainty measures associated with their output predictions. Methods for uncertainty estimation in the output of deep object detectors (DNNs) have been proposed in recent works, but have had limited success due to 1) information loss at the detectors non-maximum suppression (NMS) stage, and 2) failure to take into account the multitask, many-to-one nature of anchor-based object detection. To that end, we introduce BayesOD, an uncertainty estimation approach that reformulates the standard object detector inference and Non-Maximum suppression components from a Bayesian perspective. Experiments performed on four common object detection datasets show that BayesOD provides uncertainty estimates that are better correlated with the accuracy of detections, manifesting as a significant reduction of 9.77%-13.13% on the minimum Gaussian uncertainty error metric and a reduction of 1.63%-5.23% on the minimum Categorical uncertainty error metric. Code will be released at {\url{https://github.com/asharakeh/bayes-od-rc}}.
Tasks	Object Detection
Published	2019-03-09
URL	https://arxiv.org/abs/1903.03838v2
PDF	https://arxiv.org/pdf/1903.03838v2.pdf
PWC	https://paperswithcode.com/paper/bayesod-a-bayesian-approach-for-uncertainty
Repo	https://github.com/asharakeh/bayes-od-rc
Framework	tf

Learning Sound Event Classifiers from Web Audio with Noisy Labels


Title	Learning Sound Event Classifiers from Web Audio with Noisy Labels
Authors	Eduardo Fonseca, Manoj Plakal, Daniel P. W. Ellis, Frederic Font, Xavier Favory, Xavier Serra
Abstract	As sound event classification moves towards larger datasets, issues of label noise become inevitable. Web sites can supply large volumes of user-contributed audio and metadata, but inferring labels from this metadata introduces errors due to unreliable inputs, and limitations in the mapping. There is, however, little research into the impact of these errors. To foster the investigation of label noise in sound event classification we present FSDnoisy18k, a dataset containing 42.5 hours of audio across 20 sound classes, including a small amount of manually-labeled data and a larger quantity of real-world noisy data. We characterize the label noise empirically, and provide a CNN baseline system. Experiments suggest that training with large amounts of noisy data can outperform training with smaller amounts of carefully-labeled data. We also show that noise-robust loss functions can be effective in improving performance in presence of corrupted labels.
Tasks	Sound Event Detection
Published	2019-01-04
URL	http://arxiv.org/abs/1901.01189v2
PDF	http://arxiv.org/pdf/1901.01189v2.pdf
PWC	https://paperswithcode.com/paper/learning-sound-event-classifiers-from-web
Repo	https://github.com/edufonseca/icassp19
Framework	tf

Continual Multi-task Gaussian Processes


Title	Continual Multi-task Gaussian Processes
Authors	Pablo Moreno-Muñoz, Antonio Artés-Rodríguez, Mauricio A. Álvarez
Abstract	We address the problem of continual learning in multi-task Gaussian process (GP) models for handling sequential input-output observations. Our approach extends the existing prior-posterior recursion of online Bayesian inference, i.e.\ past posterior discoveries become future prior beliefs, to the infinite functional space setting of GP. For a reason of scalability, we introduce variational inference together with an sparse approximation based on inducing inputs. As a consequence, we obtain tractable continual lower-bounds where two novel Kullback-Leibler (KL) divergences intervene in a natural way. The key technical property of our method is the recursive reconstruction of conditional GP priors conditioned on the variational parameters learned so far. To achieve this goal, we introduce a novel factorization of past variational distributions, where the predictive GP equation propagates the posterior uncertainty forward. We then demonstrate that it is possible to derive GP models over many types of sequential observations, either discrete or continuous and amenable to stochastic optimization. The continual inference approach is also applicable to scenarios where potential multi-channel or heterogeneous observations might appear. Extensive experiments demonstrate that the method is fully scalable, shows a reliable performance and is robust to uncertainty error propagation over a plenty of synthetic and real-world datasets.
Tasks	Bayesian Inference, Continual Learning, Gaussian Processes, Stochastic Optimization
Published	2019-10-31
URL	https://arxiv.org/abs/1911.00002v1
PDF	https://arxiv.org/pdf/1911.00002v1.pdf
PWC	https://paperswithcode.com/paper/continual-multi-task-gaussian-processes
Repo	https://github.com/pmorenoz/ContinualGP
Framework	none

Function-Space Distributions over Kernels


Title	Function-Space Distributions over Kernels
Authors	Gregory W. Benton, Wesley J. Maddox, Jayson P. Salkey, Julio Albinati, Andrew Gordon Wilson
Abstract	Gaussian processes are flexible function approximators, with inductive biases controlled by a covariance kernel. Learning the kernel is the key to representation learning and strong predictive performance. In this paper, we develop functional kernel learning (FKL) to directly infer functional posteriors over kernels. In particular, we place a transformed Gaussian process over a spectral density, to induce a non-parametric distribution over kernel functions. The resulting approach enables learning of rich representations, with support for any stationary kernel, uncertainty over the values of the kernel, and an interpretable specification of a prior directly over kernels, without requiring sophisticated initialization or manual intervention. We perform inference through elliptical slice sampling, which is especially well suited to marginalizing posteriors with the strongly correlated priors typical to function space modelling. We develop our approach for non-uniform, large-scale, multi-task, and multidimensional data, and show promising performance in a wide range of settings, including interpolation, extrapolation, and kernel recovery experiments.
Tasks	Gaussian Processes, Representation Learning
Published	2019-10-29
URL	https://arxiv.org/abs/1910.13565v1
PDF	https://arxiv.org/pdf/1910.13565v1.pdf
PWC	https://paperswithcode.com/paper/function-space-distributions-over-kernels
Repo	https://github.com/wjmaddox/spectralgp
Framework	pytorch

Implicit Posterior Variational Inference for Deep Gaussian Processes


Title	Implicit Posterior Variational Inference for Deep Gaussian Processes
Authors	Haibin Yu, Yizhou Chen, Zhongxiang Dai, Kian Hsiang Low, Patrick Jaillet
Abstract	A multi-layer deep Gaussian process (DGP) model is a hierarchical composition of GP models with a greater expressive power. Exact DGP inference is intractable, which has motivated the recent development of deterministic and stochastic approximation methods. Unfortunately, the deterministic approximation methods yield a biased posterior belief while the stochastic one is computationally costly. This paper presents an implicit posterior variational inference (IPVI) framework for DGPs that can ideally recover an unbiased posterior belief and still preserve time efficiency. Inspired by generative adversarial networks, our IPVI framework achieves this by casting the DGP inference problem as a two-player game in which a Nash equilibrium, interestingly, coincides with an unbiased posterior belief. This consequently inspires us to devise a best-response dynamics algorithm to search for a Nash equilibrium (i.e., an unbiased posterior belief). Empirical evaluation shows that IPVI outperforms the state-of-the-art approximation methods for DGPs.
Tasks	Gaussian Processes
Published	2019-10-26
URL	https://arxiv.org/abs/1910.11998v1
PDF	https://arxiv.org/pdf/1910.11998v1.pdf
PWC	https://paperswithcode.com/paper/implicit-posterior-variational-inference-for
Repo	https://github.com/HeroKillerEver/ipvi-dgp
Framework	tf

Contextual Attention for Hand Detection in the Wild


Title	Contextual Attention for Hand Detection in the Wild
Authors	Supreeth Narasimhaswamy, Zhengwei Wei, Yang Wang, Justin Zhang, Minh Hoai
Abstract	We present Hand-CNN, a novel convolutional network architecture for detecting hand masks and predicting hand orientations in unconstrained images. Hand-CNN extends MaskRCNN with a novel attention mechanism to incorporate contextual cues in the detection process. This attention mechanism can be implemented as an efficient network module that captures non-local dependencies between features. This network module can be inserted at different stages of an object detection network, and the entire detector can be trained end-to-end. We also introduce a large-scale annotated hand dataset containing hands in unconstrained images for training and evaluation. We show that Hand-CNN outperforms existing methods on several datasets, including our hand detection benchmark and the publicly available PASCAL VOC human layout challenge. We also conduct ablation studies on hand detection to show the effectiveness of the proposed contextual attention module.
Tasks	Object Detection
Published	2019-04-09
URL	http://arxiv.org/abs/1904.04882v1
PDF	http://arxiv.org/pdf/1904.04882v1.pdf
PWC	https://paperswithcode.com/paper/contextual-attention-for-hand-detection-in
Repo	https://github.com/SupreethN/Hand-CNN
Framework	tf

Sparse Orthogonal Variational Inference for Gaussian Processes


Title	Sparse Orthogonal Variational Inference for Gaussian Processes
Authors	Jiaxin Shi, Michalis K. Titsias, Andriy Mnih
Abstract	We introduce a new interpretation of sparse variational approximations for Gaussian processes using inducing points, which can lead to more scalable algorithms than previous methods. It is based on decomposing a Gaussian process as a sum of two independent processes: one spanned by a finite basis of inducing points and the other capturing the remaining variation. We show that this formulation recovers existing approximations and at the same time allows to obtain tighter lower bounds on the marginal likelihood and new stochastic variational inference algorithms. We demonstrate the efficiency of these algorithms in several Gaussian process models ranging from standard regression to multi-class classification using (deep) convolutional Gaussian processes and report state-of-the-art results on CIFAR-10 among purely GP-based models.
Tasks	Gaussian Processes
Published	2019-10-23
URL	https://arxiv.org/abs/1910.10596v3
PDF	https://arxiv.org/pdf/1910.10596v3.pdf
PWC	https://paperswithcode.com/paper/sparse-orthogonal-variational-inference-for
Repo	https://github.com/thjashin/solvegp
Framework	tf

Bayesian Learning-Based Adaptive Control for Safety Critical Systems


Title	Bayesian Learning-Based Adaptive Control for Safety Critical Systems
Authors	David D. Fan, Jennifer Nguyen, Rohan Thakker, Nikhilesh Alatur, Ali-akbar Agha-mohammadi, Evangelos A. Theodorou
Abstract	Deep learning has enjoyed much recent success, and applying state-of-the-art model learning methods to controls is an exciting prospect. However, there is a strong reluctance to use these methods on safety-critical systems, which have constraints on safety, stability, and real-time performance. We propose a framework which satisfies these constraints while allowing the use of deep neural networks for learning model uncertainties. Central to our method is the use of Bayesian model learning, which provides an avenue for maintaining appropriate degrees of caution in the face of the unknown. In the proposed approach, we develop an adaptive control framework leveraging the theory of stochastic CLFs (Control Lyapunov Functions) and stochastic CBFs (Control Barrier Functions) along with tractable Bayesian model learning via Gaussian Processes or Bayesian neural networks. Under reasonable assumptions, we guarantee stability and safety while adapting to unknown dynamics with probability 1. We demonstrate this architecture for high-speed terrestrial mobility targeting potential applications in safety-critical high-speed Mars rover missions.
Tasks	Autonomous Vehicles, Bayesian Inference, Gaussian Processes, Interpretable Machine Learning
Published	2019-10-05
URL	https://arxiv.org/abs/1910.02325v2
PDF	https://arxiv.org/pdf/1910.02325v2.pdf
PWC	https://paperswithcode.com/paper/bayesian-learning-based-adaptive-control-for
Repo	https://github.com/ddfan/balsa
Framework	tf