January 27, 2020

3416 words 17 mins read

Paper Group ANR 1106

Look Further to Recognize Better: Learning Shared Topics and Category-Specific Dictionaries for Open-Ended 3D Object Recognition. Efficient data augmentation using graph imputation neural networks. Multiple Futures Prediction. Best-First Width Search for Multi Agent Privacy-preserving Planning. Deterministic and Bayesian Neural Networks for Low-lat …

Look Further to Recognize Better: Learning Shared Topics and Category-Specific Dictionaries for Open-Ended 3D Object Recognition


Title	Look Further to Recognize Better: Learning Shared Topics and Category-Specific Dictionaries for Open-Ended 3D Object Recognition
Authors	S. Hamidreza Kasaei
Abstract	Service robots are expected to operate effectively in human-centric environments for long periods of time. In such realistic scenarios, fine-grained object categorization is as important as basic-level object categorization. We tackle this problem by proposing an open-ended object recognition approach which concurrently learns both the object categories and the local features for encoding objects. In this work, each object is represented using a set of general latent visual topics and category-specific dictionaries. The general topics encode the common patterns of all categories, while the category-specific dictionary describes the content of each category in details. The proposed approach discovers both sets of general and specific representations in an unsupervised fashion and updates them incrementally using new object views. Experimental results show that our approach yields significant improvements over the previous state-of-the-art approaches concerning scalability and object classification performance. Moreover, our approach demonstrates the capability of learning from very few training examples in a real-world setting. Regarding computation time, the best result was obtained with a Bag-of-Words method followed by a variant of the Latent Dirichlet Allocation approach.
Tasks	3D Object Recognition, Object Classification, Object Recognition
Published	2019-07-26
URL	https://arxiv.org/abs/1907.12924v1
PDF	https://arxiv.org/pdf/1907.12924v1.pdf
PWC	https://paperswithcode.com/paper/look-further-to-recognize-better-learning
Repo
Framework

Efficient data augmentation using graph imputation neural networks


Title	Efficient data augmentation using graph imputation neural networks
Authors	Indro Spinelli, Simone Scardapane, Michele Scarpiniti, Aurelio Uncini
Abstract	Recently, data augmentation in the semi-supervised regime, where unlabeled data vastly outnumbers labeled data, has received a considerable attention. In this paper, we describe an efficient technique for this task, exploiting a recent framework we proposed for missing data imputation called graph imputation neural network (GINN). The key idea is to leverage both supervised and unsupervised data to build a graph of similarities between points in the dataset. Then, we augment the dataset by severely damaging a few of the nodes (up to 80% of their features), and reconstructing them using a variation of GINN. On several benchmark datasets, we show that our method can obtain significant improvements compared to a fully-supervised model, and we are able to augment the datasets up to a factor of 10x. This points to the power of graph-based neural networks to represent structural affinities in the samples for tasks of data reconstruction and augmentation.
Tasks	Data Augmentation, Imputation
Published	2019-06-20
URL	https://arxiv.org/abs/1906.08502v1
PDF	https://arxiv.org/pdf/1906.08502v1.pdf
PWC	https://paperswithcode.com/paper/efficient-data-augmentation-using-graph
Repo
Framework

Multiple Futures Prediction


Title	Multiple Futures Prediction
Authors	Yichuan Charlie Tang, Ruslan Salakhutdinov
Abstract	Temporal prediction is critical for making intelligent and robust decisions in complex dynamic environments. Motion prediction needs to model the inherently uncertain future which often contains multiple potential outcomes, due to multi-agent interactions and the latent goals of others. Towards these goals, we introduce a probabilistic framework that efficiently learns latent variables to jointly model the multi-step future motions of agents in a scene. Our framework is data-driven and learns semantically meaningful latent variables to represent the multimodal future, without requiring explicit labels. Using a dynamic attention-based state encoder, we learn to encode the past as well as the future interactions among agents, efficiently scaling to any number of agents. Finally, our model can be used for planning via computing a conditional probability density over the trajectories of other agents given a hypothetical rollout of the ‘self’ agent. We demonstrate our algorithms by predicting vehicle trajectories of both simulated and real data, demonstrating the state-of-the-art results on several vehicle trajectory datasets.
Tasks	motion prediction
Published	2019-11-04
URL	https://arxiv.org/abs/1911.00997v2
PDF	https://arxiv.org/pdf/1911.00997v2.pdf
PWC	https://paperswithcode.com/paper/multiple-futures-prediction
Repo
Framework

Best-First Width Search for Multi Agent Privacy-preserving Planning


Title	Best-First Width Search for Multi Agent Privacy-preserving Planning
Authors	Alfonso E. Gerevini, Nir Lipovetzky, Francesco Percassi, Alessandro Saetti, Ivan Serina
Abstract	In multi-agent planning, preserving the agents’ privacy has become an increasingly popular research topic. For preserving the agents’ privacy, agents jointly compute a plan that achieves mutual goals by keeping certain information private to the individual agents. Unfortunately, this can severely restrict the accuracy of the heuristic functions used while searching for solutions. It has been recently shown that, for centralized planning, the performance of goal oriented search can be improved by combining goal oriented search and width-based search. The combination of these techniques has been called best-first width search. In this paper, we investigate the usage of best-first width search in the context of (decentralised) multi-agent privacy-preserving planning, addressing the challenges related to the agents’ privacy and performance. In particular, we show that best-first width search is a very effective approach over several benchmark domains, even when the search is driven by heuristics that roughly estimate the distance from goal states, computed without using the private information of other agents. An experimental study analyses the effectiveness of our techniques and compares them with the state-of-the-art.
Tasks
Published	2019-06-10
URL	https://arxiv.org/abs/1906.03955v1
PDF	https://arxiv.org/pdf/1906.03955v1.pdf
PWC	https://paperswithcode.com/paper/best-first-width-search-for-multi-agent
Repo
Framework

Deterministic and Bayesian Neural Networks for Low-latency Gravitational Wave Parameter Estimation of Binary Black Hole Mergers


Title	Deterministic and Bayesian Neural Networks for Low-latency Gravitational Wave Parameter Estimation of Binary Black Hole Mergers
Authors	Hongyu Shen, E. A. Huerta, Zhizhen Zhao, Elise Jennings, Himanshu Sharma
Abstract	We present the first application of deep learning for gravitational wave parameter estimation of binary black hole mergers evolving on quasi-circular orbits with aligned or anti-aligned spins. We use root-leaf structured networks to ensure that common physical features are shared across all parameters. In order to cover a broad range of astrophysically motivated scenarios, we use a training dataset with over $10^7$ modeled waveforms to ensure local time- and scale-invariance. The trained models are applied to estimate the astrophysical parameters of the existing catalog of detected binary black hole mergers, and their corresponding black hole remnants, including the final spin and the gravitational wave quasi-normal frequencies. Using a deterministic neural network model, we are able to efficiently provide point-parameter estimation results, along with statistical errors caused by the noise spectrum uncertainty. We also introduce the first application of Bayesian neural networks for gravitational wave parameter estimation of real astrophysical events. These probabilistic models were trained with over $10^7$ modeled waveforms and using 1024 nodes (65,536 core processors) on the Theta supercomputer at Argonne Leadership Computing Facility to reduce the training stage to just thirty minutes. In inference mode, both the deterministic and Bayesian neural networks estimate the astrophysical parameters of binary black hole mergers within 2 milliseconds using a single Tesla V100 GPU. Both deterministic and Bayesian neural networks produce agreeing parameter estimation results, which are also consistent with Bayesian analyses used to characterize the catalog of binary black hole mergers observed by the advanced LIGO and Virgo detectors.
Tasks
Published	2019-03-05
URL	https://arxiv.org/abs/1903.01998v2
PDF	https://arxiv.org/pdf/1903.01998v2.pdf
PWC	https://paperswithcode.com/paper/deep-learning-at-scale-for-gravitational-wave
Repo
Framework

Clique pooling for graph classification


Title	Clique pooling for graph classification
Authors	Enxhell Luzhnica, Ben Day, Pietro Lio’
Abstract	We propose a novel graph pooling operation using cliques as the unit pool. As this approach is purely topological, rather than featural, it is more readily interpretable, a better analogue to image coarsening than filtering or pruning techniques, and entirely nonparametric. The operation is implemented within graph convolution network (GCN) and GraphSAGE architectures and tested against standard graph classification benchmarks. In addition, we explore the backwards compatibility of the pooling to regular graphs, demonstrating competitive performance when replacing two-by-two pooling in standard convolutional neural networks (CNNs) with our mechanism.
Tasks	Graph Classification
Published	2019-03-31
URL	http://arxiv.org/abs/1904.00374v2
PDF	http://arxiv.org/pdf/1904.00374v2.pdf
PWC	https://paperswithcode.com/paper/clique-pooling-for-graph-classification
Repo
Framework

Protocol for implementing quantum nonparametric learning with trapped ions


Title	Protocol for implementing quantum nonparametric learning with trapped ions
Authors	Dan-Bo Zhang, Shi-Liang Zhu, Z. D. Wang
Abstract	Nonparametric learning is able to make reliable predictions by extracting information from similarities between a new set of input data and all samples. Here we point out a quantum paradigm of nonparametric learning which offers an exponential speedup over the sample size. By encoding data into quantum feature space, similarity between the data is defined as an inner product of quantum states. A quantum training state is introduced to superpose all data of samples, encoding relevant information for learning in its bipartite entanglement spectrum. We demonstrate that a trained state for prediction can be obtained by entanglement spectrum transformation, using quantum matrix toolbox. We further work out a feasible protocol to implement the quantum nonparametric learning with trapped ions, and demonstrate the power of quantum superposition for machine learning.
Tasks
Published	2019-06-08
URL	https://arxiv.org/abs/1906.03388v2
PDF	https://arxiv.org/pdf/1906.03388v2.pdf
PWC	https://paperswithcode.com/paper/physical-implementation-of-quantum
Repo
Framework

A Reparameterization-Invariant Flatness Measure for Deep Neural Networks


Title	A Reparameterization-Invariant Flatness Measure for Deep Neural Networks
Authors	Henning Petzka, Linara Adilova, Michael Kamp, Cristian Sminchisescu
Abstract	The performance of deep neural networks is often attributed to their automated, task-related feature construction. It remains an open question, though, why this leads to solutions with good generalization, even in cases where the number of parameters is larger than the number of samples. Back in the 90s, Hochreiter and Schmidhuber observed that flatness of the loss surface around a local minimum correlates with low generalization error. For several flatness measures, this correlation has been empirically validated. However, it has recently been shown that existing measures of flatness cannot theoretically be related to generalization due to a lack of invariance with respect to reparameterizations. We propose a natural modification of existing flatness measures that results in invariance to reparameterization.
Tasks
Published	2019-11-29
URL	https://arxiv.org/abs/1912.00058v1
PDF	https://arxiv.org/pdf/1912.00058v1.pdf
PWC	https://paperswithcode.com/paper/a-reparameterization-invariant-flatness
Repo
Framework

Why Not to Use Zero Imputation? Correcting Sparsity Bias in Training Neural Networks


Title	Why Not to Use Zero Imputation? Correcting Sparsity Bias in Training Neural Networks
Authors	Joonyoung Yi, Juhyuk Lee, Kwang Joon Kim, Sung Ju Hwang, Eunho Yang
Abstract	Handling missing data is one of the most fundamental problems in machine learning. Among many approaches, the simplest and most intuitive way is zero imputation, which treats the value of a missing entry simply as zero. However, many studies have experimentally confirmed that zero imputation results in suboptimal performances in training neural networks. Yet, none of the existing work has explained what brings such performance degradations. In this paper, we introduce the variable sparsity problem (VSP), which describes a phenomenon where the output of a predictive model largely varies with respect to the rate of missingness in the given input, and show that it adversarially affects the model performance. We first theoretically analyze this phenomenon and propose a simple yet effective technique to handle missingness, which we refer to as Sparsity Normalization (SN), that directly targets and resolves the VSP. We further experimentally validate SN on diverse benchmark datasets, to show that debiasing the effect of input-level sparsity improves the performance and stabilizes the training of neural networks.
Tasks	Imputation
Published	2019-06-01
URL	https://arxiv.org/abs/1906.00150v5
PDF	https://arxiv.org/pdf/1906.00150v5.pdf
PWC	https://paperswithcode.com/paper/190600150
Repo
Framework

Generative Imputation and Stochastic Prediction


Title	Generative Imputation and Stochastic Prediction
Authors	Mohammad Kachuee, Kimmo Karkkainen, Orpaz Goldstein, Sajad Darabi, Majid Sarrafzadeh
Abstract	In many machine learning applications, we are faced with incomplete datasets. In the literature, missing data imputation techniques have been mostly concerned with filling missing values. However, the existence of missing values is synonymous with uncertainties not only over the distribution of missing values but also over target class assignments that require careful consideration In this paper, we propose a simple and effective method for imputing missing features and estimating the distribution of target assignments given incomplete data. In order to make imputations, we train a simple and effective generator network to generate imputations that a discriminator network is tasked to distinguish. Following this, a predictor network is trained using the imputed samples from the generator network to capture the classification uncertainties and make predictions accordingly. The proposed method is evaluated on CIFAR-10 image dataset as well as three real-world tabular classification datasets, under different missingness rates and structures. Our experimental results show the effectiveness of the proposed method in generating imputations as well as providing estimates for the class uncertainties in a classification task when faced with missing values.
Tasks	Imputation
Published	2019-05-22
URL	https://arxiv.org/abs/1905.09340v3
PDF	https://arxiv.org/pdf/1905.09340v3.pdf
PWC	https://paperswithcode.com/paper/generative-imputation-and-stochastic
Repo
Framework

Cross validation in sparse linear regression with piecewise continuous nonconvex penalties and its acceleration


Title	Cross validation in sparse linear regression with piecewise continuous nonconvex penalties and its acceleration
Authors	Tomoyuki Obuchi, Ayaka Sakata
Abstract	We investigate the signal reconstruction performance of sparse linear regression in the presence of noise when piecewise continuous nonconvex penalties are used. Among such penalties, we focus on the SCAD penalty. The contributions of this study are three-fold: We first present a theoretical analysis of a typical reconstruction performance, using the replica method, under the assumption that each component of the design matrix is given as an independent and identically distributed (i.i.d.) Gaussian variable. This clarifies the superiority of the SCAD estimator compared with $\ell_1$ in a wide parameter range, although the nonconvex nature of the penalty tends to lead to solution multiplicity in certain regions. This multiplicity is shown to be connected to replica symmetry breaking in the spin-glass theory. We also show that the global minimum of the mean square error between the estimator and the true signal is located in the replica symmetric phase. Second, we develop an approximate formula efficiently computing the cross-validation error without actually conducting the cross-validation, which is also applicable to the non-i.i.d. design matrices. It is shown that this formula is only applicable to the unique solution region and tends to be unstable in the multiple solution region. We implement instability detection procedures, which allows the approximate formula to stand alone and resultantly enables us to draw phase diagrams for any specific dataset. Third, we propose an annealing procedure, called nonconvexity annealing, to obtain the solution path efficiently. Numerical simulations are conducted on simulated datasets to examine these results to verify the theoretical results consistency and the approximate formula efficiency. Another numerical experiment on a real-world dataset is conducted; its results are consistent with those of earlier studies using the $\ell_0$ formulation.
Tasks
Published	2019-02-27
URL	https://arxiv.org/abs/1902.10375v2
PDF	https://arxiv.org/pdf/1902.10375v2.pdf
PWC	https://paperswithcode.com/paper/cross-validation-in-sparse-linear-regression
Repo
Framework

Learning Transparent Object Matting


Title	Learning Transparent Object Matting
Authors	Guanying Chen, Kai Han, Kwan-Yee K. Wong
Abstract	This paper addresses the problem of image matting for transparent objects. Existing approaches often require tedious capturing procedures and long processing time, which limit their practical use. In this paper, we formulate transparent object matting as a refractive flow estimation problem, and propose a deep learning framework, called TOM-Net, for learning the refractive flow. Our framework comprises two parts, namely a multi-scale encoder-decoder network for producing a coarse prediction, and a residual network for refinement. At test time, TOM-Net takes a single image as input, and outputs a matte (consisting of an object mask, an attenuation mask and a refractive flow field) in a fast feed-forward pass. As no off-the-shelf dataset is available for transparent object matting, we create a large-scale synthetic dataset consisting of $178K$ images of transparent objects rendered in front of images sampled from the Microsoft COCO dataset. We also capture a real dataset consisting of $876$ samples using $14$ transparent objects and $60$ background images. Besides, we show that our method can be easily extended to handle the cases where a trimap or a background image is available.Promising experimental results have been achieved on both synthetic and real data, which clearly demonstrate the effectiveness of our approach.
Tasks	Image Matting
Published	2019-07-25
URL	https://arxiv.org/abs/1907.11544v1
PDF	https://arxiv.org/pdf/1907.11544v1.pdf
PWC	https://paperswithcode.com/paper/learning-transparent-object-matting
Repo
Framework

RGB and LiDAR fusion based 3D Semantic Segmentation for Autonomous Driving


Title	RGB and LiDAR fusion based 3D Semantic Segmentation for Autonomous Driving
Authors	Khaled El Madawy, Hazem Rashed, Ahmad El Sallab, Omar Nasr, Hanan Kamel, Senthil Yogamani
Abstract	LiDAR has become a standard sensor for autonomous driving applications as they provide highly precise 3D point clouds. LiDAR is also robust for low-light scenarios at night-time or due to shadows where the performance of cameras is degraded. LiDAR perception is gradually becoming mature for algorithms including object detection and SLAM. However, semantic segmentation algorithm remains to be relatively less explored. Motivated by the fact that semantic segmentation is a mature algorithm on image data, we explore sensor fusion based 3D segmentation. Our main contribution is to convert the RGB image to a polar-grid mapping representation used for LiDAR and design early and mid-level fusion architectures. Additionally, we design a hybrid fusion architecture that combines both fusion algorithms. We evaluate our algorithm on KITTI dataset which provides segmentation annotation for cars, pedestrians and cyclists. We evaluate two state-of-the-art architectures namely SqueezeSeg and PointSeg and improve the mIoU score by 10 % in both cases relative to the LiDAR only baseline.
Tasks	3D Semantic Segmentation, Autonomous Driving, Object Detection, Semantic Segmentation, Sensor Fusion
Published	2019-06-01
URL	https://arxiv.org/abs/1906.00208v2
PDF	https://arxiv.org/pdf/1906.00208v2.pdf
PWC	https://paperswithcode.com/paper/190600208
Repo
Framework

Computational Concentration of Measure: Optimal Bounds, Reductions, and More


Title	Computational Concentration of Measure: Optimal Bounds, Reductions, and More
Authors	Omid Etesami, Saeed Mahloujifar, Mohammad Mahmoody
Abstract	Product measures of dimension $n$ are known to be concentrated in Hamming distance: for any set $S$ in the product space of probability $\epsilon$, a random point in the space, with probability $1-\delta$, has a neighbor in $S$ that is different from the original point in only $O(\sqrt{n\ln(1/(\epsilon\delta))})$ coordinates. We obtain the tight computational version of this result, showing how given a random point and access to an $S$-membership oracle, we can find such a close point in polynomial time. This resolves an open question of [Mahloujifar and Mahmoody, ALT 2019]. As corollaries, we obtain polynomial-time poisoning and (in certain settings) evasion attacks against learning algorithms when the original vulnerabilities have any cryptographically non-negligible probability. We call our algorithm MUCIO (“MUltiplicative Conditional Influence Optimizer”) since proceeding through the coordinates, it decides to change each coordinate of the given point based on a multiplicative version of the influence of that coordinate, where influence is computed conditioned on previously updated coordinates. We also define a new notion of algorithmic reduction between computational concentration of measure in different metric probability spaces. As an application, we get computational concentration of measure for high-dimensional Gaussian distributions under the $\ell_1$ metric. We prove several extensions to the results above: (1) Our computational concentration result is also true when the Hamming distance is weighted. (2) We obtain an algorithmic version of concentration around mean, more specifically, McDiarmid’s inequality. (3) Our result generalizes to discrete random processes, and this leads to new tampering algorithms for collective coin tossing protocols. (4) We prove exponential lower bounds on the average running time of non-adaptive query algorithms.
Tasks
Published	2019-07-11
URL	https://arxiv.org/abs/1907.05401v1
PDF	https://arxiv.org/pdf/1907.05401v1.pdf
PWC	https://paperswithcode.com/paper/computational-concentration-of-measure
Repo
Framework

Scalable Deep Convolutional Neural Networks for Sparse, Locally Dense Liquid Argon Time Projection Chamber Data


Title	Scalable Deep Convolutional Neural Networks for Sparse, Locally Dense Liquid Argon Time Projection Chamber Data
Authors	Laura Dominé, Kazuhiro Terao
Abstract	Deep convolutional neural networks (CNNs) show strong promise for analyzing scientific data in many domains including particle imaging detectors such as a liquid argon time projection chamber (LArTPC). Yet the high sparsity of LArTPC data challenges traditional CNNs which were designed for dense data such as photographs. A naive application of CNNs on LArTPC data results in inefficient computations and a poor scalability to large LArTPC detectors such as the Short Baseline Neutrino Program and Deep Underground Neutrino Experiment. Recently Submanifold Sparse Convolutional Networks (SSCNs) have been proposed to address this challenge. We report their performance on a 3D semantic segmentation task on simulated LArTPC samples. In comparison with standard CNNs, we observe that the computation memory and wall-time cost for inference are reduced by factor of 364 and 33 respectively without loss of accuracy. The same factors for 2D samples are found to be 93 and 3.1 respectively. Using SSCN, we present the first machine learning-based approach to the reconstruction of Michel electrons using public 3D LArTPC samples. We find a Michel electron identification efficiency of 93.9% with 96.7% of true positive rate. Reconstructed Michel electron clusters yield 95.4% in average pixel clustering efficiency and 95.5% in purity. The results are compelling to show strong promise of scalable data reconstruction technique using deep neural networks for large scale LArTPC detectors.
Tasks	3D Semantic Segmentation, Semantic Segmentation
Published	2019-03-13
URL	https://arxiv.org/abs/1903.05663v3
PDF	https://arxiv.org/pdf/1903.05663v3.pdf
PWC	https://paperswithcode.com/paper/scalable-deep-convolutional-neural-networks
Repo
Framework