July 27, 2019

2871 words 14 mins read

Paper Group ANR 663

Vectorial Dimension Reduction for Tensors Based on Bayesian Inference. Deep Gaussian Mixture Models. PCA in Data-Dependent Noise (Correlated-PCA): Nearly Optimal Finite Sample Guarantees. A Self-supervised Learning System for Object Detection using Physics Simulation and Multi-view Pose Estimation. Differentially Private Identity and Closeness Test …

Vectorial Dimension Reduction for Tensors Based on Bayesian Inference


Title	Vectorial Dimension Reduction for Tensors Based on Bayesian Inference
Authors	Fujiao Ju, Yanfeng Sun, Junbin Gao, Yongli Hu, Baocai Yin
Abstract	Dimensionality reduction for high-order tensors is a challenging problem. In conventional approaches, higher order tensors are `vectorized` via Tucker decomposition to obtain lower order tensors. This will destroy the inherent high-order structures or resulting in undesired tensors, respectively. This paper introduces a probabilistic vectorial dimensionality reduction model for tensorial data. The model represents a tensor by employing a linear combination of same order basis tensors, thus it offers a mechanism to directly reduce a tensor to a vector. Under this expression, the projection base of the model is based on the tensor CandeComp/PARAFAC (CP) decomposition and the number of free parameters in the model only grows linearly with the number of modes rather than exponentially. A Bayesian inference has been established via the variational EM approach. A criterion to set the parameters (factor number of CP decomposition and the number of extracted features) is empirically given. The model outperforms several existing PCA-based methods and CP decomposition on several publicly available databases in terms of classification and clustering accuracy.
Tasks	Bayesian Inference, Dimensionality Reduction
Published	2017-07-03
URL	http://arxiv.org/abs/1707.00380v1
PDF	http://arxiv.org/pdf/1707.00380v1.pdf
PWC	https://paperswithcode.com/paper/vectorial-dimension-reduction-for-tensors
Repo
Framework

Deep Gaussian Mixture Models


Title	Deep Gaussian Mixture Models
Authors	Cinzia Viroli, Geoffrey J. McLachlan
Abstract	Deep learning is a hierarchical inference method formed by subsequent multiple layers of learning able to more efficiently describe complex relationships. In this work, Deep Gaussian Mixture Models are introduced and discussed. A Deep Gaussian Mixture model (DGMM) is a network of multiple layers of latent variables, where, at each layer, the variables follow a mixture of Gaussian distributions. Thus, the deep mixture model consists of a set of nested mixtures of linear models, which globally provide a nonlinear model able to describe the data in a very flexible way. In order to avoid overparameterized solutions, dimension reduction by factor models can be applied at each layer of the architecture thus resulting in deep mixtures of factor analysers.
Tasks	Dimensionality Reduction
Published	2017-11-18
URL	http://arxiv.org/abs/1711.06929v1
PDF	http://arxiv.org/pdf/1711.06929v1.pdf
PWC	https://paperswithcode.com/paper/deep-gaussian-mixture-models
Repo
Framework

PCA in Data-Dependent Noise (Correlated-PCA): Nearly Optimal Finite Sample Guarantees


Title	PCA in Data-Dependent Noise (Correlated-PCA): Nearly Optimal Finite Sample Guarantees
Authors	Namrata Vaswani, Praneeth Narayanamurthy
Abstract	We study Principal Component Analysis (PCA) in a setting where a part of the corrupting noise is data-dependent and, as a result, the noise and the true data are correlated. Under a bounded-ness assumption on the true data and the noise, and a simple assumption on data-noise correlation, we obtain a nearly optimal sample complexity bound for the most commonly used PCA solution, singular value decomposition (SVD). This bound is a significant improvement over the bound obtained by Vaswani and Guo in recent work (NIPS 2016) where this “correlated-PCA” problem was first studied; and it holds under a significantly weaker data-noise correlation assumption than the one used for this earlier result.
Tasks
Published	2017-02-10
URL	http://arxiv.org/abs/1702.03070v3
PDF	http://arxiv.org/pdf/1702.03070v3.pdf
PWC	https://paperswithcode.com/paper/pca-in-data-dependent-noise-correlated-pca
Repo
Framework

A Self-supervised Learning System for Object Detection using Physics Simulation and Multi-view Pose Estimation


Title	A Self-supervised Learning System for Object Detection using Physics Simulation and Multi-view Pose Estimation
Authors	Chaitanya Mitash, Kostas E. Bekris, Abdeslam Boularias
Abstract	Progress has been achieved recently in object detection given advancements in deep learning. Nevertheless, such tools typically require a large amount of training data and significant manual effort to label objects. This limits their applicability in robotics, where solutions must scale to a large number of objects and variety of conditions. This work proposes an autonomous process for training a Convolutional Neural Network (CNN) for object detection and pose estimation in robotic setups. The focus is on detecting objects placed in cluttered, tight environments, such as a shelf with multiple objects. In particular, given access to 3D object models, several aspects of the environment are physically simulated. The models are placed in physically realistic poses with respect to their environment to generate a labeled synthetic dataset. To further improve object detection, the network self-trains over real images that are labeled using a robust multi-view pose estimation process. The proposed training process is evaluated on several existing datasets and on a dataset collected for this paper with a Motoman robotic arm. Results show that the proposed approach outperforms popular training processes relying on synthetic - but not physically realistic - data and manual annotation. The key contributions are the incorporation of physical reasoning in the synthetic data generation process and the automation of the annotation process over real images.
Tasks	Object Detection, Pose Estimation, Synthetic Data Generation
Published	2017-03-09
URL	http://arxiv.org/abs/1703.03347v2
PDF	http://arxiv.org/pdf/1703.03347v2.pdf
PWC	https://paperswithcode.com/paper/a-self-supervised-learning-system-for-object
Repo
Framework

Differentially Private Identity and Closeness Testing of Discrete Distributions


Title	Differentially Private Identity and Closeness Testing of Discrete Distributions
Authors	Maryam Aliakbarpour, Ilias Diakonikolas, Ronitt Rubinfeld
Abstract	We investigate the problems of identity and closeness testing over a discrete population from random samples. Our goal is to develop efficient testers while guaranteeing Differential Privacy to the individuals of the population. We describe an approach that yields sample-efficient differentially private testers for these problems. Our theoretical results show that there exist private identity and closeness testers that are nearly as sample-efficient as their non-private counterparts. We perform an experimental evaluation of our algorithms on synthetic data. Our experiments illustrate that our private testers achieve small type I and type II errors with sample size sublinear in the domain size of the underlying distributions.
Tasks
Published	2017-07-18
URL	http://arxiv.org/abs/1707.05497v1
PDF	http://arxiv.org/pdf/1707.05497v1.pdf
PWC	https://paperswithcode.com/paper/differentially-private-identity-and-closeness
Repo
Framework

Dual Iterative Hard Thresholding: From Non-convex Sparse Minimization to Non-smooth Concave Maximization


Title	Dual Iterative Hard Thresholding: From Non-convex Sparse Minimization to Non-smooth Concave Maximization
Authors	Bo Liu, Xiao-Tong Yuan, Lezi Wang, Qingshan Liu, Dimitris N. Metaxas
Abstract	Iterative Hard Thresholding (IHT) is a class of projected gradient descent methods for optimizing sparsity-constrained minimization models, with the best known efficiency and scalability in practice. As far as we know, the existing IHT-style methods are designed for sparse minimization in primal form. It remains open to explore duality theory and algorithms in such a non-convex and NP-hard problem setting. In this paper, we bridge this gap by establishing a duality theory for sparsity-constrained minimization with $\ell_2$-regularized loss function and proposing an IHT-style algorithm for dual maximization. Our sparse duality theory provides a set of sufficient and necessary conditions under which the original NP-hard/non-convex problem can be equivalently solved in a dual formulation. The proposed dual IHT algorithm is a super-gradient method for maximizing the non-smooth dual objective. An interesting finding is that the sparse recovery performance of dual IHT is invariant to the Restricted Isometry Property (RIP), which is required by virtually all the existing primal IHT algorithms without sparsity relaxation. Moreover, a stochastic variant of dual IHT is proposed for large-scale stochastic optimization. Numerical results demonstrate the superiority of dual IHT algorithms to the state-of-the-art primal IHT-style algorithms in model estimation accuracy and computational efficiency.
Tasks	Stochastic Optimization
Published	2017-03-01
URL	http://arxiv.org/abs/1703.00119v2
PDF	http://arxiv.org/pdf/1703.00119v2.pdf
PWC	https://paperswithcode.com/paper/dual-iterative-hard-thresholding-from-non
Repo
Framework

Deep EndoVO: A Recurrent Convolutional Neural Network (RCNN) based Visual Odometry Approach for Endoscopic Capsule Robots


Title	Deep EndoVO: A Recurrent Convolutional Neural Network (RCNN) based Visual Odometry Approach for Endoscopic Capsule Robots
Authors	Mehmet Turan, Yasin Almalioglu, Helder Araujo, Ender Konukoglu, Metin Sitti
Abstract	Ingestible wireless capsule endoscopy is an emerging minimally invasive diagnostic technology for inspection of the GI tract and diagnosis of a wide range of diseases and pathologies. Medical device companies and many research groups have recently made substantial progresses in converting passive capsule endoscopes to active capsule robots, enabling more accurate, precise, and intuitive detection of the location and size of the diseased areas. Since a reliable real time pose estimation functionality is crucial for actively controlled endoscopic capsule robots, in this study, we propose a monocular visual odometry (VO) method for endoscopic capsule robot operations. Our method lies on the application of the deep Recurrent Convolutional Neural Networks (RCNNs) for the visual odometry task, where Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs) are used for the feature extraction and inference of dynamics across the frames, respectively. Detailed analyses and evaluations made on a real pig stomach dataset proves that our system achieves high translational and rotational accuracies for different types of endoscopic capsule robot trajectories.
Tasks	Monocular Visual Odometry, Pose Estimation, Visual Odometry
Published	2017-08-22
URL	http://arxiv.org/abs/1708.06822v2
PDF	http://arxiv.org/pdf/1708.06822v2.pdf
PWC	https://paperswithcode.com/paper/deep-endovo-a-recurrent-convolutional-neural
Repo
Framework

Tree-Structured Boosting: Connections Between Gradient Boosted Stumps and Full Decision Trees


Title	Tree-Structured Boosting: Connections Between Gradient Boosted Stumps and Full Decision Trees
Authors	José Marcio Luna, Eric Eaton, Lyle H. Ungar, Eric Diffenderfer, Shane T. Jensen, Efstathios D. Gennatas, Mateo Wirth, Charles B. Simone II, Timothy D. Solberg, Gilmer Valdes
Abstract	Additive models, such as produced by gradient boosting, and full interaction models, such as classification and regression trees (CART), are widely used algorithms that have been investigated largely in isolation. We show that these models exist along a spectrum, revealing never-before-known connections between these two approaches. This paper introduces a novel technique called tree-structured boosting for creating a single decision tree, and shows that this method can produce models equivalent to CART or gradient boosted stumps at the extremes by varying a single parameter. Although tree-structured boosting is designed primarily to provide both the model interpretability and predictive performance needed for high-stake applications like medicine, it also can produce decision trees represented by hybrid models between CART and boosted stumps that can outperform either of these approaches.
Tasks
Published	2017-11-18
URL	http://arxiv.org/abs/1711.06793v1
PDF	http://arxiv.org/pdf/1711.06793v1.pdf
PWC	https://paperswithcode.com/paper/tree-structured-boosting-connections-between
Repo
Framework

Universal consistency and minimax rates for online Mondrian Forests


Title	Universal consistency and minimax rates for online Mondrian Forests
Authors	Jaouad Mourtada, Stéphane Gaïffas, Erwan Scornet
Abstract	We establish the consistency of an algorithm of Mondrian Forests, a randomized classification algorithm that can be implemented online. First, we amend the original Mondrian Forest algorithm, that considers a fixed lifetime parameter. Indeed, the fact that this parameter is fixed hinders the statistical consistency of the original procedure. Our modified Mondrian Forest algorithm grows trees with increasing lifetime parameters $\lambda_n$, and uses an alternative updating rule, allowing to work also in an online fashion. Second, we provide a theoretical analysis establishing simple conditions for consistency. Our theoretical analysis also exhibits a surprising fact: our algorithm achieves the minimax rate (optimal rate) for the estimation of a Lipschitz regression function, which is a strong extension of previous results to an arbitrary dimension.
Tasks
Published	2017-11-08
URL	http://arxiv.org/abs/1711.02887v1
PDF	http://arxiv.org/pdf/1711.02887v1.pdf
PWC	https://paperswithcode.com/paper/universal-consistency-and-minimax-rates-for
Repo
Framework

From Preference-Based to Multiobjective Sequential Decision-Making


Title	From Preference-Based to Multiobjective Sequential Decision-Making
Authors	Paul Weng
Abstract	In this paper, we present a link between preference-based and multiobjective sequential decision-making. While transforming a multiobjective problem to a preference-based one is quite natural, the other direction is a bit less obvious. We present how this transformation (from preference-based to multiobjective) can be done under the classic condition that preferences over histories can be represented by additively decomposable utilities and that the decision criterion to evaluate policies in a state is based on expectation. This link yields a new source of multiobjective sequential decision-making problems (i.e., when reward values are unknown) and justifies the use of solving methods developed in one setting in the other one.
Tasks	Decision Making
Published	2017-01-03
URL	http://arxiv.org/abs/1701.00646v1
PDF	http://arxiv.org/pdf/1701.00646v1.pdf
PWC	https://paperswithcode.com/paper/from-preference-based-to-multiobjective
Repo
Framework

Ensemble Methods as a Defense to Adversarial Perturbations Against Deep Neural Networks


Title	Ensemble Methods as a Defense to Adversarial Perturbations Against Deep Neural Networks
Authors	Thilo Strauss, Markus Hanselmann, Andrej Junginger, Holger Ulmer
Abstract	Deep learning has become the state of the art approach in many machine learning problems such as classification. It has recently been shown that deep learning is highly vulnerable to adversarial perturbations. Taking the camera systems of self-driving cars as an example, small adversarial perturbations can cause the system to make errors in important tasks, such as classifying traffic signs or detecting pedestrians. Hence, in order to use deep learning without safety concerns a proper defense strategy is required. We propose to use ensemble methods as a defense strategy against adversarial perturbations. We find that an attack leading one model to misclassify does not imply the same for other networks performing the same task. This makes ensemble methods an attractive defense strategy against adversarial attacks. We empirically show for the MNIST and the CIFAR-10 data sets that ensemble methods not only improve the accuracy of neural networks on test data but also increase their robustness against adversarial perturbations.
Tasks	Self-Driving Cars
Published	2017-09-11
URL	http://arxiv.org/abs/1709.03423v2
PDF	http://arxiv.org/pdf/1709.03423v2.pdf
PWC	https://paperswithcode.com/paper/ensemble-methods-as-a-defense-to-adversarial
Repo
Framework

Guiding Network Analysis using Graph Slepians: An Illustration for the C. Elegans Connectome


Title	Guiding Network Analysis using Graph Slepians: An Illustration for the C. Elegans Connectome
Authors	Dimitri Van De Ville, Robin Demesmaeker, Maria Giulia Preti
Abstract	Spectral approaches of network analysis heavily rely upon the eigendecomposition of the graph Laplacian. For instance, in graph signal processing, the Laplacian eigendecomposition is used to define the graph Fourier transform and then transpose signal processing operations to graphs by implementing them in the spectral domain. Here, we build on recent work that generalized Slepian functions to the graph setting. In particular, graph Slepians are band-limited graph signals with maximal energy concentration in a given subgraph. We show how this approach can be used to guide network analysis; i.e., we propose a visualization that reveals network organization of a subgraph, but while striking a balance with global network structure. These developments are illustrated for the structural connectome of the C. Elegans.
Tasks
Published	2017-08-15
URL	http://arxiv.org/abs/1708.04657v2
PDF	http://arxiv.org/pdf/1708.04657v2.pdf
PWC	https://paperswithcode.com/paper/guiding-network-analysis-using-graph-slepians
Repo
Framework

Fast Restricted Causal Inference


Title	Fast Restricted Causal Inference
Authors	Mieczysław A. Kłopotek
Abstract	Hidden variables are well known sources of disturbance when recovering belief networks from data based only on measurable variables. Hence models assuming existence of hidden variables are under development. This paper presents a new algorithm “accelerating” the known CI algorithm of Spirtes, Glymour and Scheines {Spirtes:93}. We prove that this algorithm does not produces (conditional) independencies not present in the data if statistical independence test is reliable. This result is to be considered as non-trivial since e.g. the same claim fails to be true for FCI algorithm, another “accelerator” of CI, developed in {Spirtes:93}.
Tasks	Causal Inference
Published	2017-07-13
URL	http://arxiv.org/abs/1707.04584v1
PDF	http://arxiv.org/pdf/1707.04584v1.pdf
PWC	https://paperswithcode.com/paper/fast-restricted-causal-inference
Repo
Framework

Deep Counterfactual Networks with Propensity-Dropout


Title	Deep Counterfactual Networks with Propensity-Dropout
Authors	Ahmed M. Alaa, Michael Weisz, Mihaela van der Schaar
Abstract	We propose a novel approach for inferring the individualized causal effects of a treatment (intervention) from observational data. Our approach conceptualizes causal inference as a multitask learning problem; we model a subject’s potential outcomes using a deep multitask network with a set of shared layers among the factual and counterfactual outcomes, and a set of outcome-specific layers. The impact of selection bias in the observational data is alleviated via a propensity-dropout regularization scheme, in which the network is thinned for every training example via a dropout probability that depends on the associated propensity score. The network is trained in alternating phases, where in each phase we use the training examples of one of the two potential outcomes (treated and control populations) to update the weights of the shared layers and the respective outcome-specific layers. Experiments conducted on data based on a real-world observational study show that our algorithm outperforms the state-of-the-art.
Tasks	Causal Inference
Published	2017-06-19
URL	http://arxiv.org/abs/1706.05966v1
PDF	http://arxiv.org/pdf/1706.05966v1.pdf
PWC	https://paperswithcode.com/paper/deep-counterfactual-networks-with-propensity
Repo
Framework

A Deep Causal Inference Approach to Measuring the Effects of Forming Group Loans in Online Non-profit Microfinance Platform


Title	A Deep Causal Inference Approach to Measuring the Effects of Forming Group Loans in Online Non-profit Microfinance Platform
Authors	Thai T. Pham, Yuanyuan Shen
Abstract	Kiva is an online non-profit crowdsouring microfinance platform that raises funds for the poor in the third world. The borrowers on Kiva are small business owners and individuals in urgent need of money. To raise funds as fast as possible, they have the option to form groups and post loan requests in the name of their groups. While it is generally believed that group loans pose less risk for investors than individual loans do, we study whether this is the case in a philanthropic online marketplace. In particular, we measure the effect of group loans on funding time while controlling for the loan sizes and other factors. Because loan descriptions (in the form of texts) play an important role in lenders’ decision process on Kiva, we make use of this information through deep learning in natural language processing. In this aspect, this is the first paper that uses one of the most advanced deep learning techniques to deal with unstructured data in a way that can take advantage of its superior prediction power to answer causal questions. We find that on average, forming group loans speeds up the funding time by about 3.3 days.
Tasks	Causal Inference
Published	2017-06-08
URL	http://arxiv.org/abs/1706.02795v1
PDF	http://arxiv.org/pdf/1706.02795v1.pdf
PWC	https://paperswithcode.com/paper/a-deep-causal-inference-approach-to-measuring
Repo
Framework