Paper Group ANR 485
Unsupervised Transformation Learning via Convex Relaxations. Phase Diagram of Restricted Boltzmann Machines and Generalised Hopfield Networks with Arbitrary Priors. Guided Machine Learning for power grid segmentation. Theoretical and Computational Guarantees of Mean Field Variational Inference for Community Detection. Robust Photometric Stereo Usin …
Unsupervised Transformation Learning via Convex Relaxations
| Title | Unsupervised Transformation Learning via Convex Relaxations |
| Authors | Tatsunori B. Hashimoto, John C. Duchi, Percy Liang |
| Abstract | Our goal is to extract meaningful transformations from raw images, such as varying the thickness of lines in handwriting or the lighting in a portrait. We propose an unsupervised approach to learn such transformations by attempting to reconstruct an image from a linear combination of transformations of its nearest neighbors. On handwritten digits and celebrity portraits, we show that even with linear transformations, our method generates visually high-quality modified images. Moreover, since our method is semiparametric and does not model the data distribution, the learned transformations extrapolate off the training data and can be applied to new types of images. |
| Tasks | |
| Published | 2017-11-06 |
| URL | http://arxiv.org/abs/1711.02226v1 |
| http://arxiv.org/pdf/1711.02226v1.pdf | |
| PWC | https://paperswithcode.com/paper/unsupervised-transformation-learning-via |
| Repo | |
| Framework | |
Phase Diagram of Restricted Boltzmann Machines and Generalised Hopfield Networks with Arbitrary Priors
| Title | Phase Diagram of Restricted Boltzmann Machines and Generalised Hopfield Networks with Arbitrary Priors |
| Authors | Adriano Barra, Giuseppe Genovese, Peter Sollich, Daniele Tantari |
| Abstract | Restricted Boltzmann Machines are described by the Gibbs measure of a bipartite spin glass, which in turn corresponds to the one of a generalised Hopfield network. This equivalence allows us to characterise the state of these systems in terms of retrieval capabilities, both at low and high load. We study the paramagnetic-spin glass and the spin glass-retrieval phase transitions, as the pattern (i.e. weight) distribution and spin (i.e. unit) priors vary smoothly from Gaussian real variables to Boolean discrete variables. Our analysis shows that the presence of a retrieval phase is robust and not peculiar to the standard Hopfield model with Boolean patterns. The retrieval region is larger when the pattern entries and retrieval units get more peaked and, conversely, when the hidden units acquire a broader prior and therefore have a stronger response to high fields. Moreover, at low load retrieval always exists below some critical temperature, for every pattern distribution ranging from the Boolean to the Gaussian case. |
| Tasks | |
| Published | 2017-02-20 |
| URL | http://arxiv.org/abs/1702.05882v2 |
| http://arxiv.org/pdf/1702.05882v2.pdf | |
| PWC | https://paperswithcode.com/paper/phase-diagram-of-restricted-boltzmann |
| Repo | |
| Framework | |
Guided Machine Learning for power grid segmentation
| Title | Guided Machine Learning for power grid segmentation |
| Authors | Antoine Marot, Sami Tazi, Benjamin Donnot, Patrick Panciatici |
| Abstract | The segmentation of large scale power grids into zones is crucial for control room operators when managing the grid complexity near real time. In this paper we propose a new method in two steps which is able to automatically do this segmentation, while taking into account the real time context, in order to help them handle shifting dynamics. Our method relies on a “guided” machine learning approach. As a first step, we define and compute a task specific “Influence Graph” in a guided manner. We indeed simulate on a grid state chosen interventions, representative of our task of interest (managing active power flows in our case). For visualization and interpretation, we then build a higher representation of the grid relevant to this task by applying the graph community detection algorithm \textit{Infomap} on this Influence Graph. To illustrate our method and demonstrate its practical interest, we apply it on commonly used systems, the IEEE-14 and IEEE-118. We show promising and original interpretable results, especially on the previously well studied RTS-96 system for grid segmentation. We eventually share initial investigation and results on a large-scale system, the French power grid, whose segmentation had a surprising resemblance with RTE’s historical partitioning. |
| Tasks | Community Detection |
| Published | 2017-11-13 |
| URL | http://arxiv.org/abs/1711.09715v3 |
| http://arxiv.org/pdf/1711.09715v3.pdf | |
| PWC | https://paperswithcode.com/paper/guided-machine-learning-for-power-grid |
| Repo | |
| Framework | |
Theoretical and Computational Guarantees of Mean Field Variational Inference for Community Detection
| Title | Theoretical and Computational Guarantees of Mean Field Variational Inference for Community Detection |
| Authors | Anderson Y. Zhang, Harrison H. Zhou |
| Abstract | The mean field variational Bayes method is becoming increasingly popular in statistics and machine learning. Its iterative Coordinate Ascent Variational Inference algorithm has been widely applied to large scale Bayesian inference. See Blei et al. (2017) for a recent comprehensive review. Despite the popularity of the mean field method there exist remarkably little fundamental theoretical justifications. To the best of our knowledge, the iterative algorithm has never been investigated for any high dimensional and complex model. In this paper, we study the mean field method for community detection under the Stochastic Block Model. For an iterative Batch Coordinate Ascent Variational Inference algorithm, we show that it has a linear convergence rate and converges to the minimax rate within $\log n$ iterations. This complements the results of Bickel et al. (2013) which studied the global minimum of the mean field variational Bayes and obtained asymptotic normal estimation of global model parameters. In addition, we obtain similar optimality results for Gibbs sampling and an iterative procedure to calculate maximum likelihood estimation, which can be of independent interest. |
| Tasks | Bayesian Inference, Community Detection |
| Published | 2017-10-30 |
| URL | http://arxiv.org/abs/1710.11268v3 |
| http://arxiv.org/pdf/1710.11268v3.pdf | |
| PWC | https://paperswithcode.com/paper/theoretical-and-computational-guarantees-of |
| Repo | |
| Framework | |
Robust Photometric Stereo Using Learned Image and Gradient Dictionaries
| Title | Robust Photometric Stereo Using Learned Image and Gradient Dictionaries |
| Authors | Andrew J. Wagenmaker, Brian E. Moore, Raj Rao Nadakuditi |
| Abstract | Photometric stereo is a method for estimating the normal vectors of an object from images of the object under varying lighting conditions. Motivated by several recent works that extend photometric stereo to more general objects and lighting conditions, we study a new robust approach to photometric stereo that utilizes dictionary learning. Specifically, we propose and analyze two approaches to adaptive dictionary regularization for the photometric stereo problem. First, we propose an image preprocessing step that utilizes an adaptive dictionary learning model to remove noise and other non-idealities from the image dataset before estimating the normal vectors. We also propose an alternative model where we directly apply the adaptive dictionary regularization to the normal vectors themselves during estimation. We study the practical performance of both methods through extensive simulations, which demonstrate the state-of-the-art performance of both methods in the presence of noise. |
| Tasks | Dictionary Learning |
| Published | 2017-09-30 |
| URL | http://arxiv.org/abs/1710.00002v1 |
| http://arxiv.org/pdf/1710.00002v1.pdf | |
| PWC | https://paperswithcode.com/paper/robust-photometric-stereo-using-learned-image |
| Repo | |
| Framework | |
Primal-Dual Optimization Algorithms over Riemannian Manifolds: an Iteration Complexity Analysis
| Title | Primal-Dual Optimization Algorithms over Riemannian Manifolds: an Iteration Complexity Analysis |
| Authors | Junyu Zhang, Shiqian Ma, Shuzhong Zhang |
| Abstract | In this paper we study nonconvex and nonsmooth multi-block optimization over Riemannian manifolds with coupled linear constraints. Such optimization problems naturally arise from machine learning, statistical learning, compressive sensing, image processing, and tensor PCA, among others. We develop an ADMM-like primal-dual approach based on decoupled solvable subroutines such as linearized proximal mappings. First, we introduce the optimality conditions for the afore-mentioned optimization models. Then, the notion of $\epsilon$-stationary solutions is introduced as a result. The main part of the paper is to show that the proposed algorithms enjoy an iteration complexity of $O(1/\epsilon^2)$ to reach an $\epsilon$-stationary solution. For prohibitively large-size tensor or machine learning models, we present a sampling-based stochastic algorithm with the same iteration complexity bound in expectation. In case the subproblems are not analytically solvable, a feasible curvilinear line-search variant of the algorithm based on retraction operators is proposed. Finally, we show specifically how the algorithms can be implemented to solve a variety of practical problems such as the NP-hard maximum bisection problem, the $\ell_q$ regularized sparse tensor principal component analysis and the community detection problem. Our preliminary numerical results show great potentials of the proposed methods. |
| Tasks | Community Detection, Compressive Sensing |
| Published | 2017-10-05 |
| URL | http://arxiv.org/abs/1710.02236v1 |
| http://arxiv.org/pdf/1710.02236v1.pdf | |
| PWC | https://paperswithcode.com/paper/primal-dual-optimization-algorithms-over |
| Repo | |
| Framework | |
Bayesian estimation from few samples: community detection and related problems
| Title | Bayesian estimation from few samples: community detection and related problems |
| Authors | Samuel B. Hopkins, David Steurer |
| Abstract | We propose an efficient meta-algorithm for Bayesian estimation problems that is based on low-degree polynomials, semidefinite programming, and tensor decomposition. The algorithm is inspired by recent lower bound constructions for sum-of-squares and related to the method of moments. Our focus is on sample complexity bounds that are as tight as possible (up to additive lower-order terms) and often achieve statistical thresholds or conjectured computational thresholds. Our algorithm recovers the best known bounds for community detection in the sparse stochastic block model, a widely-studied class of estimation problems for community detection in graphs. We obtain the first recovery guarantees for the mixed-membership stochastic block model (Airoldi et el.) in constant average degree graphs—up to what we conjecture to be the computational threshold for this model. We show that our algorithm exhibits a sharp computational threshold for the stochastic block model with multiple communities beyond the Kesten–Stigum bound—giving evidence that this task may require exponential time. The basic strategy of our algorithm is strikingly simple: we compute the best-possible low-degree approximation for the moments of the posterior distribution of the parameters and use a robust tensor decomposition algorithm to recover the parameters from these approximate posterior moments. |
| Tasks | Community Detection |
| Published | 2017-09-30 |
| URL | http://arxiv.org/abs/1710.00264v1 |
| http://arxiv.org/pdf/1710.00264v1.pdf | |
| PWC | https://paperswithcode.com/paper/bayesian-estimation-from-few-samples |
| Repo | |
| Framework | |
Lattice Long Short-Term Memory for Human Action Recognition
| Title | Lattice Long Short-Term Memory for Human Action Recognition |
| Authors | Lin Sun, Kui Jia, Kevin Chen, Dit Yan Yeung, Bertram E. Shi, Silvio Savarese |
| Abstract | Human actions captured in video sequences are three-dimensional signals characterizing visual appearance and motion dynamics. To learn action patterns, existing methods adopt Convolutional and/or Recurrent Neural Networks (CNNs and RNNs). CNN based methods are effective in learning spatial appearances, but are limited in modeling long-term motion dynamics. RNNs, especially Long Short-Term Memory (LSTM), are able to learn temporal motion dynamics. However, naively applying RNNs to video sequences in a convolutional manner implicitly assumes that motions in videos are stationary across different spatial locations. This assumption is valid for short-term motions but invalid when the duration of the motion is long. In this work, we propose Lattice-LSTM (L2STM), which extends LSTM by learning independent hidden state transitions of memory cells for individual spatial locations. This method effectively enhances the ability to model dynamics across time and addresses the non-stationary issue of long-term motion dynamics without significantly increasing the model complexity. Additionally, we introduce a novel multi-modal training procedure for training our network. Unlike traditional two-stream architectures which use RGB and optical flow information as input, our two-stream model leverages both modalities to jointly train both input gates and both forget gates in the network rather than treating the two streams as separate entities with no information about the other. We apply this end-to-end system to benchmark datasets (UCF-101 and HMDB-51) of human action recognition. Experiments show that on both datasets, our proposed method outperforms all existing ones that are based on LSTM and/or CNNs of similar model complexities. |
| Tasks | Optical Flow Estimation, Temporal Action Localization |
| Published | 2017-08-13 |
| URL | http://arxiv.org/abs/1708.03958v1 |
| http://arxiv.org/pdf/1708.03958v1.pdf | |
| PWC | https://paperswithcode.com/paper/lattice-long-short-term-memory-for-human |
| Repo | |
| Framework | |
A simple expression for the map of Asplund’s distances with the multiplicative Logarithmic Image Processing (LIP) law
| Title | A simple expression for the map of Asplund’s distances with the multiplicative Logarithmic Image Processing (LIP) law |
| Authors | Guillaume Noyel, Michel Jourlin |
| Abstract | We introduce a simple expression for the map of Asplund’s distances with the multiplicative Logarithmic Image Processing (LIP) law. It is a difference between a morphological dilation and a morphological erosion with an additive structuring function which corresponds to a morphological gradient. |
| Tasks | |
| Published | 2017-08-23 |
| URL | http://arxiv.org/abs/1708.08992v2 |
| http://arxiv.org/pdf/1708.08992v2.pdf | |
| PWC | https://paperswithcode.com/paper/a-simple-expression-for-the-map-of-asplunds |
| Repo | |
| Framework | |
Maximum A Posteriori Estimation of Distances Between Deep Features in Still-to-Video Face Recognition
| Title | Maximum A Posteriori Estimation of Distances Between Deep Features in Still-to-Video Face Recognition |
| Authors | Andrey V. Savchenko, Natalya S. Belova |
| Abstract | The paper deals with the still-to-video face recognition for the small sample size problem based on computation of distances between high-dimensional deep bottleneck features. We present the novel statistical recognition method, in which the still-to-video recognition task is casted into Maximum A Posteriori estimation. In this method we maximize the joint probabilistic density of the distances to all reference still images. It is shown that this likelihood can be estimated with the known asymptotically normal distribution of the Kullback-Leibler discriminations between nonnegative features. The experimental study with the LFW (Labeled Faces in the Wild), YTF (YouTube Faces) and IJB-A (IARPA Janus Benchmark A) datasets has been provided. We demonstrated, that the proposed approach can be applied with the state-of-the-art deep features and dissimilarity measures. Our algorithm achieves 3-5% higher accuracy when compared with conventional aggregation of decisions obtained for all frames. |
| Tasks | Face Recognition, Video Recognition |
| Published | 2017-08-26 |
| URL | http://arxiv.org/abs/1708.07972v1 |
| http://arxiv.org/pdf/1708.07972v1.pdf | |
| PWC | https://paperswithcode.com/paper/maximum-a-posteriori-estimation-of-distances |
| Repo | |
| Framework | |
Learning a bidirectional mapping between human whole-body motion and natural language using deep recurrent neural networks
| Title | Learning a bidirectional mapping between human whole-body motion and natural language using deep recurrent neural networks |
| Authors | Matthias Plappert, Christian Mandery, Tamim Asfour |
| Abstract | Linking human whole-body motion and natural language is of great interest for the generation of semantic representations of observed human behaviors as well as for the generation of robot behaviors based on natural language input. While there has been a large body of research in this area, most approaches that exist today require a symbolic representation of motions (e.g. in the form of motion primitives), which have to be defined a-priori or require complex segmentation algorithms. In contrast, recent advances in the field of neural networks and especially deep learning have demonstrated that sub-symbolic representations that can be learned end-to-end usually outperform more traditional approaches, for applications such as machine translation. In this paper we propose a generative model that learns a bidirectional mapping between human whole-body motion and natural language using deep recurrent neural networks (RNNs) and sequence-to-sequence learning. Our approach does not require any segmentation or manual feature engineering and learns a distributed representation, which is shared for all motions and descriptions. We evaluate our approach on 2,846 human whole-body motions and 6,187 natural language descriptions thereof from the KIT Motion-Language Dataset. Our results clearly demonstrate the effectiveness of the proposed model: We show that our model generates a wide variety of realistic motions only from descriptions thereof in form of a single sentence. Conversely, our model is also capable of generating correct and detailed natural language descriptions from human motions. |
| Tasks | Feature Engineering, Machine Translation |
| Published | 2017-05-18 |
| URL | http://arxiv.org/abs/1705.06400v2 |
| http://arxiv.org/pdf/1705.06400v2.pdf | |
| PWC | https://paperswithcode.com/paper/learning-a-bidirectional-mapping-between |
| Repo | |
| Framework | |
Fast PET reconstruction using Multi-scale Fully Convolutional Neural Networks
| Title | Fast PET reconstruction using Multi-scale Fully Convolutional Neural Networks |
| Authors | Jieqing Jiao, Sebastien Ourselin |
| Abstract | Reconstruction of PET images is an ill-posed inverse problem and often requires iterative algorithms to achieve good image quality for reliable clinical use in practice, at huge computational costs. In this paper, we consider the PET reconstruction a dense prediction problem where the large scale contextual information is essential, and propose a novel architecture of multi-scale fully convolutional neural networks (msfCNN) for fast PET image reconstruction. The proposed msfCNN gains large receptive fields with both memory and computational efficiency, by using a downscaling-upscaling structure and dilated convolutions. Instead of pooling and deconvolution, we propose to use the periodic shuffling operation from sub-pixel convolution and its inverse to scale the size of feature maps without losing resolution. Residual connections were added to improve training. We trained the proposed msfCNN model with simulated data, and applied it to clinical PET data acquired on a Siemens mMR scanner. The results from real oncological and neurodegenerative cases show that the proposed msfCNN-based reconstruction outperforms the iterative approaches in terms of computational time while achieving comparable image quality for quantification. The proposed msfCNN model can be applied to other dense prediction tasks, and fast msfCNN-based PET reconstruction could facilitate the potential use of molecular imaging in interventional/surgical procedures, where cancer surgery can particularly benefit. |
| Tasks | Image Reconstruction |
| Published | 2017-04-24 |
| URL | http://arxiv.org/abs/1704.07244v1 |
| http://arxiv.org/pdf/1704.07244v1.pdf | |
| PWC | https://paperswithcode.com/paper/fast-pet-reconstruction-using-multi-scale |
| Repo | |
| Framework | |
Stacked Structure Learning for Lifted Relational Neural Networks
| Title | Stacked Structure Learning for Lifted Relational Neural Networks |
| Authors | Gustav Sourek, Martin Svatos, Filip Zelezny, Steven Schockaert, Ondrej Kuzelka |
| Abstract | Lifted Relational Neural Networks (LRNNs) describe relational domains using weighted first-order rules which act as templates for constructing feed-forward neural networks. While previous work has shown that using LRNNs can lead to state-of-the-art results in various ILP tasks, these results depended on hand-crafted rules. In this paper, we extend the framework of LRNNs with structure learning, thus enabling a fully automated learning process. Similarly to many ILP methods, our structure learning algorithm proceeds in an iterative fashion by top-down searching through the hypothesis space of all possible Horn clauses, considering the predicates that occur in the training examples as well as invented soft concepts entailed by the best weighted rules found so far. In the experiments, we demonstrate the ability to automatically induce useful hierarchical soft concepts leading to deep LRNNs with a competitive predictive power. |
| Tasks | |
| Published | 2017-10-05 |
| URL | http://arxiv.org/abs/1710.02221v1 |
| http://arxiv.org/pdf/1710.02221v1.pdf | |
| PWC | https://paperswithcode.com/paper/stacked-structure-learning-for-lifted |
| Repo | |
| Framework | |
Bayesian nonparametric Principal Component Analysis
| Title | Bayesian nonparametric Principal Component Analysis |
| Authors | Clément Elvira, Pierre Chainais, Nicolas Dobigeon |
| Abstract | Principal component analysis (PCA) is very popular to perform dimension reduction. The selection of the number of significant components is essential but often based on some practical heuristics depending on the application. Only few works have proposed a probabilistic approach able to infer the number of significant components. To this purpose, this paper introduces a Bayesian nonparametric principal component analysis (BNP-PCA). The proposed model projects observations onto a random orthogonal basis which is assigned a prior distribution defined on the Stiefel manifold. The prior on factor scores involves an Indian buffet process to model the uncertainty related to the number of components. The parameters of interest as well as the nuisance parameters are finally inferred within a fully Bayesian framework via Monte Carlo sampling. A study of the (in-)consistence of the marginal maximum a posteriori estimator of the latent dimension is carried out. A new estimator of the subspace dimension is proposed. Moreover, for sake of statistical significance, a Kolmogorov-Smirnov test based on the posterior distribution of the principal components is used to refine this estimate. The behaviour of the algorithm is first studied on various synthetic examples. Finally, the proposed BNP dimension reduction approach is shown to be easily yet efficiently coupled with clustering or latent factor models within a unique framework. |
| Tasks | Dimensionality Reduction |
| Published | 2017-09-17 |
| URL | http://arxiv.org/abs/1709.05667v1 |
| http://arxiv.org/pdf/1709.05667v1.pdf | |
| PWC | https://paperswithcode.com/paper/bayesian-nonparametric-principal-component |
| Repo | |
| Framework | |
Plausible Deniability for Privacy-Preserving Data Synthesis
| Title | Plausible Deniability for Privacy-Preserving Data Synthesis |
| Authors | Vincent Bindschaedler, Reza Shokri, Carl A. Gunter |
| Abstract | Releasing full data records is one of the most challenging problems in data privacy. On the one hand, many of the popular techniques such as data de-identification are problematic because of their dependence on the background knowledge of adversaries. On the other hand, rigorous methods such as the exponential mechanism for differential privacy are often computationally impractical to use for releasing high dimensional data or cannot preserve high utility of original data due to their extensive data perturbation. This paper presents a criterion called plausible deniability that provides a formal privacy guarantee, notably for releasing sensitive datasets: an output record can be released only if a certain amount of input records are indistinguishable, up to a privacy parameter. This notion does not depend on the background knowledge of an adversary. Also, it can efficiently be checked by privacy tests. We present mechanisms to generate synthetic datasets with similar statistical properties to the input data and the same format. We study this technique both theoretically and experimentally. A key theoretical result shows that, with proper randomization, the plausible deniability mechanism generates differentially private synthetic data. We demonstrate the efficiency of this generative technique on a large dataset; it is shown to preserve the utility of original data with respect to various statistical analysis and machine learning measures. |
| Tasks | |
| Published | 2017-08-26 |
| URL | http://arxiv.org/abs/1708.07975v1 |
| http://arxiv.org/pdf/1708.07975v1.pdf | |
| PWC | https://paperswithcode.com/paper/plausible-deniability-for-privacy-preserving |
| Repo | |
| Framework | |
