July 27, 2019

2562 words 13 mins read

Paper Group ANR 501

Randomness in Deconvolutional Networks for Visual Representation. Dense Multi-view 3D-reconstruction Without Dense Correspondences. Towards Understanding the Invertibility of Convolutional Neural Networks. Deep Learning Autoencoder Approach for Handwritten Arabic Digits Recognition. Assessing State-of-the-Art Sentiment Models on State-of-the-Art Se …

Randomness in Deconvolutional Networks for Visual Representation


Title	Randomness in Deconvolutional Networks for Visual Representation
Authors	Kun He, Jingbo Wang, Haochuan Li, Yao Shu, Mengxiao Zhang, Man Zhu, Liwei Wang, John E. Hopcroft
Abstract	Toward a deeper understanding on the inner work of deep neural networks, we investigate CNN (convolutional neural network) using DCN (deconvolutional network) and randomization technique, and gain new insights for the intrinsic property of this network architecture. For the random representations of an untrained CNN, we train the corresponding DCN to reconstruct the input images. Compared with the image inversion on pre-trained CNN, our training converges faster and the yielding network exhibits higher quality for image reconstruction. It indicates there is rich information encoded in the random features; the pre-trained CNN may discard information irrelevant for classification and encode relevant features in a way favorable for classification but harder for reconstruction. We further explore the property of the overall random CNN-DCN architecture. Surprisingly, images can be inverted with satisfactory quality. Extensive empirical evidence as well as theoretical analysis are provided.
Tasks	Image Reconstruction
Published	2017-04-02
URL	http://arxiv.org/abs/1704.00330v3
PDF	http://arxiv.org/pdf/1704.00330v3.pdf
PWC	https://paperswithcode.com/paper/randomness-in-deconvolutional-networks-for
Repo
Framework

Dense Multi-view 3D-reconstruction Without Dense Correspondences


Title	Dense Multi-view 3D-reconstruction Without Dense Correspondences
Authors	Yvain Quéau, Jean Mélou, Jean-Denis Durou, Daniel Cremers
Abstract	We introduce a variational method for multi-view shape-from-shading under natural illumination. The key idea is to couple PDE-based solutions for single-image based shape-from-shading problems across multiple images and multiple color channels by means of a variational formulation. Rather than alternatingly solving the individual SFS problems and optimizing the consistency across images and channels which is known to lead to suboptimal results, we propose an efficient solution of the coupled problem by means of an ADMM algorithm. In numerous experiments on both simulated and real imagery, we demonstrate that the proposed fusion of multiple-view reconstruction and shape-from-shading provides highly accurate dense reconstructions without the need to compute dense correspondences. With the proposed variational integration across multiple views shape-from-shading techniques become applicable to challenging real-world reconstruction problems, giving rise to highly detailed geometry even in areas of smooth brightness variation and lacking texture.
Tasks	3D Reconstruction
Published	2017-04-02
URL	http://arxiv.org/abs/1704.00337v1
PDF	http://arxiv.org/pdf/1704.00337v1.pdf
PWC	https://paperswithcode.com/paper/dense-multi-view-3d-reconstruction-without
Repo
Framework

Towards Understanding the Invertibility of Convolutional Neural Networks


Title	Towards Understanding the Invertibility of Convolutional Neural Networks
Authors	Anna C. Gilbert, Yi Zhang, Kibok Lee, Yuting Zhang, Honglak Lee
Abstract	Several recent works have empirically observed that Convolutional Neural Nets (CNNs) are (approximately) invertible. To understand this approximate invertibility phenomenon and how to leverage it more effectively, we focus on a theoretical explanation and develop a mathematical model of sparse signal recovery that is consistent with CNNs with random weights. We give an exact connection to a particular model of model-based compressive sensing (and its recovery algorithms) and random-weight CNNs. We show empirically that several learned networks are consistent with our mathematical analysis and then demonstrate that with such a simple theoretical framework, we can obtain reasonable re- construction results on real images. We also discuss gaps between our model assumptions and the CNN trained for classification in practical scenarios.
Tasks	Compressive Sensing
Published	2017-05-24
URL	http://arxiv.org/abs/1705.08664v1
PDF	http://arxiv.org/pdf/1705.08664v1.pdf
PWC	https://paperswithcode.com/paper/towards-understanding-the-invertibility-of
Repo
Framework

Deep Learning Autoencoder Approach for Handwritten Arabic Digits Recognition


Title	Deep Learning Autoencoder Approach for Handwritten Arabic Digits Recognition
Authors	Mohamed Loey, Ahmed El-Sawy, Hazem EL-Bakry
Abstract	This paper presents a new unsupervised learning approach with stacked autoencoder (SAE) for Arabic handwritten digits categorization. Recently, Arabic handwritten digits recognition has been an important area due to its applications in several fields. This work is focusing on the recognition part of handwritten Arabic digits recognition that face several challenges, including the unlimited variation in human handwriting and the large public databases. Arabic digits contains ten numbers that were descended from the Indian digits system. Stacked autoencoder (SAE) tested and trained the MADBase database (Arabic handwritten digits images) that contain 10000 testing images and 60000 training images. We show that the use of SAE leads to significant improvements across different machine-learning classification algorithms. SAE is giving an average accuracy of 98.5%.
Tasks
Published	2017-06-21
URL	http://arxiv.org/abs/1706.06720v1
PDF	http://arxiv.org/pdf/1706.06720v1.pdf
PWC	https://paperswithcode.com/paper/deep-learning-autoencoder-approach-for
Repo
Framework

Assessing State-of-the-Art Sentiment Models on State-of-the-Art Sentiment Datasets


Title	Assessing State-of-the-Art Sentiment Models on State-of-the-Art Sentiment Datasets
Authors	Jeremy Barnes, Roman Klinger, Sabine Schulte im Walde
Abstract	There has been a good amount of progress in sentiment analysis over the past 10 years, including the proposal of new methods and the creation of benchmark datasets. In some papers, however, there is a tendency to compare models only on one or two datasets, either because of time restraints or because the model is tailored to a specific task. Accordingly, it is hard to understand how well a certain model generalizes across different tasks and datasets. In this paper, we contribute to this situation by comparing several models on six different benchmarks, which belong to different domains and additionally have different levels of granularity (binary, 3-class, 4-class and 5-class). We show that Bi-LSTMs perform well across datasets and that both LSTMs and Bi-LSTMs are particularly good at fine-grained sentiment tasks (i. e., with more than two classes). Incorporating sentiment information into word embeddings during training gives good results for datasets that are lexically similar to the training data. With our experiments, we contribute to a better understanding of the performance of different model architectures on different data sets. Consequently, we detect novel state-of-the-art results on the SenTube datasets.
Tasks	Sentiment Analysis, Word Embeddings
Published	2017-09-13
URL	http://arxiv.org/abs/1709.04219v1
PDF	http://arxiv.org/pdf/1709.04219v1.pdf
PWC	https://paperswithcode.com/paper/assessing-state-of-the-art-sentiment-models
Repo
Framework

Practical Gauss-Newton Optimisation for Deep Learning


Title	Practical Gauss-Newton Optimisation for Deep Learning
Authors	Aleksandar Botev, Hippolyt Ritter, David Barber
Abstract	We present an efficient block-diagonal ap- proximation to the Gauss-Newton matrix for feedforward neural networks. Our result- ing algorithm is competitive against state- of-the-art first order optimisation methods, with sometimes significant improvement in optimisation performance. Unlike first-order methods, for which hyperparameter tuning of the optimisation parameters is often a labo- rious process, our approach can provide good performance even when used with default set- tings. A side result of our work is that for piecewise linear transfer functions, the net- work objective function can have no differ- entiable local maxima, which may partially explain why such transfer functions facilitate effective optimisation.
Tasks
Published	2017-06-12
URL	http://arxiv.org/abs/1706.03662v2
PDF	http://arxiv.org/pdf/1706.03662v2.pdf
PWC	https://paperswithcode.com/paper/practical-gauss-newton-optimisation-for-deep
Repo
Framework

Information-theoretic Model Identification and Policy Search using Physics Engines with Application to Robotic Manipulation


Title	Information-theoretic Model Identification and Policy Search using Physics Engines with Application to Robotic Manipulation
Authors	Shaojun Zhu, Andrew Kimmel, Abdeslam Boularias
Abstract	We consider the problem of a robot learning the mechanical properties of objects through physical interaction with the object, and introduce a practical, data-efficient approach for identifying the motion models of these objects. The proposed method utilizes a physics engine, where the robot seeks to identify the inertial and friction parameters of the object by simulating its motion under different values of the parameters and identifying those that result in a simulation which matches the observed real motions. The problem is solved in a Bayesian optimization framework. The same framework is used for both identifying the model of an object online and searching for a policy that would minimize a given cost function according to the identified model. Experimental results both in simulation and using a real robot indicate that the proposed method outperforms state-of-the-art model-free reinforcement learning approaches.
Tasks
Published	2017-03-22
URL	http://arxiv.org/abs/1703.07822v1
PDF	http://arxiv.org/pdf/1703.07822v1.pdf
PWC	https://paperswithcode.com/paper/information-theoretic-model-identification
Repo
Framework

Feedforward and Recurrent Neural Networks Backward Propagation and Hessian in Matrix Form


Title	Feedforward and Recurrent Neural Networks Backward Propagation and Hessian in Matrix Form
Authors	Maxim Naumov
Abstract	In this paper we focus on the linear algebra theory behind feedforward (FNN) and recurrent (RNN) neural networks. We review backward propagation, including backward propagation through time (BPTT). Also, we obtain a new exact expression for Hessian, which represents second order effects. We show that for $t$ time steps the weight gradient can be expressed as a rank-$t$ matrix, while the weight Hessian is as a sum of $t^{2}$ Kronecker products of rank-$1$ and $W^{T}AW$ matrices, for some matrix $A$ and weight matrix $W$. Also, we show that for a mini-batch of size $r$, the weight update can be expressed as a rank-$rt$ matrix. Finally, we briefly comment on the eigenvalues of the Hessian matrix.
Tasks
Published	2017-09-16
URL	http://arxiv.org/abs/1709.06080v1
PDF	http://arxiv.org/pdf/1709.06080v1.pdf
PWC	https://paperswithcode.com/paper/feedforward-and-recurrent-neural-networks
Repo
Framework

Conformal predictive distributions with kernels


Title	Conformal predictive distributions with kernels
Authors	Vladimir Vovk, Ilia Nouretdinov, Valery Manokhin, Alex Gammerman
Abstract	This paper reviews the checkered history of predictive distributions in statistics and discusses two developments, one from recent literature and the other new. The first development is bringing predictive distributions into machine learning, whose early development was so deeply influenced by two remarkable groups at the Institute of Automation and Remote Control. The second development is combining predictive distributions with kernel methods, which were originated by one of those groups, including Emmanuel Braverman.
Tasks
Published	2017-10-24
URL	http://arxiv.org/abs/1710.08894v1
PDF	http://arxiv.org/pdf/1710.08894v1.pdf
PWC	https://paperswithcode.com/paper/conformal-predictive-distributions-with
Repo
Framework

Field of Groves: An Energy-Efficient Random Forest


Title	Field of Groves: An Energy-Efficient Random Forest
Authors	Zafar Takhirov, Joseph Wang, Marcia S. Louis, Venkatesh Saligrama, Ajay Joshi
Abstract	Machine Learning (ML) algorithms, like Convolutional Neural Networks (CNN), Support Vector Machines (SVM), etc. have become widespread and can achieve high statistical performance. However their accuracy decreases significantly in energy-constrained mobile and embedded systems space, where all computations need to be completed under a tight energy budget. In this work, we present a field of groves (FoG) implementation of random forests (RF) that achieves an accuracy comparable to CNNs and SVMs under tight energy budgets. Evaluation of the FoG shows that at comparable accuracy it consumes ~1.48x, ~24x, ~2.5x, and ~34.7x lower energy per classification compared to conventional RF, SVM_RBF , MLP, and CNN, respectively. FoG is ~6.5x less energy efficient than SVM_LR, but achieves 18% higher accuracy on average across all considered datasets.
Tasks
Published	2017-04-10
URL	http://arxiv.org/abs/1704.02978v1
PDF	http://arxiv.org/pdf/1704.02978v1.pdf
PWC	https://paperswithcode.com/paper/field-of-groves-an-energy-efficient-random
Repo
Framework

Simultaneous merging multiple grid maps using the robust motion averaging


Title	Simultaneous merging multiple grid maps using the robust motion averaging
Authors	Zutao Jiang, Jihua Zhu, Yaochen Li, Zhongyu Li, Huimin Lu
Abstract	Mapping in the GPS-denied environment is an important and challenging task in the field of robotics. In the large environment, mapping can be significantly accelerated by multiple robots exploring different parts of the environment. Accordingly, a key problem is how to integrate these local maps built by different robots into a single global map. In this paper, we propose an approach for simultaneous merging of multiple grid maps by the robust motion averaging. The main idea of this approach is to recover all global motions for map merging from a set of relative motions. Therefore, it firstly adopts the pair-wise map merging method to estimate relative motions for grid map pairs. To obtain as many reliable relative motions as possible, a graph-based sampling scheme is utilized to efficiently remove unreliable relative motions obtained from the pair-wise map merging. Subsequently, the accurate global motions can be recovered from the set of reliable relative motions by the motion averaging. Experimental results carried on real robot data sets demonstrate that proposed approach can achieve simultaneous merging of multiple grid maps with good performances.
Tasks
Published	2017-06-14
URL	http://arxiv.org/abs/1706.04463v1
PDF	http://arxiv.org/pdf/1706.04463v1.pdf
PWC	https://paperswithcode.com/paper/simultaneous-merging-multiple-grid-maps-using
Repo
Framework

Dependent relevance determination for smooth and structured sparse regression


Title	Dependent relevance determination for smooth and structured sparse regression
Authors	Anqi Wu, Oluwasanmi Koyejo, Jonathan W. Pillow
Abstract	In many problem settings, parameter vectors are not merely sparse but dependent in such a way that non-zero coefficients tend to cluster together. We refer to this form of dependency as “region sparsity.” Classical sparse regression methods, such as the lasso and automatic relevance determination (ARD), which model parameters as independent a priori, and therefore do not exploit such dependencies. Here we introduce a hierarchical model for smooth, region-sparse weight vectors and tensors in a linear regression setting. Our approach represents a hierarchical extension of the relevance determination framework, where we add a transformed Gaussian process to model the dependencies between the prior variances of regression weights. We combine this with a structured model of the prior variances of Fourier coefficients, which eliminates unnecessary high frequencies. The resulting prior encourages weights to be region-sparse in two different bases simultaneously. We develop Laplace approximation and Monte Carlo Markov Chain (MCMC) sampling to provide efficient inference for the posterior. Furthermore, a two-stage convex relaxation of the Laplace approximation approach is also provided to relax the inevitable non-convexity during the optimization. We finally show substantial improvements over comparable methods for both simulated and real datasets from brain imaging.
Tasks
Published	2017-11-28
URL	http://arxiv.org/abs/1711.10058v3
PDF	http://arxiv.org/pdf/1711.10058v3.pdf
PWC	https://paperswithcode.com/paper/dependent-relevance-determination-for-smooth
Repo
Framework

Attentive Explanations: Justifying Decisions and Pointing to the Evidence (Extended Abstract)


Title	Attentive Explanations: Justifying Decisions and Pointing to the Evidence (Extended Abstract)
Authors	Dong Huk Park, Lisa Anne Hendricks, Zeynep Akata, Anna Rohrbach, Bernt Schiele, Trevor Darrell, Marcus Rohrbach
Abstract	Deep models are the defacto standard in visual decision problems due to their impressive performance on a wide array of visual tasks. On the other hand, their opaqueness has led to a surge of interest in explainable systems. In this work, we emphasize the importance of model explanation in various forms such as visual pointing and textual justification. The lack of data with justification annotations is one of the bottlenecks of generating multimodal explanations. Thus, we propose two large-scale datasets with annotations that visually and textually justify a classification decision for various activities, i.e. ACT-X, and for question answering, i.e. VQA-X. We also introduce a multimodal methodology for generating visual and textual explanations simultaneously. We quantitatively show that training with the textual explanations not only yields better textual justification models, but also models that better localize the evidence that support their decision.
Tasks	Question Answering, Visual Question Answering
Published	2017-11-17
URL	http://arxiv.org/abs/1711.07373v1
PDF	http://arxiv.org/pdf/1711.07373v1.pdf
PWC	https://paperswithcode.com/paper/attentive-explanations-justifying-decisions-1
Repo
Framework

Understanding Concept Drift


Title	Understanding Concept Drift
Authors	Geoffrey I. Webb, Loong Kuan Lee, François Petitjean, Bart Goethals
Abstract	Concept drift is a major issue that greatly affects the accuracy and reliability of many real-world applications of machine learning. We argue that to tackle concept drift it is important to develop the capacity to describe and analyze it. We propose tools for this purpose, arguing for the importance of quantitative descriptions of drift in marginal distributions. We present quantitative drift analysis techniques along with methods for communicating their results. We demonstrate their effectiveness by application to three real-world learning tasks.
Tasks
Published	2017-04-02
URL	http://arxiv.org/abs/1704.00362v1
PDF	http://arxiv.org/pdf/1704.00362v1.pdf
PWC	https://paperswithcode.com/paper/understanding-concept-drift
Repo
Framework

Learning Based on CC1 and CC4 Neural Networks


Title	Learning Based on CC1 and CC4 Neural Networks
Authors	Subhash Kak
Abstract	We propose that a general learning system should have three kinds of agents corresponding to sensory, short-term, and long-term memory that implicitly will facilitate context-free and context-sensitive aspects of learning. These three agents perform mututally complementary functions that capture aspects of the human cognition system. We investigate the use of CC1 and CC4 networks for use as models of short-term and sensory memory.
Tasks
Published	2017-12-22
URL	http://arxiv.org/abs/1712.09331v1
PDF	http://arxiv.org/pdf/1712.09331v1.pdf
PWC	https://paperswithcode.com/paper/learning-based-on-cc1-and-cc4-neural-networks
Repo
Framework