July 28, 2019

3163 words 15 mins read

Paper Group ANR 224

Human Pose Forecasting via Deep Markov Models. A Bootstrap Method for Error Estimation in Randomized Matrix Multiplication. Inverse Classification for Comparison-based Interpretability in Machine Learning. Learning Deep Representations for Word Spotting Under Weak Supervision. On the use of convolutional neural networks for robust classification of …

Human Pose Forecasting via Deep Markov Models


Title	Human Pose Forecasting via Deep Markov Models
Authors	Sam Toyer, Anoop Cherian, Tengda Han, Stephen Gould
Abstract	Human pose forecasting is an important problem in computer vision with applications to human-robot interaction, visual surveillance, and autonomous driving. Usually, forecasting algorithms use 3D skeleton sequences and are trained to forecast for a few milliseconds into the future. Long-range forecasting is challenging due to the difficulty of estimating how long a person continues an activity. To this end, our contributions are threefold: (i) we propose a generative framework for poses using variational autoencoders based on Deep Markov Models (DMMs); (ii) we evaluate our pose forecasts using a pose-based action classifier, which we argue better reflects the subjective quality of pose forecasts than distance in coordinate space; (iii) last, for evaluation of the new model, we introduce a 480,000-frame video dataset called Ikea Furniture Assembly (Ikea FA), which depicts humans repeatedly assembling and disassembling furniture. We demonstrate promising results for our approach on both Ikea FA and the existing NTU RGB+D dataset.
Tasks	Autonomous Driving, Human Pose Forecasting
Published	2017-07-24
URL	http://arxiv.org/abs/1707.09240v2
PDF	http://arxiv.org/pdf/1707.09240v2.pdf
PWC	https://paperswithcode.com/paper/human-pose-forecasting-via-deep-markov-models
Repo
Framework

A Bootstrap Method for Error Estimation in Randomized Matrix Multiplication


Title	A Bootstrap Method for Error Estimation in Randomized Matrix Multiplication
Authors	Miles E. Lopes, Shusen Wang, Michael W. Mahoney
Abstract	In recent years, randomized methods for numerical linear algebra have received growing interest as a general approach to large-scale problems. Typically, the essential ingredient of these methods is some form of randomized dimension reduction, which accelerates computations, but also creates random approximation error. In this way, the dimension reduction step encodes a tradeoff between cost and accuracy. However, the exact numerical relationship between cost and accuracy is typically unknown, and consequently, it may be difficult for the user to precisely know (1) how accurate a given solution is, or (2) how much computation is needed to achieve a given level of accuracy. In the current paper, we study randomized matrix multiplication (sketching) as a prototype setting for addressing these general problems. As a solution, we develop a bootstrap method for \emph{directly estimating} the accuracy as a function of the reduced dimension (as opposed to deriving worst-case bounds on the accuracy in terms of the reduced dimension). From a computational standpoint, the proposed method does not substantially increase the cost of standard sketching methods, and this is made possible by an “extrapolation” technique. In addition, we provide both theoretical and empirical results to demonstrate the effectiveness of the proposed method.
Tasks	Dimensionality Reduction
Published	2017-08-06
URL	http://arxiv.org/abs/1708.01945v2
PDF	http://arxiv.org/pdf/1708.01945v2.pdf
PWC	https://paperswithcode.com/paper/a-bootstrap-method-for-error-estimation-in
Repo
Framework

Inverse Classification for Comparison-based Interpretability in Machine Learning


Title	Inverse Classification for Comparison-based Interpretability in Machine Learning
Authors	Thibault Laugel, Marie-Jeanne Lesot, Christophe Marsala, Xavier Renard, Marcin Detyniecki
Abstract	In the context of post-hoc interpretability, this paper addresses the task of explaining the prediction of a classifier, considering the case where no information is available, neither on the classifier itself, nor on the processed data (neither the training nor the test data). It proposes an instance-based approach whose principle consists in determining the minimal changes needed to alter a prediction: given a data point whose classification must be explained, the proposed method consists in identifying a close neighbour classified differently, where the closeness definition integrates a sparsity constraint. This principle is implemented using observation generation in the Growing Spheres algorithm. Experimental results on two datasets illustrate the relevance of the proposed approach that can be used to gain knowledge about the classifier.
Tasks
Published	2017-12-22
URL	http://arxiv.org/abs/1712.08443v1
PDF	http://arxiv.org/pdf/1712.08443v1.pdf
PWC	https://paperswithcode.com/paper/inverse-classification-for-comparison-based
Repo
Framework

Learning Deep Representations for Word Spotting Under Weak Supervision


Title	Learning Deep Representations for Word Spotting Under Weak Supervision
Authors	Neha Gurjar, Sebastian Sudholt, Gernot A. Fink
Abstract	Convolutional Neural Networks have made their mark in various fields of computer vision in recent years. They have achieved state-of-the-art performance in the field of document analysis as well. However, CNNs require a large amount of annotated training data and, hence, great manual effort. In our approach, we introduce a method to drastically reduce the manual annotation effort while retaining the high performance of a CNN for word spotting in handwritten documents. The model is learned with weak supervision using a combination of synthetically generated training data and a small subset of the training partition of the handwritten data set. We show that the network achieves results highly competitive to the state-of-the-art in word spotting with shorter training times and a fraction of the annotation effort.
Tasks	Word Spotting In Handwritten Documents
Published	2017-12-01
URL	http://arxiv.org/abs/1712.00250v3
PDF	http://arxiv.org/pdf/1712.00250v3.pdf
PWC	https://paperswithcode.com/paper/learning-deep-representations-for-word
Repo
Framework

On the use of convolutional neural networks for robust classification of multiple fingerprint captures


Title	On the use of convolutional neural networks for robust classification of multiple fingerprint captures
Authors	Daniel Peralta, Isaac Triguero, Salvador García, Yvan Saeys, Jose M. Benitez, Francisco Herrera
Abstract	Fingerprint classification is one of the most common approaches to accelerate the identification in large databases of fingerprints. Fingerprints are grouped into disjoint classes, so that an input fingerprint is compared only with those belonging to the predicted class, reducing the penetration rate of the search. The classification procedure usually starts by the extraction of features from the fingerprint image, frequently based on visual characteristics. In this work, we propose an approach to fingerprint classification using convolutional neural networks, which avoid the necessity of an explicit feature extraction process by incorporating the image processing within the training of the classifier. Furthermore, such an approach is able to predict a class even for low-quality fingerprints that are rejected by commonly used algorithms, such as FingerCode. The study gives special importance to the robustness of the classification for different impressions of the same fingerprint, aiming to minimize the penetration in the database. In our experiments, convolutional neural networks yielded better accuracy and penetration rate than state-of-the-art classifiers based on explicit feature extraction. The tested networks also improved on the runtime, as a result of the joint optimization of both feature extraction and classification.
Tasks
Published	2017-03-21
URL	http://arxiv.org/abs/1703.07270v3
PDF	http://arxiv.org/pdf/1703.07270v3.pdf
PWC	https://paperswithcode.com/paper/on-the-use-of-convolutional-neural-networks
Repo
Framework

Evaluation of bioinspired algorithms for the solution of the job scheduling problem


Title	Evaluation of bioinspired algorithms for the solution of the job scheduling problem
Authors	Edson Florez, Nelson Diaz, Wilfredo Gomez, Lola Bautista, Dario Delgado
Abstract	In this research we used bio-inspired metaheuristics, as artificial immune systems and ant colony algorithms that are based on a number of characteristics and behaviors of living things that are interesting in the computer science area. This paper presents an evaluation of bio-inspired solutions to combinatorial optimization problem, called the Job Shop Scheduling or planning work, in a simple way the objective is to find a configuration or job stream that has the least amount of time to be executed in machine settings. The performance of the algorithms was characterized and evaluated for reference instances of the job shop scheduling problem, comparing the quality of the solutions obtained with respect to the best known solution of the most effective methods. The solutions were evaluated in two aspects, first in relation of quality of solutions, taking as reference the makespan and secondly in relation of performance, taking the number evaluations performed by the algorithm to obtain the best solution.
Tasks	Combinatorial Optimization
Published	2017-11-21
URL	http://arxiv.org/abs/1711.07821v1
PDF	http://arxiv.org/pdf/1711.07821v1.pdf
PWC	https://paperswithcode.com/paper/evaluation-of-bioinspired-algorithms-for-the
Repo
Framework

Maximizing Non-monotone/Non-submodular Functions by Multi-objective Evolutionary Algorithms


Title	Maximizing Non-monotone/Non-submodular Functions by Multi-objective Evolutionary Algorithms
Authors	Chao Qian, Yang Yu, Ke Tang, Xin Yao, Zhi-Hua Zhou
Abstract	Evolutionary algorithms (EAs) are a kind of nature-inspired general-purpose optimization algorithm, and have shown empirically good performance in solving various real-word optimization problems. However, due to the highly randomized and complex behavior, the theoretical analysis of EAs is difficult and is an ongoing challenge, which has attracted a lot of research attentions. During the last two decades, promising results on the running time analysis (one essential theoretical aspect) of EAs have been obtained, while most of them focused on isolated combinatorial optimization problems, which do not reflect the general-purpose nature of EAs. To provide a general theoretical explanation of the behavior of EAs, it is desirable to study the performance of EAs on a general class of combinatorial optimization problems. To the best of our knowledge, this direction has been rarely touched and the only known result is the provably good approximation guarantees of EAs for the problem class of maximizing monotone submodular set functions with matroid constraints, which includes many NP-hard combinatorial optimization problems. The aim of this work is to contribute to this line of research. As many combinatorial optimization problems also involve non-monotone or non-submodular objective functions, we consider these two general problem classes, maximizing non-monotone submodular functions without constraints and maximizing monotone non-submodular functions with a size constraint. We prove that a simple multi-objective EA called GSEMO can generally achieve good approximation guarantees in polynomial expected running time.
Tasks	Combinatorial Optimization
Published	2017-11-20
URL	http://arxiv.org/abs/1711.07214v1
PDF	http://arxiv.org/pdf/1711.07214v1.pdf
PWC	https://paperswithcode.com/paper/maximizing-non-monotonenon-submodular
Repo
Framework

Learning Sparse Representations in Reinforcement Learning with Sparse Coding


Title	Learning Sparse Representations in Reinforcement Learning with Sparse Coding
Authors	Lei Le, Raksha Kumaraswamy, Martha White
Abstract	A variety of representation learning approaches have been investigated for reinforcement learning; much less attention, however, has been given to investigating the utility of sparse coding. Outside of reinforcement learning, sparse coding representations have been widely used, with non-convex objectives that result in discriminative representations. In this work, we develop a supervised sparse coding objective for policy evaluation. Despite the non-convexity of this objective, we prove that all local minima are global minima, making the approach amenable to simple optimization strategies. We empirically show that it is key to use a supervised objective, rather than the more straightforward unsupervised sparse coding approach. We compare the learned representations to a canonical fixed sparse representation, called tile-coding, demonstrating that the sparse coding representation outperforms a wide variety of tilecoding representations.
Tasks	Representation Learning
Published	2017-07-26
URL	http://arxiv.org/abs/1707.08316v1
PDF	http://arxiv.org/pdf/1707.08316v1.pdf
PWC	https://paperswithcode.com/paper/learning-sparse-representations-in
Repo
Framework

Sensor Selection and Random Field Reconstruction for Robust and Cost-effective Heterogeneous Weather Sensor Networks for the Developing World


Title	Sensor Selection and Random Field Reconstruction for Robust and Cost-effective Heterogeneous Weather Sensor Networks for the Developing World
Authors	Pengfei Zhang, Ido Nevat, Gareth W. Peters, Wolfgang Fruehwirt, Yongchao Huang, Ivonne Anders, Michael Osborne
Abstract	We address the two fundamental problems of spatial field reconstruction and sensor selection in heterogeneous sensor networks: (i) how to efficiently perform spatial field reconstruction based on measurements obtained simultaneously from networks with both high and low quality sensors; and (ii) how to perform query based sensor set selection with predictive MSE performance guarantee. For the first problem, we developed a low complexity algorithm based on the spatial best linear unbiased estimator (S-BLUE). Next, building on the S-BLUE, we address the second problem, and develop an efficient algorithm for query based sensor set selection with performance guarantee. Our algorithm is based on the Cross Entropy method which solves the combinatorial optimization problem in an efficient manner.
Tasks	Combinatorial Optimization
Published	2017-11-12
URL	http://arxiv.org/abs/1711.04308v3
PDF	http://arxiv.org/pdf/1711.04308v3.pdf
PWC	https://paperswithcode.com/paper/sensor-selection-and-random-field
Repo
Framework

Deep Learning is Robust to Massive Label Noise


Title	Deep Learning is Robust to Massive Label Noise
Authors	David Rolnick, Andreas Veit, Serge Belongie, Nir Shavit
Abstract	Deep neural networks trained on large supervised datasets have led to impressive results in image classification and other tasks. However, well-annotated datasets can be time-consuming and expensive to collect, lending increased interest to larger but noisy datasets that are more easily obtained. In this paper, we show that deep neural networks are capable of generalizing from training data for which true labels are massively outnumbered by incorrect labels. We demonstrate remarkably high test performance after training on corrupted data from MNIST, CIFAR, and ImageNet. For example, on MNIST we obtain test accuracy above 90 percent even after each clean training example has been diluted with 100 randomly-labeled examples. Such behavior holds across multiple patterns of label noise, even when erroneous labels are biased towards confusing classes. We show that training in this regime requires a significant but manageable increase in dataset size that is related to the factor by which correct labels have been diluted. Finally, we provide an analysis of our results that shows how increasing noise decreases the effective batch size.
Tasks	Image Classification
Published	2017-05-30
URL	http://arxiv.org/abs/1705.10694v3
PDF	http://arxiv.org/pdf/1705.10694v3.pdf
PWC	https://paperswithcode.com/paper/deep-learning-is-robust-to-massive-label
Repo
Framework

Learning in the Machine: the Symmetries of the Deep Learning Channel


Title	Learning in the Machine: the Symmetries of the Deep Learning Channel
Authors	Pierre Baldi, Peter Sadowski, Zhiqin Lu
Abstract	In a physical neural system, learning rules must be local both in space and time. In order for learning to occur, non-local information must be communicated to the deep synapses through a communication channel, the deep learning channel. We identify several possible architectures for this learning channel (Bidirectional, Conjoined, Twin, Distinct) and six symmetry challenges: 1) symmetry of architectures; 2) symmetry of weights; 3) symmetry of neurons; 4) symmetry of derivatives; 5) symmetry of processing; and 6) symmetry of learning rules. Random backpropagation (RBP) addresses the second and third symmetry, and some of its variations, such as skipped RBP (SRBP) address the first and the fourth symmetry. Here we address the last two desirable symmetries showing through simulations that they can be achieved and that the learning channel is particularly robust to symmetry variations. Specifically, random backpropagation and its variations can be performed with the same non-linear neurons used in the main input-output forward channel, and the connections in the learning channel can be adapted using the same algorithm used in the forward channel, removing the need for any specialized hardware in the learning channel. Finally, we provide mathematical results in simple cases showing that the learning equations in the forward and backward channels converge to fixed points, for almost any initial conditions. In symmetric architectures, if the weights in both channels are small at initialization, adaptation in both channels leads to weights that are essentially symmetric during and after learning. Biological connections are discussed.
Tasks
Published	2017-12-22
URL	http://arxiv.org/abs/1712.08608v1
PDF	http://arxiv.org/pdf/1712.08608v1.pdf
PWC	https://paperswithcode.com/paper/learning-in-the-machine-the-symmetries-of-the
Repo
Framework

Quantifying Aspect Bias in Ordinal Ratings using a Bayesian Approach


Title	Quantifying Aspect Bias in Ordinal Ratings using a Bayesian Approach
Authors	Lahari Poddar, Wynne Hsu, Mong Li Lee
Abstract	User opinions expressed in the form of ratings can influence an individual’s view of an item. However, the true quality of an item is often obfuscated by user biases, and it is not obvious from the observed ratings the importance different users place on different aspects of an item. We propose a probabilistic modeling of the observed aspect ratings to infer (i) each user’s aspect bias and (ii) latent intrinsic quality of an item. We model multi-aspect ratings as ordered discrete data and encode the dependency between different aspects by using a latent Gaussian structure. We handle the Gaussian-Categorical non-conjugacy using a stick-breaking formulation coupled with P'{o}lya-Gamma auxiliary variable augmentation for a simple, fully Bayesian inference. On two real world datasets, we demonstrate the predictive ability of our model and its effectiveness in learning explainable user biases to provide insights towards a more reliable product quality estimation.
Tasks	Bayesian Inference
Published	2017-05-15
URL	http://arxiv.org/abs/1705.05098v2
PDF	http://arxiv.org/pdf/1705.05098v2.pdf
PWC	https://paperswithcode.com/paper/quantifying-aspect-bias-in-ordinal-ratings
Repo
Framework

Local Feature Descriptor Learning with Adaptive Siamese Network


Title	Local Feature Descriptor Learning with Adaptive Siamese Network
Authors	Chong Huang, Qiong Liu, Yan-Ying Chen, Kwang-Ting, Cheng
Abstract	Although the recent progress in the deep neural network has led to the development of learnable local feature descriptors, there is no explicit answer for estimation of the necessary size of a neural network. Specifically, the local feature is represented in a low dimensional space, so the neural network should have more compact structure. The small networks required for local feature descriptor learning may be sensitive to initial conditions and learning parameters and more likely to become trapped in local minima. In order to address the above problem, we introduce an adaptive pruning Siamese Architecture based on neuron activation to learn local feature descriptors, making the network more computationally efficient with an improved recognition rate over more complex networks. Our experiments demonstrate that our learned local feature descriptors outperform the state-of-art methods in patch matching.
Tasks
Published	2017-06-16
URL	http://arxiv.org/abs/1706.05358v1
PDF	http://arxiv.org/pdf/1706.05358v1.pdf
PWC	https://paperswithcode.com/paper/local-feature-descriptor-learning-with
Repo
Framework

Knowledge Adaptation: Teaching to Adapt


Title	Knowledge Adaptation: Teaching to Adapt
Authors	Sebastian Ruder, Parsa Ghaffari, John G. Breslin
Abstract	Domain adaptation is crucial in many real-world applications where the distribution of the training data differs from the distribution of the test data. Previous Deep Learning-based approaches to domain adaptation need to be trained jointly on source and target domain data and are therefore unappealing in scenarios where models need to be adapted to a large number of domains or where a domain is evolving, e.g. spam detection where attackers continuously change their tactics. To fill this gap, we propose Knowledge Adaptation, an extension of Knowledge Distillation (Bucilua et al., 2006; Hinton et al., 2015) to the domain adaptation scenario. We show how a student model achieves state-of-the-art results on unsupervised domain adaptation from multiple sources on a standard sentiment analysis benchmark by taking into account the domain-specific expertise of multiple teachers and the similarities between their domains. When learning from a single teacher, using domain similarity to gauge trustworthiness is inadequate. To this end, we propose a simple metric that correlates well with the teacher’s accuracy in the target domain. We demonstrate that incorporating high-confidence examples selected by this metric enables the student model to achieve state-of-the-art performance in the single-source scenario.
Tasks	Domain Adaptation, Sentiment Analysis, Unsupervised Domain Adaptation
Published	2017-02-07
URL	http://arxiv.org/abs/1702.02052v1
PDF	http://arxiv.org/pdf/1702.02052v1.pdf
PWC	https://paperswithcode.com/paper/knowledge-adaptation-teaching-to-adapt
Repo
Framework

Performance Bounds for Graphical Record Linkage


Title	Performance Bounds for Graphical Record Linkage
Authors	Rebecca C. Steorts, Matt Barnes, Willie Neiswanger
Abstract	Record linkage involves merging records in large, noisy databases to remove duplicate entities. It has become an important area because of its widespread occurrence in bibliometrics, public health, official statistics production, political science, and beyond. Traditional linkage methods directly linking records to one another are computationally infeasible as the number of records grows. As a result, it is increasingly common for researchers to treat record linkage as a clustering task, in which each latent entity is associated with one or more noisy database records. We critically assess performance bounds using the Kullback-Leibler (KL) divergence under a Bayesian record linkage framework, making connections to Kolchin partition models. We provide an upper bound using the KL divergence and a lower bound on the minimum probability of misclassifying a latent entity. We give insights for when our bounds hold using simulated data and provide practical user guidance.
Tasks
Published	2017-03-08
URL	http://arxiv.org/abs/1703.02679v1
PDF	http://arxiv.org/pdf/1703.02679v1.pdf
PWC	https://paperswithcode.com/paper/performance-bounds-for-graphical-record
Repo
Framework