July 28, 2019

3016 words 15 mins read

Paper Group ANR 181

Adaptive Feeding: Achieving Fast and Accurate Detections by Adaptively Combining Object Detectors. Bridging the Gap between Constant Step Size Stochastic Gradient Descent and Markov Chains. Reasoning in Systems with Elements that Randomly Switch Characteristics. Unsupervised Clustering and Active Learning of Hyperspectral Images with Nonlinear Diff …

Adaptive Feeding: Achieving Fast and Accurate Detections by Adaptively Combining Object Detectors


Title	Adaptive Feeding: Achieving Fast and Accurate Detections by Adaptively Combining Object Detectors
Authors	Hong-Yu Zhou, Bin-Bin Gao, Jianxin Wu
Abstract	Object detection aims at high speed and accuracy simultaneously. However, fast models are usually less accurate, while accurate models cannot satisfy our need for speed. A fast model can be 10 times faster but 50% less accurate than an accurate model. In this paper, we propose Adaptive Feeding (AF) to combine a fast (but less accurate) detector and an accurate (but slow) detector, by adaptively determining whether an image is easy or hard and choosing an appropriate detector for it. In practice, we build a cascade of detectors, including the AF classifier which make the easy vs. hard decision and the two detectors. The AF classifier can be tuned to obtain different tradeoff between speed and accuracy, which has negligible training time and requires no additional training data. Experimental results on the PASCAL VOC, MS COCO and Caltech Pedestrian datasets confirm that AF has the ability to achieve comparable speed as the fast detector and comparable accuracy as the accurate one at the same time. As an example, by combining the fast SSD300 with the accurate SSD500 detector, AF leads to 50% speedup over SSD500 with the same precision on the VOC2007 test set.
Tasks	Object Detection
Published	2017-07-20
URL	http://arxiv.org/abs/1707.06399v1
PDF	http://arxiv.org/pdf/1707.06399v1.pdf
PWC	https://paperswithcode.com/paper/adaptive-feeding-achieving-fast-and-accurate
Repo
Framework

Bridging the Gap between Constant Step Size Stochastic Gradient Descent and Markov Chains


Title	Bridging the Gap between Constant Step Size Stochastic Gradient Descent and Markov Chains
Authors	Aymeric Dieuleveut, Alain Durmus, Francis Bach
Abstract	We consider the minimization of an objective function given access to unbiased estimates of its gradient through stochastic gradient descent (SGD) with constant step-size. While the detailed analysis was only performed for quadratic functions, we provide an explicit asymptotic expansion of the moments of the averaged SGD iterates that outlines the dependence on initial conditions, the effect of noise and the step-size, as well as the lack of convergence in the general (non-quadratic) case. For this analysis, we bring tools from Markov chain theory into the analysis of stochastic gradient. We then show that Richardson-Romberg extrapolation may be used to get closer to the global optimum and we show empirical improvements of the new extrapolation scheme.
Tasks
Published	2017-07-20
URL	http://arxiv.org/abs/1707.06386v2
PDF	http://arxiv.org/pdf/1707.06386v2.pdf
PWC	https://paperswithcode.com/paper/bridging-the-gap-between-constant-step-size
Repo
Framework

Reasoning in Systems with Elements that Randomly Switch Characteristics


Title	Reasoning in Systems with Elements that Randomly Switch Characteristics
Authors	Subhash Kak
Abstract	We examine the issue of stability of probability in reasoning about complex systems with uncertainty in structure. Normally, propositions are viewed as probability functions on an abstract random graph where it is implicitly assumed that the nodes of the graph have stable properties. But what if some of the nodes change their characteristics? This is a situation that cannot be covered by abstractions of either static or dynamic sets when these changes take place at regular intervals. We propose the use of sets with elements that change, and modular forms are proposed to account for one type of such change. An expression for the dependence of the mean on the probability of the switching elements has been determined. The system is also analyzed from the perspective of decision between different hypotheses. Such sets are likely to be of use in complex system queries and in analysis of surveys.
Tasks
Published	2017-12-13
URL	http://arxiv.org/abs/1712.04909v1
PDF	http://arxiv.org/pdf/1712.04909v1.pdf
PWC	https://paperswithcode.com/paper/reasoning-in-systems-with-elements-that
Repo
Framework

Unsupervised Clustering and Active Learning of Hyperspectral Images with Nonlinear Diffusion


Title	Unsupervised Clustering and Active Learning of Hyperspectral Images with Nonlinear Diffusion
Authors	James M. Murphy, Mauro Maggioni
Abstract	The problem of unsupervised learning and segmentation of hyperspectral images is a significant challenge in remote sensing. The high dimensionality of hyperspectral data, presence of substantial noise, and overlap of classes all contribute to the difficulty of automatically clustering and segmenting hyperspectral images. We propose an unsupervised learning technique called spectral-spatial diffusion learning (DLSS) that combines a geometric estimation of class modes with a diffusion-inspired labeling that incorporates both spectral and spatial information. The mode estimation incorporates the geometry of the hyperspectral data by using diffusion distance to promote learning a unique mode from each class. These class modes are then used to label all points by a joint spectral-spatial nonlinear diffusion process. A related variation of DLSS is also discussed, which enables active learning by requesting labels for a very small number of well-chosen pixels, dramatically boosting overall clustering results. Extensive experimental analysis demonstrates the efficacy of the proposed methods against benchmark and state-of-the-art hyperspectral analysis techniques on a variety of real datasets, their robustness to choices of parameters, and their low computational complexity.
Tasks	Active Learning
Published	2017-04-26
URL	http://arxiv.org/abs/1704.07961v5
PDF	http://arxiv.org/pdf/1704.07961v5.pdf
PWC	https://paperswithcode.com/paper/unsupervised-clustering-and-active-learning
Repo
Framework

Gradually Updated Neural Networks for Large-Scale Image Recognition


Title	Gradually Updated Neural Networks for Large-Scale Image Recognition
Authors	Siyuan Qiao, Zhishuai Zhang, Wei Shen, Bo Wang, Alan Yuille
Abstract	Depth is one of the keys that make neural networks succeed in the task of large-scale image recognition. The state-of-the-art network architectures usually increase the depths by cascading convolutional layers or building blocks. In this paper, we present an alternative method to increase the depth. Our method is by introducing computation orderings to the channels within convolutional layers or blocks, based on which we gradually compute the outputs in a channel-wise manner. The added orderings not only increase the depths and the learning capacities of the networks without any additional computation costs, but also eliminate the overlap singularities so that the networks are able to converge faster and perform better. Experiments show that the networks based on our method achieve the state-of-the-art performances on CIFAR and ImageNet datasets.
Tasks
Published	2017-11-25
URL	http://arxiv.org/abs/1711.09280v2
PDF	http://arxiv.org/pdf/1711.09280v2.pdf
PWC	https://paperswithcode.com/paper/gradually-updated-neural-networks-for-large
Repo
Framework

Towards the Improvement of Automated Scientific Document Categorization by Deep Learning


Title	Towards the Improvement of Automated Scientific Document Categorization by Deep Learning
Authors	Thomas Krause
Abstract	This master thesis describes an algorithm for automated categorization of scientific documents using deep learning techniques and compares the results to the results of existing classification algorithms. As an additional goal a reusable API is to be developed allowing the automation of classification tasks in existing software. A design will be proposed using a convolutional neural network as a classifier and integrating this into a REST based API. This is then used as the basis for an actual proof of concept implementation presented as well in this thesis. It will be shown that the deep learning classifier provides very good result in the context of multi-class document categorization and that it is feasible to integrate such classifiers into a larger ecosystem using REST based services.
Tasks
Published	2017-06-18
URL	http://arxiv.org/abs/1706.05719v1
PDF	http://arxiv.org/pdf/1706.05719v1.pdf
PWC	https://paperswithcode.com/paper/towards-the-improvement-of-automated
Repo
Framework

Distance and Similarity Measures Effect on the Performance of K-Nearest Neighbor Classifier – A Review


Title	Distance and Similarity Measures Effect on the Performance of K-Nearest Neighbor Classifier – A Review
Authors	V. B. Surya Prasath, Haneen Arafat Abu Alfeilat, Ahmad B. A. Hassanat, Omar Lasassmeh, Ahmad S. Tarawneh, Mahmoud Bashir Alhasanat, Hamzeh S. Eyal Salman
Abstract	The K-nearest neighbor (KNN) classifier is one of the simplest and most common classifiers, yet its performance competes with the most complex classifiers in the literature. The core of this classifier depends mainly on measuring the distance or similarity between the tested examples and the training examples. This raises a major question about which distance measures to be used for the KNN classifier among a large number of distance and similarity measures available? This review attempts to answer this question through evaluating the performance (measured by accuracy, precision and recall) of the KNN using a large number of distance measures, tested on a number of real-world datasets, with and without adding different levels of noise. The experimental results show that the performance of KNN classifier depends significantly on the distance used, and the results showed large gaps between the performances of different distances. We found that a recently proposed non-convex distance performed the best when applied on most datasets comparing to the other tested distances. In addition, the performance of the KNN with this top performing distance degraded only about $20%$ while the noise level reaches $90%$, this is true for most of the distances used as well. This means that the KNN classifier using any of the top $10$ distances tolerate noise to a certain degree. Moreover, the results show that some distances are less affected by the added noise comparing to other distances.
Tasks
Published	2017-08-14
URL	https://arxiv.org/abs/1708.04321v3
PDF	https://arxiv.org/pdf/1708.04321v3.pdf
PWC	https://paperswithcode.com/paper/distance-and-similarity-measures-effect-on
Repo
Framework

LS-VO: Learning Dense Optical Subspace for Robust Visual Odometry Estimation


Title	LS-VO: Learning Dense Optical Subspace for Robust Visual Odometry Estimation
Authors	Gabriele Costante, Thomas A. Ciarfuglia
Abstract	This work proposes a novel deep network architecture to solve the camera Ego-Motion estimation problem. A motion estimation network generally learns features similar to Optical Flow (OF) fields starting from sequences of images. This OF can be described by a lower dimensional latent space. Previous research has shown how to find linear approximations of this space. We propose to use an Auto-Encoder network to find a non-linear representation of the OF manifold. In addition, we propose to learn the latent space jointly with the estimation task, so that the learned OF features become a more robust description of the OF input. We call this novel architecture LS-VO. The experiments show that LS-VO achieves a considerable increase in performances in respect to baselines, while the number of parameters of the estimation network only slightly increases.
Tasks	Motion Estimation, Optical Flow Estimation, Visual Odometry
Published	2017-09-18
URL	http://arxiv.org/abs/1709.06019v2
PDF	http://arxiv.org/pdf/1709.06019v2.pdf
PWC	https://paperswithcode.com/paper/ls-vo-learning-dense-optical-subspace-for
Repo
Framework

Decontamination of Mutual Contamination Models


Title	Decontamination of Mutual Contamination Models
Authors	Julian Katz-Samuels, Gilles Blanchard, Clayton Scott
Abstract	Many machine learning problems can be characterized by mutual contamination models. In these problems, one observes several random samples from different convex combinations of a set of unknown base distributions and the goal is to infer these base distributions. This paper considers the general setting where the base distributions are defined on arbitrary probability spaces. We examine three popular machine learning problems that arise in this general setting: multiclass classification with label noise, demixing of mixed membership models, and classification with partial labels. In each case, we give sufficient conditions for identifiability and present algorithms for the infinite and finite sample settings, with associated performance guarantees.
Tasks
Published	2017-09-30
URL	http://arxiv.org/abs/1710.01167v2
PDF	http://arxiv.org/pdf/1710.01167v2.pdf
PWC	https://paperswithcode.com/paper/decontamination-of-mutual-contamination
Repo
Framework

Distribution of degrees of freedom over structure and motion of rigid bodies


Title	Distribution of degrees of freedom over structure and motion of rigid bodies
Authors	Mieczysław A. Kłopotek
Abstract	This paper is concerned with recovery of motion and structure parameters from multiframes under orthogonal projection when only points are traced. The main question is how many points and/or how many frames are necessary for the task. It is demonstrated that 3 frames and 3 points are the absolute minimum. Closed-form solution is presented. Furthermore, it is shown that the task may be linearized if either four points or four frames are available. It is demonstrated that no increase in the number of points may lead to recovery of structure and motion parameters from two frames only. It is shown that instead the increase in the number of points may support the task of tracing the points from frame to frame.
Tasks
Published	2017-05-11
URL	http://arxiv.org/abs/1705.03986v1
PDF	http://arxiv.org/pdf/1705.03986v1.pdf
PWC	https://paperswithcode.com/paper/distribution-of-degrees-of-freedom-over
Repo
Framework

Tensor-Based Backpropagation in Neural Networks with Non-Sequential Input


Title	Tensor-Based Backpropagation in Neural Networks with Non-Sequential Input
Authors	Hirsh R. Agarwal, Andrew Huang
Abstract	Neural networks have been able to achieve groundbreaking accuracy at tasks conventionally considered only doable by humans. Using stochastic gradient descent, optimization in many dimensions is made possible, albeit at a relatively high computational cost. By splitting training data into batches, networks can be distributed and trained vastly more efficiently and with minimal accuracy loss. We have explored the mathematics behind efficiently implementing tensor-based batch backpropagation algorithms. A common approach to batch training is iterating over batch items individually. Explicitly using tensor operations to backpropagate allows training to be performed non-linearly, increasing computational efficiency.
Tasks
Published	2017-07-13
URL	http://arxiv.org/abs/1707.04324v1
PDF	http://arxiv.org/pdf/1707.04324v1.pdf
PWC	https://paperswithcode.com/paper/tensor-based-backpropagation-in-neural
Repo
Framework

Translation of Patent Sentences with a Large Vocabulary of Technical Terms Using Neural Machine Translation


Title	Translation of Patent Sentences with a Large Vocabulary of Technical Terms Using Neural Machine Translation
Authors	Zi Long, Takehito Utsuro, Tomoharu Mitsuhashi, Mikio Yamamoto
Abstract	Neural machine translation (NMT), a new approach to machine translation, has achieved promising results comparable to those of traditional approaches such as statistical machine translation (SMT). Despite its recent success, NMT cannot handle a larger vocabulary because training complexity and decoding complexity proportionally increase with the number of target words. This problem becomes even more serious when translating patent documents, which contain many technical terms that are observed infrequently. In NMTs, words that are out of vocabulary are represented by a single unknown token. In this paper, we propose a method that enables NMT to translate patent sentences comprising a large vocabulary of technical terms. We train an NMT system on bilingual data wherein technical terms are replaced with technical term tokens; this allows it to translate most of the source sentences except technical terms. Further, we use it as a decoder to translate source sentences with technical term tokens and replace the tokens with technical term translations using SMT. We also use it to rerank the 1,000-best SMT translations on the basis of the average of the SMT score and that of the NMT rescoring of the translated sentences with technical term tokens. Our experiments on Japanese-Chinese patent sentences show that the proposed NMT system achieves a substantial improvement of up to 3.1 BLEU points and 2.3 RIBES points over traditional SMT systems and an improvement of approximately 0.6 BLEU points and 0.8 RIBES points over an equivalent NMT system without our proposed technique.
Tasks	Machine Translation
Published	2017-04-14
URL	http://arxiv.org/abs/1704.04521v1
PDF	http://arxiv.org/pdf/1704.04521v1.pdf
PWC	https://paperswithcode.com/paper/translation-of-patent-sentences-with-a-large
Repo
Framework

Momentum and Stochastic Momentum for Stochastic Gradient, Newton, Proximal Point and Subspace Descent Methods


Title	Momentum and Stochastic Momentum for Stochastic Gradient, Newton, Proximal Point and Subspace Descent Methods
Authors	Nicolas Loizou, Peter Richtárik
Abstract	In this paper we study several classes of stochastic optimization algorithms enriched with heavy ball momentum. Among the methods studied are: stochastic gradient descent, stochastic Newton, stochastic proximal point and stochastic dual subspace ascent. This is the first time momentum variants of several of these methods are studied. We choose to perform our analysis in a setting in which all of the above methods are equivalent. We prove global nonassymptotic linear convergence rates for all methods and various measures of success, including primal function values, primal iterates (in L2 sense), and dual function values. We also show that the primal iterates converge at an accelerated linear rate in the L1 sense. This is the first time a linear rate is shown for the stochastic heavy ball method (i.e., stochastic gradient descent method with momentum). Under somewhat weaker conditions, we establish a sublinear convergence rate for Cesaro averages of primal iterates. Moreover, we propose a novel concept, which we call stochastic momentum, aimed at decreasing the cost of performing the momentum step. We prove linear convergence of several stochastic methods with stochastic momentum, and show that in some sparse data regimes and for sufficiently small momentum parameters, these methods enjoy better overall complexity than methods with deterministic momentum. Finally, we perform extensive numerical testing on artificial and real datasets, including data coming from average consensus problems.
Tasks	Stochastic Optimization
Published	2017-12-27
URL	http://arxiv.org/abs/1712.09677v2
PDF	http://arxiv.org/pdf/1712.09677v2.pdf
PWC	https://paperswithcode.com/paper/momentum-and-stochastic-momentum-for
Repo
Framework

DART: Distribution Aware Retinal Transform for Event-based Cameras


Title	DART: Distribution Aware Retinal Transform for Event-based Cameras
Authors	Bharath Ramesh, Hong Yang, Garrick Orchard, Ngoc Anh Le Thi, Shihao Zhang, Cheng Xiang
Abstract	We introduce a generic visual descriptor, termed as distribution aware retinal transform (DART), that encodes the structural context using log-polar grids for event cameras. The DART descriptor is applied to four different problems, namely object classification, tracking, detection and feature matching: (1) The DART features are directly employed as local descriptors in a bag-of-features classification framework and testing is carried out on four standard event-based object datasets (N-MNIST, MNIST-DVS, CIFAR10-DVS, NCaltech-101). (2) Extending the classification system, tracking is demonstrated using two key novelties: (i) For overcoming the low-sample problem for the one-shot learning of a binary classifier, statistical bootstrapping is leveraged with online learning; (ii) To achieve tracker robustness, the scale and rotation equivariance property of the DART descriptors is exploited for the one-shot learning. (3) To solve the long-term object tracking problem, an object detector is designed using the principle of cluster majority voting. The detection scheme is then combined with the tracker to result in a high intersection-over-union score with augmented ground truth annotations on the publicly available event camera dataset. (4) Finally, the event context encoded by DART greatly simplifies the feature correspondence problem, especially for spatio-temporal slices far apart in time, which has not been explicitly tackled in the event-based vision domain.
Tasks	Event-based vision, Object Classification, Object Tracking, One-Shot Learning
Published	2017-10-30
URL	http://arxiv.org/abs/1710.10800v3
PDF	http://arxiv.org/pdf/1710.10800v3.pdf
PWC	https://paperswithcode.com/paper/dart-distribution-aware-retinal-transform-for
Repo
Framework

Deep Learning for Medical Image Analysis


Title	Deep Learning for Medical Image Analysis
Authors	Mina Rezaei, Haojin Yang, Christoph Meinel
Abstract	This report describes my research activities in the Hasso Plattner Institute and summarizes my Ph.D. plan and several novels, end-to-end trainable approaches for analyzing medical images using deep learning algorithm. In this report, as an example, we explore different novel methods based on deep learning for brain abnormality detection, recognition, and segmentation. This report prepared for the doctoral consortium in the AIME-2017 conference.
Tasks	Anomaly Detection
Published	2017-08-17
URL	http://arxiv.org/abs/1708.08987v1
PDF	http://arxiv.org/pdf/1708.08987v1.pdf
PWC	https://paperswithcode.com/paper/deep-learning-for-medical-image-analysis
Repo
Framework