October 18, 2019

3076 words 15 mins read

Paper Group ANR 541

Optical Flow Based Background Subtraction with a Moving Camera: Application to Autonomous Driving. Convolutional Hashing for Automated Scene Matching. To compress or not to compress: Understanding the Interactions between Adversarial Attacks and Neural Network Compression. Autotune: A Derivative-free Optimization Framework for Hyperparameter Tuning …

Optical Flow Based Background Subtraction with a Moving Camera: Application to Autonomous Driving


Title	Optical Flow Based Background Subtraction with a Moving Camera: Application to Autonomous Driving
Authors	Sotirios Diamantas, Kostas Alexis
Abstract	In this research we present a novel algorithm for background subtraction using a moving camera. Our algorithm is based purely on visual information obtained from a camera mounted on an electric bus, operating in downtown Reno which automatically detects moving objects of interest with the view to provide a fully autonomous vehicle. In our approach we exploit the optical flow vectors generated by the motion of the camera while keeping parameter assumptions a minimum. At first, we estimate the Focus of Expansion, which is used to model and simulate 3D points given the intrinsic parameters of the camera, and perform multiple linear regression to estimate the regression equation parameters and implement on the real data set of every frame to identify moving objects. We validated our algorithm using data taken from a common bus route.
Tasks	Autonomous Driving, Optical Flow Estimation
Published	2018-11-16
URL	http://arxiv.org/abs/1811.06660v1
PDF	http://arxiv.org/pdf/1811.06660v1.pdf
PWC	https://paperswithcode.com/paper/optical-flow-based-background-subtraction
Repo
Framework

Convolutional Hashing for Automated Scene Matching


Title	Convolutional Hashing for Automated Scene Matching
Authors	Martin Loncaric, Bowei Liu, Ryan Weber
Abstract	We present a powerful new loss function and training scheme for learning binary hash functions. In particular, we demonstrate our method by creating for the first time a neural network that outperforms state-of-the-art Haar wavelets and color layout descriptors at the task of automated scene matching. By accurately relating distance on the manifold of network outputs to distance in Hamming space, we achieve a 100-fold reduction in nontrivial false positive rate and significantly higher true positive rate. We expect our insights to provide large wins for hashing models applied to other information retrieval hashing tasks as well.
Tasks	Information Retrieval
Published	2018-02-09
URL	http://arxiv.org/abs/1802.03101v1
PDF	http://arxiv.org/pdf/1802.03101v1.pdf
PWC	https://paperswithcode.com/paper/convolutional-hashing-for-automated-scene
Repo
Framework

To compress or not to compress: Understanding the Interactions between Adversarial Attacks and Neural Network Compression


Title	To compress or not to compress: Understanding the Interactions between Adversarial Attacks and Neural Network Compression
Authors	Yiren Zhao, Ilia Shumailov, Robert Mullins, Ross Anderson
Abstract	As deep neural networks (DNNs) become widely used, pruned and quantised models are becoming ubiquitous on edge devices; such compressed DNNs are popular for lowering computational requirements. Meanwhile, recent studies show that adversarial samples can be effective at making DNNs misclassify. We, therefore, investigate the extent to which adversarial samples are transferable between uncompressed and compressed DNNs. We find that adversarial samples remain transferable for both pruned and quantised models. For pruning, the adversarial samples generated from heavily pruned models remain effective on uncompressed models. For quantisation, we find the transferability of adversarial samples is highly sensitive to integer precision.
Tasks	Neural Network Compression
Published	2018-09-29
URL	http://arxiv.org/abs/1810.00208v1
PDF	http://arxiv.org/pdf/1810.00208v1.pdf
PWC	https://paperswithcode.com/paper/to-compress-or-not-to-compress-understanding
Repo
Framework

Autotune: A Derivative-free Optimization Framework for Hyperparameter Tuning


Title	Autotune: A Derivative-free Optimization Framework for Hyperparameter Tuning
Authors	Patrick Koch, Oleg Golovidov, Steven Gardner, Brett Wujek, Joshua Griffin, Yan Xu
Abstract	Machine learning applications often require hyperparameter tuning. The hyperparameters usually drive both the efficiency of the model training process and the resulting model quality. For hyperparameter tuning, machine learning algorithms are complex black-boxes. This creates a class of challenging optimization problems, whose objective functions tend to be nonsmooth, discontinuous, unpredictably varying in computational expense, and include continuous, categorical, and/or integer variables. Further, function evaluations can fail for a variety of reasons including numerical difficulties or hardware failures. Additionally, not all hyperparameter value combinations are compatible, which creates so called hidden constraints. Robust and efficient optimization algorithms are needed for hyperparameter tuning. In this paper we present an automated parallel derivative-free optimization framework called \textbf{Autotune}, which combines a number of specialized sampling and search methods that are very effective in tuning machine learning models despite these challenges. Autotune provides significantly improved models over using default hyperparameter settings with minimal user interaction on real-world applications. Given the inherent expense of training numerous candidate models, we demonstrate the effectiveness of Autotune’s search methods and the efficient distributed and parallel paradigms for training and tuning models, and also discuss the resource trade-offs associated with the ability to both distribute the training process and parallelize the tuning process.
Tasks
Published	2018-04-20
URL	http://arxiv.org/abs/1804.07824v2
PDF	http://arxiv.org/pdf/1804.07824v2.pdf
PWC	https://paperswithcode.com/paper/autotune-a-derivative-free-optimization
Repo
Framework

Universal Perturbation Attack Against Image Retrieval


Title	Universal Perturbation Attack Against Image Retrieval
Authors	Jie Li, Rongrong Ji, Hong Liu, Xiaopeng Hong, Yue Gao, Qi Tian
Abstract	Universal adversarial perturbations (UAPs), a.k.a. input-agnostic perturbations, has been proved to exist and be able to fool cutting-edge deep learning models on most of the data samples. Existing UAP methods mainly focus on attacking image classification models. Nevertheless, little attention has been paid to attacking image retrieval systems. In this paper, we make the first attempt in attacking image retrieval systems. Concretely, image retrieval attack is to make the retrieval system return irrelevant images to the query at the top ranking list. It plays an important role to corrupt the neighbourhood relationships among features in image retrieval attack. To this end, we propose a novel method to generate retrieval-against UAP to break the neighbourhood relationships of image features via degrading the corresponding ranking metric. To expand the attack method to scenarios with varying input sizes or untouchable network parameters, a multi-scale random resizing scheme and a ranking distillation strategy are proposed. We evaluate the proposed method on four widely-used image retrieval datasets, and report a significant performance drop in terms of different metrics, such as mAP and mP@10. Finally, we test our attack methods on the real-world visual search engine, i.e., Google Images, which demonstrates the practical potentials of our methods.
Tasks	Image Classification, Image Retrieval
Published	2018-12-03
URL	https://arxiv.org/abs/1812.00552v2
PDF	https://arxiv.org/pdf/1812.00552v2.pdf
PWC	https://paperswithcode.com/paper/universal-perturbation-attack-against-image
Repo
Framework

Multi-task Maximum Entropy Inverse Reinforcement Learning


Title	Multi-task Maximum Entropy Inverse Reinforcement Learning
Authors	Adam Gleave, Oliver Habryka
Abstract	Multi-task Inverse Reinforcement Learning (IRL) is the problem of inferring multiple reward functions from expert demonstrations. Prior work, built on Bayesian IRL, is unable to scale to complex environments due to computational constraints. This paper contributes a formulation of multi-task IRL in the more computationally efficient Maximum Causal Entropy (MCE) IRL framework. Experiments show our approach can perform one-shot imitation learning in a gridworld environment that single-task IRL algorithms need hundreds of demonstrations to solve. We outline preliminary work using meta-learning to extend our method to the function approximator setting of modern MCE IRL algorithms. Evaluating on multi-task variants of common simulated robotics benchmarks, we discover serious limitations of these IRL algorithms, and conclude with suggestions for further work.
Tasks	Imitation Learning, Meta-Learning
Published	2018-05-22
URL	http://arxiv.org/abs/1805.08882v2
PDF	http://arxiv.org/pdf/1805.08882v2.pdf
PWC	https://paperswithcode.com/paper/multi-task-maximum-entropy-inverse
Repo
Framework

Dynamic texture analysis with diffusion in networks


Title	Dynamic texture analysis with diffusion in networks
Authors	Lucas C. Ribas, Wesley N. Goncalves, Odemir M. Bruno
Abstract	Dynamic texture is a field of research that has gained considerable interest from computer vision community due to the explosive growth of multimedia databases. In addition, dynamic texture is present in a wide range of videos, which makes it very important in expert systems based on videos such as medical systems, traffic monitoring systems, forest fire detection system, among others. In this paper, a new method for dynamic texture characterization based on diffusion in directed networks is proposed. The dynamic texture is modeled as a directed network. The method consists in the analysis of the dynamic of this network after a series of graph cut transformations based on the edge weights. For each network transformation, the activity for each vertex is estimated. The activity is the relative frequency that one vertex is visited by random walks in balance. Then, texture descriptor is constructed by concatenating the activity histograms. The main contributions of this paper are the use of directed network modeling and diffusion in network to dynamic texture characterization. These tend to provide better performance in dynamic textures classification. Experiments with rotation and interference of the motion pattern were conducted in order to demonstrate the robustness of the method. The proposed approach is compared to other dynamic texture methods on two very well know dynamic texture database and on traffic condition classification, and outperform in most of the cases.
Tasks	Texture Classification
Published	2018-06-27
URL	http://arxiv.org/abs/1806.10681v1
PDF	http://arxiv.org/pdf/1806.10681v1.pdf
PWC	https://paperswithcode.com/paper/dynamic-texture-analysis-with-diffusion-in
Repo
Framework

AI Neurotechnology for Aging Societies – Task-load and Dementia EEG Digital Biomarker Development Using Information Geometry Machine Learning Methods


Title	AI Neurotechnology for Aging Societies – Task-load and Dementia EEG Digital Biomarker Development Using Information Geometry Machine Learning Methods
Authors	Tomasz M. Rutkowski, Qibin Zhao, Masao S. Abe, Mihoko Otake
Abstract	Dementia and especially Alzheimer’s disease (AD) are the most common causes of cognitive decline in elderly people. A spread of the above mentioned mental health problems in aging societies is causing a significant medical and economic burden in many countries around the world. According to a recent World Health Organization (WHO) report, it is approximated that currently, worldwide, about 47 million people live with a dementia spectrum of neurocognitive disorders. This number is expected to triple by 2050, which calls for possible application of AI-based technologies to support an early screening for preventive interventions and a subsequent mental wellbeing monitoring as well as maintenance with so-called digital-pharma or beyond a pill therapeutical approaches. This paper discusses our attempt and preliminary results of brainwave (EEG) techniques to develop digital biomarkers for dementia progress detection and monitoring. We present an information geometry-based classification approach for automatic EEG-derived event related responses (ERPs) discrimination of low versus high task-load auditory or tactile stimuli recognition, of which amplitude and latency variabilities are similar to those in dementia. The discussed approach is a step forward to develop AI, and especially machine learning (ML) approaches, for the subsequent application to mild-cognitive impairment (MCI) and AD diagnostics.
Tasks	EEG
Published	2018-11-30
URL	http://arxiv.org/abs/1811.12642v1
PDF	http://arxiv.org/pdf/1811.12642v1.pdf
PWC	https://paperswithcode.com/paper/ai-neurotechnology-for-aging-societies-task
Repo
Framework

Generative Adversarial Networks for Unpaired Voice Transformation on Impaired Speech


Title	Generative Adversarial Networks for Unpaired Voice Transformation on Impaired Speech
Authors	Li-Wei Chen, Hung-Yi Lee, Yu Tsao
Abstract	This paper focuses on using voice conversion (VC) to improve the speech intelligibility of surgical patients who have had parts of their articulators removed. Due to the difficulty of data collection, VC without parallel data is highly desired. Although techniques for unparallel VC, for example, CycleGAN, have been developed, they usually focus on transforming the speaker identity, and directly transforming the speech of one speaker to that of another speaker and as such do not address the task here. In this paper, we propose a new approach for unparallel VC. The proposed approach transforms impaired speech to normal speech while preserving the linguistic content and speaker characteristics. To our knowledge, this is the first end-to-end GAN-based unsupervised VC model applied to impaired speech. The experimental results show that the proposed approach outperforms CycleGAN.
Tasks	Voice Conversion
Published	2018-10-30
URL	https://arxiv.org/abs/1810.12656v3
PDF	https://arxiv.org/pdf/1810.12656v3.pdf
PWC	https://paperswithcode.com/paper/generative-adversarial-networks-for-unpaired
Repo
Framework

Preference-based Online Learning with Dueling Bandits: A Survey


Title	Preference-based Online Learning with Dueling Bandits: A Survey
Authors	Robert Busa-Fekete, Eyke Hüllermeier, Adil El Mesaoudi-Paul
Abstract	In machine learning, the notion of multi-armed bandits refers to a class of online learning problems, in which an agent is supposed to simultaneously explore and exploit a given set of choice alternatives in the course of a sequential decision process. In the standard setting, the agent learns from stochastic feedback in the form of real-valued rewards. In many applications, however, numerical reward signals are not readily available – instead, only weaker information is provided, in particular relative preferences in the form of qualitative comparisons between pairs of alternatives. This observation has motivated the study of variants of the multi-armed bandit problem, in which more general representations are used both for the type of feedback to learn from and the target of prediction. The aim of this paper is to provide a survey of the state of the art in this field, referred to as preference-based multi-armed bandits or dueling bandits. To this end, we provide an overview of problems that have been considered in the literature as well as methods for tackling them. Our taxonomy is mainly based on the assumptions made by these methods about the data-generating process and, related to this, the properties of the preference-based feedback.
Tasks	Multi-Armed Bandits
Published	2018-07-30
URL	http://arxiv.org/abs/1807.11398v1
PDF	http://arxiv.org/pdf/1807.11398v1.pdf
PWC	https://paperswithcode.com/paper/preference-based-online-learning-with-dueling
Repo
Framework

Learning View-Specific Deep Networks for Person Re-Identification


Title	Learning View-Specific Deep Networks for Person Re-Identification
Authors	Zhanxiang Feng, Jianhuang Lai, Xiaohua Xie
Abstract	In recent years, a growing body of research has focused on the problem of person re-identification (re-id). The re-id techniques attempt to match the images of pedestrians from disjoint non-overlapping camera views. A major challenge of re-id is the serious intra-class variations caused by changing viewpoints. To overcome this challenge, we propose a deep neural network-based framework which utilizes the view information in the feature extraction stage. The proposed framework learns a view-specific network for each camera view with a cross-view Euclidean constraint (CV-EC) and a cross-view center loss (CV-CL). We utilize CV-EC to decrease the margin of the features between diverse views and extend the center loss metric to a view-specific version to better adapt the re-id problem. Moreover, we propose an iterative algorithm to optimize the parameters of the view-specific networks from coarse to fine. The experiments demonstrate that our approach significantly improves the performance of the existing deep networks and outperforms the state-of-the-art methods on the VIPeR, CUHK01, CUHK03, SYSU-mReId, and Market-1501 benchmarks.
Tasks	Person Re-Identification
Published	2018-03-30
URL	http://arxiv.org/abs/1803.11333v1
PDF	http://arxiv.org/pdf/1803.11333v1.pdf
PWC	https://paperswithcode.com/paper/learning-view-specific-deep-networks-for
Repo
Framework

Small nonlinearities in activation functions create bad local minima in neural networks


Title	Small nonlinearities in activation functions create bad local minima in neural networks
Authors	Chulhee Yun, Suvrit Sra, Ali Jadbabaie
Abstract	We investigate the loss surface of neural networks. We prove that even for one-hidden-layer networks with “slightest” nonlinearity, the empirical risks have spurious local minima in most cases. Our results thus indicate that in general “no spurious local minima” is a property limited to deep linear networks, and insights obtained from linear networks may not be robust. Specifically, for ReLU(-like) networks we constructively prove that for almost all practical datasets there exist infinitely many local minima. We also present a counterexample for more general activations (sigmoid, tanh, arctan, ReLU, etc.), for which there exists a bad local minimum. Our results make the least restrictive assumptions relative to existing results on spurious local optima in neural networks. We complete our discussion by presenting a comprehensive characterization of global optimality for deep linear networks, which unifies other results on this topic.
Tasks
Published	2018-02-10
URL	https://arxiv.org/abs/1802.03487v4
PDF	https://arxiv.org/pdf/1802.03487v4.pdf
PWC	https://paperswithcode.com/paper/small-nonlinearities-in-activation-functions
Repo
Framework

Generalization Error Bounds with Probabilistic Guarantee for SGD in Nonconvex Optimization


Title	Generalization Error Bounds with Probabilistic Guarantee for SGD in Nonconvex Optimization
Authors	Yi Zhou, Yingbin Liang, Huishuai Zhang
Abstract	The success of deep learning has led to a rising interest in the generalization property of the stochastic gradient descent (SGD) method, and stability is one popular approach to study it. Existing works based on stability have studied nonconvex loss functions, but only considered the generalization error of the SGD in expectation. In this paper, we establish various generalization error bounds with probabilistic guarantee for the SGD. Specifically, for both general nonconvex loss functions and gradient dominant loss functions, we characterize the on-average stability of the iterates generated by SGD in terms of the on-average variance of the stochastic gradients. Such characterization leads to improved bounds for the generalization error for SGD. We then study the regularized risk minimization problem with strongly convex regularizers, and obtain improved generalization error bounds for proximal SGD. With strongly convex regularizers, we further establish the generalization error bounds for nonconvex loss functions under proximal SGD with high-probability guarantee, i.e., exponential concentration in probability.
Tasks
Published	2018-02-19
URL	http://arxiv.org/abs/1802.06903v3
PDF	http://arxiv.org/pdf/1802.06903v3.pdf
PWC	https://paperswithcode.com/paper/generalization-error-bounds-with
Repo
Framework

Ischemic Stroke Lesion Segmentation in CT Perfusion Scans using Pyramid Pooling and Focal Loss


Title	Ischemic Stroke Lesion Segmentation in CT Perfusion Scans using Pyramid Pooling and Focal Loss
Authors	S. Mazdak Abulnaga, Jonathan Rubin
Abstract	We present a fully convolutional neural network for segmenting ischemic stroke lesions in CT perfusion images for the ISLES 2018 challenge. Treatment of stroke is time sensitive and current standards for lesion identification require manual segmentation, a time consuming and challenging process. Automatic segmentation methods present the possibility of accurately identifying lesions and improving treatment planning. Our model is based on the PSPNet, a network architecture that makes use of pyramid pooling to provide global and local contextual information. To learn the varying shapes of the lesions, we train our network using focal loss, a loss function designed for the network to focus on learning the more difficult samples. We compare our model to networks trained using the U-Net and V-Net architectures. Our approach demonstrates effective performance in lesion segmentation and ranked among the top performers at the challenge conclusion.
Tasks	Ischemic Stroke Lesion Segmentation, Lesion Segmentation
Published	2018-11-02
URL	http://arxiv.org/abs/1811.01085v1
PDF	http://arxiv.org/pdf/1811.01085v1.pdf
PWC	https://paperswithcode.com/paper/ischemic-stroke-lesion-segmentation-in-ct
Repo
Framework

Efficient Training on Very Large Corpora via Gramian Estimation


Title	Efficient Training on Very Large Corpora via Gramian Estimation
Authors	Walid Krichene, Nicolas Mayoraz, Steffen Rendle, Li Zhang, Xinyang Yi, Lichan Hong, Ed Chi, John Anderson
Abstract	We study the problem of learning similarity functions over very large corpora using neural network embedding models. These models are typically trained using SGD with sampling of random observed and unobserved pairs, with a number of samples that grows quadratically with the corpus size, making it expensive to scale to very large corpora. We propose new efficient methods to train these models without having to sample unobserved pairs. Inspired by matrix factorization, our approach relies on adding a global quadratic penalty to all pairs of examples and expressing this term as the matrix-inner-product of two generalized Gramians. We show that the gradient of this term can be efficiently computed by maintaining estimates of the Gramians, and develop variance reduction schemes to improve the quality of the estimates. We conduct large-scale experiments that show a significant improvement in training time and generalization quality compared to traditional sampling methods.
Tasks	Network Embedding
Published	2018-07-18
URL	http://arxiv.org/abs/1807.07187v1
PDF	http://arxiv.org/pdf/1807.07187v1.pdf
PWC	https://paperswithcode.com/paper/efficient-training-on-very-large-corpora-via
Repo
Framework