July 29, 2019

2888 words 14 mins read

Paper Group ANR 68

Finding Competitive Network Architectures Within a Day Using UCT. Black-Box Attacks against RNN based Malware Detection Algorithms. Hierarchical Policy Search via Return-Weighted Density Estimation. Efficient Large-scale Approximate Nearest Neighbor Search on the GPU. Peephole: Predicting Network Performance Before Training. Evaluating the quality …

Finding Competitive Network Architectures Within a Day Using UCT


Title	Finding Competitive Network Architectures Within a Day Using UCT
Authors	Martin Wistuba
Abstract	The design of neural network architectures for a new data set is a laborious task which requires human deep learning expertise. In order to make deep learning available for a broader audience, automated methods for finding a neural network architecture are vital. Recently proposed methods can already achieve human expert level performances. However, these methods have run times of months or even years of GPU computing time, ignoring hardware constraints as faced by many researchers and companies. We propose the use of Monte Carlo planning in combination with two different UCT (upper confidence bound applied to trees) derivations to search for network architectures. We adapt the UCT algorithm to the needs of network architecture search by proposing two ways of sharing information between different branches of the search tree. In an empirical study we are able to demonstrate that this method is able to find competitive networks for MNIST, SVHN and CIFAR-10 in just a single GPU day. Extending the search time to five GPU days, we are able to outperform human architectures and our competitors which consider the same types of layers.
Tasks	Neural Architecture Search
Published	2017-12-20
URL	http://arxiv.org/abs/1712.07420v2
PDF	http://arxiv.org/pdf/1712.07420v2.pdf
PWC	https://paperswithcode.com/paper/finding-competitive-network-architectures
Repo
Framework

Black-Box Attacks against RNN based Malware Detection Algorithms


Title	Black-Box Attacks against RNN based Malware Detection Algorithms
Authors	Weiwei Hu, Ying Tan
Abstract	Recent researches have shown that machine learning based malware detection algorithms are very vulnerable under the attacks of adversarial examples. These works mainly focused on the detection algorithms which use features with fixed dimension, while some researchers have begun to use recurrent neural networks (RNN) to detect malware based on sequential API features. This paper proposes a novel algorithm to generate sequential adversarial examples, which are used to attack a RNN based malware detection system. It is usually hard for malicious attackers to know the exact structures and weights of the victim RNN. A substitute RNN is trained to approximate the victim RNN. Then we propose a generative RNN to output sequential adversarial examples from the original sequential malware inputs. Experimental results showed that RNN based malware detection algorithms fail to detect most of the generated malicious adversarial examples, which means the proposed model is able to effectively bypass the detection algorithms.
Tasks	Malware Detection
Published	2017-05-23
URL	http://arxiv.org/abs/1705.08131v1
PDF	http://arxiv.org/pdf/1705.08131v1.pdf
PWC	https://paperswithcode.com/paper/black-box-attacks-against-rnn-based-malware
Repo
Framework

Hierarchical Policy Search via Return-Weighted Density Estimation


Title	Hierarchical Policy Search via Return-Weighted Density Estimation
Authors	Takayuki Osa, Masashi Sugiyama
Abstract	Learning an optimal policy from a multi-modal reward function is a challenging problem in reinforcement learning (RL). Hierarchical RL (HRL) tackles this problem by learning a hierarchical policy, where multiple option policies are in charge of different strategies corresponding to modes of a reward function and a gating policy selects the best option for a given context. Although HRL has been demonstrated to be promising, current state-of-the-art methods cannot still perform well in complex real-world problems due to the difficulty of identifying modes of the reward function. In this paper, we propose a novel method called hierarchical policy search via return-weighted density estimation (HPSDE), which can efficiently identify the modes through density estimation with return-weighted importance sampling. Our proposed method finds option policies corresponding to the modes of the return function and automatically determines the number and the location of option policies, which significantly reduces the burden of hyper-parameters tuning. Through experiments, we demonstrate that the proposed HPSDE successfully learns option policies corresponding to modes of the return function and that it can be successfully applied to a challenging motion planning problem of a redundant robotic manipulator.
Tasks	Density Estimation, Motion Planning
Published	2017-11-28
URL	http://arxiv.org/abs/1711.10173v2
PDF	http://arxiv.org/pdf/1711.10173v2.pdf
PWC	https://paperswithcode.com/paper/hierarchical-policy-search-via-return
Repo
Framework

Efficient Large-scale Approximate Nearest Neighbor Search on the GPU


Title	Efficient Large-scale Approximate Nearest Neighbor Search on the GPU
Authors	Patrick Wieschollek, Oliver Wang, Alexander Sorkine-Hornung, Hendrik P. A. Lensch
Abstract	We present a new approach for efficient approximate nearest neighbor (ANN) search in high dimensional spaces, extending the idea of Product Quantization. We propose a two-level product and vector quantization tree that reduces the number of vector comparisons required during tree traversal. Our approach also includes a novel highly parallelizable re-ranking method for candidate vectors by efficiently reusing already computed intermediate values. Due to its small memory footprint during traversal, the method lends itself to an efficient, parallel GPU implementation. This Product Quantization Tree (PQT) approach significantly outperforms recent state of the art methods for high dimensional nearest neighbor queries on standard reference datasets. Ours is the first work that demonstrates GPU performance superior to CPU performance on high dimensional, large scale ANN problems in time-critical real-world applications, like loop-closing in videos.
Tasks	Quantization
Published	2017-02-20
URL	http://arxiv.org/abs/1702.05911v1
PDF	http://arxiv.org/pdf/1702.05911v1.pdf
PWC	https://paperswithcode.com/paper/efficient-large-scale-approximate-nearest
Repo
Framework

Peephole: Predicting Network Performance Before Training


Title	Peephole: Predicting Network Performance Before Training
Authors	Boyang Deng, Junjie Yan, Dahua Lin
Abstract	The quest for performant networks has been a significant force that drives the advancements of deep learning in recent years. While rewarding, improving network design has never been an easy journey. The large design space combined with the tremendous cost required for network training poses a major obstacle to this endeavor. In this work, we propose a new approach to this problem, namely, predicting the performance of a network before training, based on its architecture. Specifically, we develop a unified way to encode individual layers into vectors and bring them together to form an integrated description via LSTM. Taking advantage of the recurrent network’s strong expressive power, this method can reliably predict the performances of various network architectures. Our empirical studies showed that it not only achieved accurate predictions but also produced consistent rankings across datasets – a key desideratum in performance prediction.
Tasks
Published	2017-12-09
URL	http://arxiv.org/abs/1712.03351v1
PDF	http://arxiv.org/pdf/1712.03351v1.pdf
PWC	https://paperswithcode.com/paper/peephole-predicting-network-performance
Repo
Framework

Evaluating the quality of tourist agendas customized to different travel styles


Title	Evaluating the quality of tourist agendas customized to different travel styles
Authors	Jesús Ibáñez-Ruiz, Laura Sebastiá, Eva Onaindia
Abstract	Many tourist applications provide a personalized tourist agenda with the list of recommended activities to the user. These applications must undoubtedly deal with the constraints and preferences that define the user interests. Among these preferences, we can find those that define the travel style of the user, such as the rhythm of the trip, the number of visits to include in the tour or the priority to visits of special interest for the user. In this paper, we deal with the task of creating a customized tourist agenda as a planning and scheduling application capable of conveniently scheduling the most appropriate goals (visits) so as to maximize the user satisfaction with the tourist route. This paper makes an analysis of the meaning of the travel style preferences and compares the quality of the solutions obtained by two different solvers, a PDDL-based planner and a Constraint Satisfaction Problem solver. We also define several quality metrics and perform extensive experiments in order to evaluate the results obtained with both solvers.
Tasks
Published	2017-06-17
URL	http://arxiv.org/abs/1706.05518v1
PDF	http://arxiv.org/pdf/1706.05518v1.pdf
PWC	https://paperswithcode.com/paper/evaluating-the-quality-of-tourist-agendas
Repo
Framework

A Universal Marginalizer for Amortized Inference in Generative Models


Title	A Universal Marginalizer for Amortized Inference in Generative Models
Authors	Laura Douglas, Iliyan Zarov, Konstantinos Gourgoulias, Chris Lucas, Chris Hart, Adam Baker, Maneesh Sahani, Yura Perov, Saurabh Johri
Abstract	We consider the problem of inference in a causal generative model where the set of available observations differs between data instances. We show how combining samples drawn from the graphical model with an appropriate masking function makes it possible to train a single neural network to approximate all the corresponding conditional marginal distributions and thus amortize the cost of inference. We further demonstrate that the efficiency of importance sampling may be improved by basing proposals on the output of the neural network. We also outline how the same network can be used to generate samples from an approximate joint posterior via a chain decomposition of the graph.
Tasks
Published	2017-11-02
URL	http://arxiv.org/abs/1711.00695v1
PDF	http://arxiv.org/pdf/1711.00695v1.pdf
PWC	https://paperswithcode.com/paper/a-universal-marginalizer-for-amortized
Repo
Framework

Facial Feature Tracking under Varying Facial Expressions and Face Poses based on Restricted Boltzmann Machines


Title	Facial Feature Tracking under Varying Facial Expressions and Face Poses based on Restricted Boltzmann Machines
Authors	Yue Wu, Zuoguan Wang, Qiang Ji
Abstract	Facial feature tracking is an active area in computer vision due to its relevance to many applications. It is a nontrivial task, since faces may have varying facial expressions, poses or occlusions. In this paper, we address this problem by proposing a face shape prior model that is constructed based on the Restricted Boltzmann Machines (RBM) and their variants. Specifically, we first construct a model based on Deep Belief Networks to capture the face shape variations due to varying facial expressions for near-frontal view. To handle pose variations, the frontal face shape prior model is incorporated into a 3-way RBM model that could capture the relationship between frontal face shapes and non-frontal face shapes. Finally, we introduce methods to systematically combine the face shape prior models with image measurements of facial feature points. Experiments on benchmark databases show that with the proposed method, facial feature points can be tracked robustly and accurately even if faces have significant facial expressions and poses.
Tasks
Published	2017-09-18
URL	http://arxiv.org/abs/1709.05731v1
PDF	http://arxiv.org/pdf/1709.05731v1.pdf
PWC	https://paperswithcode.com/paper/facial-feature-tracking-under-varying-facial
Repo
Framework

BlockDrop: Dynamic Inference Paths in Residual Networks


Title	BlockDrop: Dynamic Inference Paths in Residual Networks
Authors	Zuxuan Wu, Tushar Nagarajan, Abhishek Kumar, Steven Rennie, Larry S. Davis, Kristen Grauman, Rogerio Feris
Abstract	Very deep convolutional neural networks offer excellent recognition results, yet their computational expense limits their impact for many real-world applications. We introduce BlockDrop, an approach that learns to dynamically choose which layers of a deep network to execute during inference so as to best reduce total computation without degrading prediction accuracy. Exploiting the robustness of Residual Networks (ResNets) to layer dropping, our framework selects on-the-fly which residual blocks to evaluate for a given novel image. In particular, given a pretrained ResNet, we train a policy network in an associative reinforcement learning setting for the dual reward of utilizing a minimal number of blocks while preserving recognition accuracy. We conduct extensive experiments on CIFAR and ImageNet. The results provide strong quantitative and qualitative evidence that these learned policies not only accelerate inference but also encode meaningful visual information. Built upon a ResNet-101 model, our method achieves a speedup of 20% on average, going as high as 36% for some images, while maintaining the same 76.4% top-1 accuracy on ImageNet.
Tasks
Published	2017-11-22
URL	http://arxiv.org/abs/1711.08393v4
PDF	http://arxiv.org/pdf/1711.08393v4.pdf
PWC	https://paperswithcode.com/paper/blockdrop-dynamic-inference-paths-in-residual
Repo
Framework

Memory Based Online Learning of Deep Representations from Video Streams


Title	Memory Based Online Learning of Deep Representations from Video Streams
Authors	Federico Pernici, Federico Bartoli, Matteo Bruni, Alberto Del Bimbo
Abstract	We present a novel online unsupervised method for face identity learning from video streams. The method exploits deep face descriptors together with a memory based learning mechanism that takes advantage of the temporal coherence of visual data. Specifically, we introduce a discriminative feature matching solution based on Reverse Nearest Neighbour and a feature forgetting strategy that detect redundant features and discard them appropriately while time progresses. It is shown that the proposed learning procedure is asymptotically stable and can be effectively used in relevant applications like multiple face identification and tracking from unconstrained video streams. Experimental results show that the proposed method achieves comparable results in the task of multiple face tracking and better performance in face identification with offline approaches exploiting future information. Code will be publicly available.
Tasks	Face Identification
Published	2017-11-17
URL	http://arxiv.org/abs/1711.07368v1
PDF	http://arxiv.org/pdf/1711.07368v1.pdf
PWC	https://paperswithcode.com/paper/memory-based-online-learning-of-deep
Repo
Framework

Combating Human Trafficking with Deep Multimodal Models


Title	Combating Human Trafficking with Deep Multimodal Models
Authors	Edmund Tong, Amir Zadeh, Cara Jones, Louis-Philippe Morency
Abstract	Human trafficking is a global epidemic affecting millions of people across the planet. Sex trafficking, the dominant form of human trafficking, has seen a significant rise mostly due to the abundance of escort websites, where human traffickers can openly advertise among at-will escort advertisements. In this paper, we take a major step in the automatic detection of advertisements suspected to pertain to human trafficking. We present a novel dataset called Trafficking-10k, with more than 10,000 advertisements annotated for this task. The dataset contains two sources of information per advertisement: text and images. For the accurate detection of trafficking advertisements, we designed and trained a deep multimodal model called the Human Trafficking Deep Network (HTDN).
Tasks
Published	2017-05-08
URL	http://arxiv.org/abs/1705.02735v1
PDF	http://arxiv.org/pdf/1705.02735v1.pdf
PWC	https://paperswithcode.com/paper/combating-human-trafficking-with-deep
Repo
Framework

On the construction of probabilistic Newton-type algorithms


Title	On the construction of probabilistic Newton-type algorithms
Authors	Adrian G. Wills, Thomas B. Schön
Abstract	It has recently been shown that many of the existing quasi-Newton algorithms can be formulated as learning algorithms, capable of learning local models of the cost functions. Importantly, this understanding allows us to safely start assembling probabilistic Newton-type algorithms, applicable in situations where we only have access to noisy observations of the cost function and its derivatives. This is where our interest lies. We make contributions to the use of the non-parametric and probabilistic Gaussian process models in solving these stochastic optimisation problems. Specifically, we present a new algorithm that unites these approximations together with recent probabilistic line search routines to deliver a probabilistic quasi-Newton approach. We also show that the probabilistic optimisation algorithms deliver promising results on challenging nonlinear system identification problems where the very nature of the problem is such that we can only access the cost function and its derivative via noisy observations, since there are no closed-form expressions available.
Tasks
Published	2017-04-05
URL	http://arxiv.org/abs/1704.01382v1
PDF	http://arxiv.org/pdf/1704.01382v1.pdf
PWC	https://paperswithcode.com/paper/on-the-construction-of-probabilistic-newton
Repo
Framework

Efficient Online Surface Correction for Real-time Large-Scale 3D Reconstruction


Title	Efficient Online Surface Correction for Real-time Large-Scale 3D Reconstruction
Authors	Robert Maier, Raphael Schaller, Daniel Cremers
Abstract	State-of-the-art methods for large-scale 3D reconstruction from RGB-D sensors usually reduce drift in camera tracking by globally optimizing the estimated camera poses in real-time without simultaneously updating the reconstructed surface on pose changes. We propose an efficient on-the-fly surface correction method for globally consistent dense 3D reconstruction of large-scale scenes. Our approach uses a dense Visual RGB-D SLAM system that estimates the camera motion in real-time on a CPU and refines it in a global pose graph optimization. Consecutive RGB-D frames are locally fused into keyframes, which are incorporated into a sparse voxel hashed Signed Distance Field (SDF) on the GPU. On pose graph updates, the SDF volume is corrected on-the-fly using a novel keyframe re-integration strategy with reduced GPU-host streaming. We demonstrate in an extensive quantitative evaluation that our method is up to 93% more runtime efficient compared to the state-of-the-art and requires significantly less memory, with only negligible loss of surface quality. Overall, our system requires only a single GPU and allows for real-time surface correction of large environments.
Tasks	3D Reconstruction
Published	2017-09-12
URL	http://arxiv.org/abs/1709.03763v1
PDF	http://arxiv.org/pdf/1709.03763v1.pdf
PWC	https://paperswithcode.com/paper/efficient-online-surface-correction-for-real
Repo
Framework

Vertex Nomination Via Seeded Graph Matching


Title	Vertex Nomination Via Seeded Graph Matching
Authors	Heather G. Patsolic, Youngser Park, Vince Lyzinski, Carey E. Priebe
Abstract	Consider two networks on overlapping, non-identical vertex sets. Given vertices of interest in the first network, we seek to identify the corresponding vertices, if any exist, in the second network. While in moderately sized networks graph matching methods can be applied directly to recover the missing correspondences, herein we present a principled methodology appropriate for situations in which the networks are too large for brute-force graph matching. Our methodology identifies vertices in a local neighborhood of the vertices of interest in the first network that have verifiable corresponding vertices in the second network. Leveraging these known correspondences, referred to as seeds, we match the induced subgraphs in each network generated by the neighborhoods of these verified seeds, and rank the vertices of the second network in terms of the most likely matches to the original vertices of interest. We demonstrate the applicability of our methodology through simulations and real data examples.
Tasks	Graph Matching
Published	2017-05-01
URL	https://arxiv.org/abs/1705.00674v5
PDF	https://arxiv.org/pdf/1705.00674v5.pdf
PWC	https://paperswithcode.com/paper/vertex-nomination-via-seeded-graph-matching
Repo
Framework

Approximate Inference with Amortised MCMC


Title	Approximate Inference with Amortised MCMC
Authors	Yingzhen Li, Richard E. Turner, Qiang Liu
Abstract	We propose a novel approximate inference algorithm that approximates a target distribution by amortising the dynamics of a user-selected MCMC sampler. The idea is to initialise MCMC using samples from an approximation network, apply the MCMC operator to improve these samples, and finally use the samples to update the approximation network thereby improving its quality. This provides a new generic framework for approximate inference, allowing us to deploy highly complex, or implicitly defined approximation families with intractable densities, including approximations produced by warping a source of randomness through a deep neural network. Experiments consider image modelling with deep generative models as a challenging test for the method. Deep models trained using amortised MCMC are shown to generate realistic looking samples as well as producing diverse imputations for images with regions of missing pixels.
Tasks
Published	2017-02-27
URL	http://arxiv.org/abs/1702.08343v2
PDF	http://arxiv.org/pdf/1702.08343v2.pdf
PWC	https://paperswithcode.com/paper/approximate-inference-with-amortised-mcmc
Repo
Framework