October 16, 2019

3196 words 16 mins read

Paper Group ANR 1118

Naive Bayes Entrapment Detection for Planetary Rovers. Graph-Dependent Implicit Regularisation for Distributed Stochastic Subgradient Descent. $\mathcal{G}$-SGD: Optimizing ReLU Neural Networks in its Positively Scale-Invariant Space. Theoretical Guarantees of Deep Embedding Losses Under Label Noise. Deceiving End-to-End Deep Learning Malware Detec …

Naive Bayes Entrapment Detection for Planetary Rovers


Title	Naive Bayes Entrapment Detection for Planetary Rovers
Authors	Dicong Qiu
Abstract	Entrapment detection is a prerequisite for planetary rovers to perform autonomous rescue procedure. In this study, rover entrapment and approximated entrapment criteria are formally defined. Entrapment detection using Naive Bayes classifiers is proposed and discussed along with results from experiments where the Naive Bayes entrapment detector is applied to AutoKralwer rovers. And final conclusions and further discussions are presented in the final section.
Tasks
Published	2018-01-31
URL	http://arxiv.org/abs/1801.10571v1
PDF	http://arxiv.org/pdf/1801.10571v1.pdf
PWC	https://paperswithcode.com/paper/naive-bayes-entrapment-detection-for
Repo
Framework

Graph-Dependent Implicit Regularisation for Distributed Stochastic Subgradient Descent


Title	Graph-Dependent Implicit Regularisation for Distributed Stochastic Subgradient Descent
Authors	Dominic Richards, Patrick Rebeschini
Abstract	We propose graph-dependent implicit regularisation strategies for distributed stochastic subgradient descent (Distributed SGD) for convex problems in multi-agent learning. Under the standard assumptions of convexity, Lipschitz continuity, and smoothness, we establish statistical learning rates that retain, up to logarithmic terms, centralised statistical guarantees through implicit regularisation (step size tuning and early stopping) with appropriate dependence on the graph topology. Our approach avoids the need for explicit regularisation in decentralised learning problems, such as adding constraints to the empirical risk minimisation rule. Particularly for distributed methods, the use of implicit regularisation allows the algorithm to remain simple, without projections or dual methods. To prove our results, we establish graph-independent generalisation bounds for Distributed SGD that match the centralised setting (using algorithmic stability), and we establish graph-dependent optimisation bounds that are of independent interest. We present numerical experiments to show that the qualitative nature of the upper bounds we derive can be representative of real behaviours.
Tasks
Published	2018-09-18
URL	http://arxiv.org/abs/1809.06958v1
PDF	http://arxiv.org/pdf/1809.06958v1.pdf
PWC	https://paperswithcode.com/paper/graph-dependent-implicit-regularisation-for
Repo
Framework

$\mathcal{G}$-SGD: Optimizing ReLU Neural Networks in its Positively Scale-Invariant Space


Title	$\mathcal{G}$-SGD: Optimizing ReLU Neural Networks in its Positively Scale-Invariant Space
Authors	Qi Meng, Shuxin Zheng, Huishuai Zhang, Wei Chen, Zhi-Ming Ma, Tie-Yan Liu
Abstract	It is well known that neural networks with rectified linear units (ReLU) activation functions are positively scale-invariant. Conventional algorithms like stochastic gradient descent optimize the neural networks in the vector space of weights, which is, however, not positively scale-invariant. This mismatch may lead to problems during the optimization process. Then, a natural question is: \emph{can we construct a new vector space that is positively scale-invariant and sufficient to represent ReLU neural networks so as to better facilitate the optimization process }? In this paper, we provide our positive answer to this question. First, we conduct a formal study on the positive scaling operators which forms a transformation group, denoted as $\mathcal{G}$. We show that the value of a path (i.e. the product of the weights along the path) in the neural network is invariant to positive scaling and prove that the value vector of all the paths is sufficient to represent the neural networks under mild conditions. Second, we show that one can identify some basis paths out of all the paths and prove that the linear span of their value vectors (denoted as $\mathcal{G}$-space) is an invariant space with lower dimension under the positive scaling group. Finally, we design stochastic gradient descent algorithm in $\mathcal{G}$-space (abbreviated as $\mathcal{G}$-SGD) to optimize the value vector of the basis paths of neural networks with little extra cost by leveraging back-propagation. Our experiments show that $\mathcal{G}$-SGD significantly outperforms the conventional SGD algorithm in optimizing ReLU networks on benchmark datasets.
Tasks
Published	2018-02-11
URL	http://arxiv.org/abs/1802.03713v7
PDF	http://arxiv.org/pdf/1802.03713v7.pdf
PWC	https://paperswithcode.com/paper/mathcalg-sgd-optimizing-relu-neural-networks
Repo
Framework

Theoretical Guarantees of Deep Embedding Losses Under Label Noise


Title	Theoretical Guarantees of Deep Embedding Losses Under Label Noise
Authors	Nam Le, Jean-Marc Odobez
Abstract	Collecting labeled data to train deep neural networks is costly and even impractical for many tasks. Thus, research effort has been focused in automatically curated datasets or unsupervised and weakly supervised learning. The common problem in these directions is learning with unreliable label information. In this paper, we address the tolerance of deep embedding learning losses against label noise, i.e. when the observed labels are different from the true labels. Specifically, we provide the sufficient conditions to achieve theoretical guarantees for the 2 common loss functions: marginal loss and triplet loss. From these theoretical results, we can estimate how sampling strategies and initialization can affect the level of resistance against label noise. The analysis also helps providing more effective guidelines in unsupervised and weakly supervised deep embedding learning.
Tasks
Published	2018-12-06
URL	http://arxiv.org/abs/1812.02676v2
PDF	http://arxiv.org/pdf/1812.02676v2.pdf
PWC	https://paperswithcode.com/paper/theoretical-guarantees-of-deep-embedding
Repo
Framework

Deceiving End-to-End Deep Learning Malware Detectors using Adversarial Examples


Title	Deceiving End-to-End Deep Learning Malware Detectors using Adversarial Examples
Authors	Felix Kreuk, Assi Barak, Shir Aviv-Reuven, Moran Baruch, Benny Pinkas, Joseph Keshet
Abstract	In recent years, deep learning has shown performance breakthroughs in many applications, such as image detection, image segmentation, pose estimation, and speech recognition. However, this comes with a major concern: deep networks have been found to be vulnerable to adversarial examples. Adversarial examples are slightly modified inputs that are intentionally designed to cause a misclassification by the model. In the domains of images and speech, the modifications are so small that they are not seen or heard by humans, but nevertheless greatly affect the classification of the model. Deep learning models have been successfully applied to malware detection. In this domain, generating adversarial examples is not straightforward, as small modifications to the bytes of the file could lead to significant changes in its functionality and validity. We introduce a novel loss function for generating adversarial examples specifically tailored for discrete input sets, such as executable bytes. We modify malicious binaries so that they would be detected as benign, while preserving their original functionality, by injecting a small sequence of bytes (payload) in the binary file. We applied this approach to an end-to-end convolutional deep learning malware detection model and show a high rate of detection evasion. Moreover, we show that our generated payload is robust enough to be transferable within different locations of the same file and across different files, and that its entropy is low and similar to that of benign data sections.
Tasks	Malware Detection, Pose Estimation, Semantic Segmentation, Speech Recognition
Published	2018-02-13
URL	http://arxiv.org/abs/1802.04528v3
PDF	http://arxiv.org/pdf/1802.04528v3.pdf
PWC	https://paperswithcode.com/paper/deceiving-end-to-end-deep-learning-malware
Repo
Framework

Learning distributions of shape trajectories from longitudinal datasets: a hierarchical model on a manifold of diffeomorphisms


Title	Learning distributions of shape trajectories from longitudinal datasets: a hierarchical model on a manifold of diffeomorphisms
Authors	Alexandre Bône, Olivier Colliot, Stanley Durrleman
Abstract	We propose a method to learn a distribution of shape trajectories from longitudinal data, i.e. the collection of individual objects repeatedly observed at multiple time-points. The method allows to compute an average spatiotemporal trajectory of shape changes at the group level, and the individual variations of this trajectory both in terms of geometry and time dynamics. First, we formulate a non-linear mixed-effects statistical model as the combination of a generic statistical model for manifold-valued longitudinal data, a deformation model defining shape trajectories via the action of a finite-dimensional set of diffeomorphisms with a manifold structure, and an efficient numerical scheme to compute parallel transport on this manifold. Second, we introduce a MCMC-SAEM algorithm with a specific approach to shape sampling, an adaptive scheme for proposal variances, and a log-likelihood tempering strategy to estimate our model. Third, we validate our algorithm on 2D simulated data, and then estimate a scenario of alteration of the shape of the hippocampus 3D brain structure during the course of Alzheimer’s disease. The method shows for instance that hippocampal atrophy progresses more quickly in female subjects, and occurs earlier in APOE4 mutation carriers. We finally illustrate the potential of our method for classifying pathological trajectories versus normal ageing.
Tasks
Published	2018-03-27
URL	http://arxiv.org/abs/1803.10119v2
PDF	http://arxiv.org/pdf/1803.10119v2.pdf
PWC	https://paperswithcode.com/paper/learning-distributions-of-shape-trajectories
Repo
Framework

HeNet: A Deep Learning Approach on Intel$^\circledR$ Processor Trace for Effective Exploit Detection


Title	HeNet: A Deep Learning Approach on Intel$^\circledR$ Processor Trace for Effective Exploit Detection
Authors	Li Chen, Salmin Sultana, Ravi Sahita
Abstract	This paper presents HeNet, a hierarchical ensemble neural network, applied to classify hardware-generated control flow traces for malware detection. Deep learning-based malware detection has so far focused on analyzing executable files and runtime API calls. Static code analysis approaches face challenges due to obfuscated code and adversarial perturbations. Behavioral data collected during execution is more difficult to obfuscate but recent research has shown successful attacks against API call based malware classifiers. We investigate control flow based characterization of a program execution to build robust deep learning malware classifiers. HeNet consists of a low-level behavior model and a top-level ensemble model. The low-level model is a per-application behavior model, trained via transfer learning on a time-series of images generated from control flow trace of an execution. We use Intel$^\circledR$ Processor Trace enabled processor for low overhead execution tracing and design a lightweight image conversion and segmentation of the control flow trace. The top-level ensemble model aggregates the behavior classification of all the trace segments and detects an attack. The use of hardware trace adds portability to our system and the use of deep learning eliminates the manual effort of feature engineering. We evaluate HeNet against real-world exploitations of PDF readers. HeNet achieves 100% accuracy and 0% false positive on test set, and higher classification accuracy compared to classical machine learning algorithms.
Tasks	Feature Engineering, Malware Detection, Time Series, Transfer Learning
Published	2018-01-08
URL	http://arxiv.org/abs/1801.02318v1
PDF	http://arxiv.org/pdf/1801.02318v1.pdf
PWC	https://paperswithcode.com/paper/henet-a-deep-learning-approach-on
Repo
Framework

RF-PUF: IoT Security Enhancement through Authentication of Wireless Nodes using In-situ Machine Learning


Title	RF-PUF: IoT Security Enhancement through Authentication of Wireless Nodes using In-situ Machine Learning
Authors	Baibhab Chatterjee, Debayan Das, Shreyas Sen
Abstract	Physical unclonable functions (PUF) in silicon exploit die-to-die manufacturing variations during fabrication for uniquely identifying each die. Since it is practically a hard problem to recreate exact silicon features across dies, a PUFbased authentication system is robust, secure and cost-effective, as long as bias removal and error correction are taken into account. In this work, we utilize the effects of inherent process variation on analog and radio-frequency (RF) properties of multiple wireless transmitters (Tx) in a sensor network, and detect the features at the receiver (Rx) using a deep neural network based framework. The proposed mechanism/framework, called RF-PUF, harnesses already existing RF communication hardware and does not require any additional PUF-generation circuitry in the Tx for practical implementation. Simulation results indicate that the RF-PUF framework can distinguish up to 10000 transmitters (with standard foundry defined variations for a 65 nm process, leading to non-idealities such as LO offset and I-Q imbalance) under varying channel conditions, with a probability of false detection < 10e-3
Tasks
Published	2018-05-02
URL	http://arxiv.org/abs/1805.01048v1
PDF	http://arxiv.org/pdf/1805.01048v1.pdf
PWC	https://paperswithcode.com/paper/rf-puf-iot-security-enhancement-through
Repo
Framework

Transfer Learning in Multilingual Neural Machine Translation with Dynamic Vocabulary


Title	Transfer Learning in Multilingual Neural Machine Translation with Dynamic Vocabulary
Authors	Surafel M. Lakew, Aliia Erofeeva, Matteo Negri, Marcello Federico, Marco Turchi
Abstract	We propose a method to transfer knowledge across neural machine translation (NMT) models by means of a shared dynamic vocabulary. Our approach allows to extend an initial model for a given language pair to cover new languages by adapting its vocabulary as long as new data become available (i.e., introducing new vocabulary items if they are not included in the initial model). The parameter transfer mechanism is evaluated in two scenarios: i) to adapt a trained single language NMT system to work with a new language pair and ii) to continuously add new language pairs to grow to a multilingual NMT system. In both the scenarios our goal is to improve the translation performance, while minimizing the training convergence time. Preliminary experiments spanning five languages with different training data sizes (i.e., 5k and 50k parallel sentences) show a significant performance gain ranging from +3.85 up to +13.63 BLEU in different language directions. Moreover, when compared with training an NMT model from scratch, our transfer-learning approach allows us to reach higher performance after training up to 4% of the total training steps.
Tasks	Machine Translation, Transfer Learning
Published	2018-11-03
URL	http://arxiv.org/abs/1811.01137v1
PDF	http://arxiv.org/pdf/1811.01137v1.pdf
PWC	https://paperswithcode.com/paper/transfer-learning-in-multilingual-neural
Repo
Framework

Robust Face Recognition with Deeply Normalized Depth Images


Title	Robust Face Recognition with Deeply Normalized Depth Images
Authors	Ziqing Feng, Qijun Zhao
Abstract	Depth information has been proven useful for face recognition. However, existing depth-image-based face recognition methods still suffer from noisy depth values and varying poses and expressions. In this paper, we propose a novel method for normalizing facial depth images to frontal pose and neutral expression and extracting robust features from the normalized depth images. The method is implemented via two deep convolutional neural networks (DCNN), normalization network ($Net_{N}$) and feature extraction network ($Net_{F}$). Given a facial depth image, $Net_{N}$ first converts it to an HHA image, from which the 3D face is reconstructed via a DCNN. $Net_{N}$ then generates a pose-and-expression normalized (PEN) depth image from the reconstructed 3D face. The PEN depth image is finally passed to $Net_{F}$, which extracts a robust feature representation via another DCNN for face recognition. Our preliminary evaluation results demonstrate the superiority of the proposed method in recognizing faces of arbitrary poses and expressions with depth images.
Tasks	Face Recognition, Robust Face Recognition
Published	2018-05-01
URL	http://arxiv.org/abs/1805.00406v1
PDF	http://arxiv.org/pdf/1805.00406v1.pdf
PWC	https://paperswithcode.com/paper/robust-face-recognition-with-deeply
Repo
Framework

Privacy and Utility Tradeoff in Approximate Differential Privacy


Title	Privacy and Utility Tradeoff in Approximate Differential Privacy
Authors	Quan Geng, Wei Ding, Ruiqi Guo, Sanjiv Kumar
Abstract	We characterize the minimum noise amplitude and power for noise-adding mechanisms in $(\epsilon, \delta)$-differential privacy for single real-valued query function. We derive new lower bounds using the duality of linear programming, and new upper bounds by proposing a new class of $(\epsilon,\delta)$-differentially private mechanisms, the \emph{truncated Laplacian} mechanisms. We show that the multiplicative gap of the lower bounds and upper bounds goes to zero in various high privacy regimes, proving the tightness of the lower and upper bounds and thus establishing the optimality of the truncated Laplacian mechanism. In particular, our results close the previous constant multiplicative gap in the discrete setting. Numeric experiments show the improvement of the truncated Laplacian mechanism over the optimal Gaussian mechanism in all privacy regimes.
Tasks
Published	2018-10-01
URL	https://arxiv.org/abs/1810.00877v2
PDF	https://arxiv.org/pdf/1810.00877v2.pdf
PWC	https://paperswithcode.com/paper/privacy-and-utility-tradeoff-in-approximate
Repo
Framework

Urban Driving with Multi-Objective Deep Reinforcement Learning


Title	Urban Driving with Multi-Objective Deep Reinforcement Learning
Authors	Changjian Li, Krzysztof Czarnecki
Abstract	Autonomous driving is a challenging domain that entails multiple aspects: a vehicle should be able to drive to its destination as fast as possible while avoiding collision, obeying traffic rules and ensuring the comfort of passengers. In this paper, we present a deep learning variant of thresholded lexicographic Q-learning for the task of urban driving. Our multi-objective DQN agent learns to drive on multi-lane roads and intersections, yielding and changing lanes according to traffic rules. We also propose an extension for factored Markov Decision Processes to the DQN architecture that provides auxiliary features for the Q function. This is shown to significantly improve data efficiency. We then show that the learned policy is able to zero-shot transfer to a ring road without sacrificing performance.
Tasks	Autonomous Driving, Q-Learning
Published	2018-11-21
URL	http://arxiv.org/abs/1811.08586v2
PDF	http://arxiv.org/pdf/1811.08586v2.pdf
PWC	https://paperswithcode.com/paper/urban-driving-with-multi-objective-deep
Repo
Framework

Improving Myocardium Segmentation in Cardiac CT Angiography using Spectral Information


Title	Improving Myocardium Segmentation in Cardiac CT Angiography using Spectral Information
Authors	Steffen Bruns, Jelmer M. Wolterink, Robbert W. van Hamersvelt, Majd Zreik, Tim Leiner, Ivana Išgum
Abstract	Accurate segmentation of the left ventricle myocardium in cardiac CT angiography (CCTA) is essential for e.g. the assessment of myocardial perfusion. Automatic deep learning methods for segmentation in CCTA might suffer from differences in contrast-agent attenuation between training and test data due to non-standardized contrast administration protocols and varying cardiac output. We propose augmentation of the training data with virtual mono-energetic reconstructions from a spectral CT scanner which show different attenuation levels of the contrast agent. We compare this to an augmentation by linear scaling of all intensity values, and combine both types of augmentation. We train a 3D fully convolutional network (FCN) with 10 conventional CCTA images and corresponding virtual mono-energetic reconstructions acquired on a spectral CT scanner, and evaluate on 40 CCTA scans acquired on a conventional CT scanner. We show that training with data augmentation using virtual mono-energetic images improves upon training with only conventional images (Dice similarity coefficient (DSC) 0.895 $\pm$ 0.039 vs. 0.846 $\pm$ 0.125). In comparison, training with data augmentation using linear scaling improves the DSC to 0.890 $\pm$ 0.039. Moreover, combining the results of both augmentation methods leads to a DSC of 0.901 $\pm$ 0.036, showing that both augmentations lead to different local improvements of the segmentations. Our results indicate that virtual mono-energetic images improve the generalization of an FCN used for myocardium segmentation in CCTA images.
Tasks	Data Augmentation
Published	2018-09-27
URL	http://arxiv.org/abs/1810.03968v2
PDF	http://arxiv.org/pdf/1810.03968v2.pdf
PWC	https://paperswithcode.com/paper/improving-myocardium-segmentation-in-cardiac
Repo
Framework

Function Estimation Using Data Adaptive Kernel Estimation - How Much Smoothing?


Title	Function Estimation Using Data Adaptive Kernel Estimation - How Much Smoothing?
Authors	Kurt S. Riedel, A. Sidorenko
Abstract	We determine the expected error by smoothing the data locally. Then we optimize the shape of the kernel smoother to minimize the error. Because the optimal estimator depends on the unknown function, our scheme automatically adjusts to the unknown function. By self-consistently adjusting the kernel smoother, the total estimator adapts to the data. Goodness of fit estimators select a kernel halfwidth by minimizing a function of the halfwidth which is based on the average square residual fit error: $ASR(h)$. A penalty term is included to adjust for using the same data to estimate the function and to evaluate the mean square error. Goodness of fit estimators are relatively simple to implement, but the minimum (of the goodness of fit functional) tends to be sensitive to small perturbations. To remedy this sensitivity problem, we fit the mean square error %goodness of fit functional to a two parameter model prior to determining the optimal halfwidth. Plug-in derivative estimators estimate the second derivative of the unknown function in an initial step, and then substitute this estimate into the asymptotic formula.
Tasks
Published	2018-03-11
URL	https://arxiv.org/abs/1803.03999v1
PDF	https://arxiv.org/pdf/1803.03999v1.pdf
PWC	https://paperswithcode.com/paper/function-estimation-using-data-adaptive
Repo
Framework

VR IQA NET: Deep Virtual Reality Image Quality Assessment using Adversarial Learning


Title	VR IQA NET: Deep Virtual Reality Image Quality Assessment using Adversarial Learning
Authors	Heoun-taek Lim, Hak Gu Kim, Yong Man Ro
Abstract	In this paper, we propose a novel virtual reality image quality assessment (VR IQA) with adversarial learning for omnidirectional images. To take into account the characteristics of the omnidirectional image, we devise deep networks including novel quality score predictor and human perception guider. The proposed quality score predictor automatically predicts the quality score of distorted image using the latent spatial and position feature. The proposed human perception guider criticizes the predicted quality score of the predictor with the human perceptual score using adversarial learning. For evaluation, we conducted extensive subjective experiments with omnidirectional image dataset. Experimental results show that the proposed VR IQA metric outperforms the 2-D IQA and the state-of-the-arts VR IQA.
Tasks	Image Quality Assessment
Published	2018-04-11
URL	http://arxiv.org/abs/1804.03943v1
PDF	http://arxiv.org/pdf/1804.03943v1.pdf
PWC	https://paperswithcode.com/paper/vr-iqa-net-deep-virtual-reality-image-quality
Repo
Framework