October 16, 2019

3355 words 16 mins read

Paper Group ANR 1149

Show, Attend and Translate: Unsupervised Image Translation with Self-Regularization and Attention. On the Selection of Initialization and Activation Function for Deep Neural Networks. Modeling Local Dependence in Natural Language with Multi-channel Recurrent Neural Networks. Modified Multidimensional Scaling and High Dimensional Clustering. Feasibi …

Show, Attend and Translate: Unsupervised Image Translation with Self-Regularization and Attention


Title	Show, Attend and Translate: Unsupervised Image Translation with Self-Regularization and Attention
Authors	Chao Yang, Taehwan Kim, Ruizhe Wang, Hao Peng, C. -C. Jay Kuo
Abstract	Image translation between two domains is a class of problems aiming to learn mapping from an input image in the source domain to an output image in the target domain. It has been applied to numerous domains, such as data augmentation, domain adaptation, and unsupervised training. When paired training data is not accessible, image translation becomes an ill-posed problem. We constrain the problem with the assumption that the translated image needs to be perceptually similar to the original image and also appears to be drawn from the new domain, and propose a simple yet effective image translation model consisting of a single generator trained with a self-regularization term and an adversarial term. We further notice that existing image translation techniques are agnostic to the subjects of interest and often introduce unwanted changes or artifacts to the input. Thus we propose to add an attention module to predict an attention map to guide the image translation process. The module learns to attend to key parts of the image while keeping everything else unaltered, essentially avoiding undesired artifacts or changes. The predicted attention map also opens door to applications such as unsupervised segmentation and saliency detection. Extensive experiments and evaluations show that our model while being simpler, achieves significantly better performance than existing image translation methods.
Tasks	Data Augmentation, Domain Adaptation, Saliency Detection
Published	2018-06-16
URL	https://arxiv.org/abs/1806.06195v3
PDF	https://arxiv.org/pdf/1806.06195v3.pdf
PWC	https://paperswithcode.com/paper/show-attend-and-translate-unsupervised-image
Repo
Framework

On the Selection of Initialization and Activation Function for Deep Neural Networks


Title	On the Selection of Initialization and Activation Function for Deep Neural Networks
Authors	Soufiane Hayou, Arnaud Doucet, Judith Rousseau
Abstract	The weight initialization and the activation function of deep neural networks have a crucial impact on the performance of the training procedure. An inappropriate selection can lead to the loss of information of the input during forward propagation and the exponential vanishing/exploding of gradients during back-propagation. Understanding the theoretical properties of untrained random networks is key to identifying which deep networks may be trained successfully as recently demonstrated by Schoenholz et al. (2017) who showed that for deep feedforward neural networks only a specific choice of hyperparameters known as the `edge of chaos’ can lead to good performance. We complete this analysis by providing quantitative results showing that, for a class of ReLU-like activation functions, the information propagates indeed deeper for an initialization at the edge of chaos. By further extending this analysis, we identify a class of activation functions that improve the information propagation over ReLU-like functions. This class includes the Swish activation, $\phi_{swish}(x) = x \cdot \text{sigmoid}(x)$, used in Hendrycks & Gimpel (2016), Elfwing et al. (2017) and Ramachandran et al. (2017). This provides a theoretical grounding for the excellent empirical performance of $\phi_{swish}$ observed in these contributions. We complement those previous results by illustrating the benefit of using a random initialization on the edge of chaos in this context. \|
Tasks
Published	2018-05-21
URL	http://arxiv.org/abs/1805.08266v2
PDF	http://arxiv.org/pdf/1805.08266v2.pdf
PWC	https://paperswithcode.com/paper/on-the-selection-of-initialization-and
Repo
Framework

Modeling Local Dependence in Natural Language with Multi-channel Recurrent Neural Networks


Title	Modeling Local Dependence in Natural Language with Multi-channel Recurrent Neural Networks
Authors	Chang Xu, Weiran Huang, Hongwei Wang, Gang Wang, Tie-Yan Liu
Abstract	Recurrent Neural Networks (RNNs) have been widely used in processing natural language tasks and achieve huge success. Traditional RNNs usually treat each token in a sentence uniformly and equally. However, this may miss the rich semantic structure information of a sentence, which is useful for understanding natural languages. Since semantic structures such as word dependence patterns are not parameterized, it is a challenge to capture and leverage structure information. In this paper, we propose an improved variant of RNN, Multi-Channel RNN (MC-RNN), to dynamically capture and leverage local semantic structure information. Concretely, MC-RNN contains multiple channels, each of which represents a local dependence pattern at a time. An attention mechanism is introduced to combine these patterns at each step, according to the semantic information. Then we parameterize structure information by adaptively selecting the most appropriate connection structures among channels. In this way, diverse local structures and dependence patterns in sentences can be well captured by MC-RNN. To verify the effectiveness of MC-RNN, we conduct extensive experiments on typical natural language processing tasks, including neural machine translation, abstractive summarization, and language modeling. Experimental results on these tasks all show significant improvements of MC-RNN over current top systems.
Tasks	Abstractive Text Summarization, Language Modelling, Machine Translation
Published	2018-11-13
URL	http://arxiv.org/abs/1811.05121v1
PDF	http://arxiv.org/pdf/1811.05121v1.pdf
PWC	https://paperswithcode.com/paper/modeling-local-dependence-in-natural-language
Repo
Framework

Modified Multidimensional Scaling and High Dimensional Clustering


Title	Modified Multidimensional Scaling and High Dimensional Clustering
Authors	Xiucai Ding, Qiang Sun
Abstract	Multidimensional scaling is an important dimension reduction tool in statistics and machine learning. Yet few theoretical results characterizing its statistical performance exist, not to mention any in high dimensions. By considering a unified framework that includes low, moderate and high dimensions, we study multidimensional scaling in the setting of clustering noisy data. Our results suggest that, the classical multidimensional scaling can be modified to further improve the quality of embedded samples, especially when the noise level increases. To this end, we propose {\it modified multidimensional scaling} which applies a nonlinear transformation to the sample eigenvalues. The nonlinear transformation depends on the dimensionality, sample size and moment of noise. We show that modified multidimensional scaling followed by various clustering algorithms can achieve exact recovery, i.e., all the cluster labels can be recovered correctly with probability tending to one. Numerical simulations and two real data applications lend strong support to our proposed methodology.
Tasks	Dimensionality Reduction
Published	2018-10-24
URL	http://arxiv.org/abs/1810.10172v2
PDF	http://arxiv.org/pdf/1810.10172v2.pdf
PWC	https://paperswithcode.com/paper/modified-multidimensional-scaling-and-high
Repo
Framework

Feasibility of Supervised Machine Learning for Cloud Security


Title	Feasibility of Supervised Machine Learning for Cloud Security
Authors	Deval Bhamare, Tara Salman, Mohammed Samaka, Aiman Erbad, Raj Jain
Abstract	Cloud computing is gaining significant attention, however, security is the biggest hurdle in its wide acceptance. Users of cloud services are under constant fear of data loss, security threats and availability issues. Recently, learning-based methods for security applications are gaining popularity in the literature with the advents in machine learning techniques. However, the major challenge in these methods is obtaining real-time and unbiased datasets. Many datasets are internal and cannot be shared due to privacy issues or may lack certain statistical characteristics. As a result of this, researchers prefer to generate datasets for training and testing purpose in the simulated or closed experimental environments which may lack comprehensiveness. Machine learning models trained with such a single dataset generally result in a semantic gap between results and their application. There is a dearth of research work which demonstrates the effectiveness of these models across multiple datasets obtained in different environments. We argue that it is necessary to test the robustness of the machine learning models, especially in diversified operating conditions, which are prevalent in cloud scenarios. In this work, we use the UNSW dataset to train the supervised machine learning models. We then test these models with ISOT dataset. We present our results and argue that more research in the field of machine learning is still required for its applicability to the cloud security.
Tasks
Published	2018-10-23
URL	http://arxiv.org/abs/1810.09878v1
PDF	http://arxiv.org/pdf/1810.09878v1.pdf
PWC	https://paperswithcode.com/paper/feasibility-of-supervised-machine-learning
Repo
Framework

Combined Image- and World-Space Tracking in Traffic Scenes


Title	Combined Image- and World-Space Tracking in Traffic Scenes
Authors	Aljosa Osep, Wolfgang Mehner, Markus Mathias, Bastian Leibe
Abstract	Tracking in urban street scenes plays a central role in autonomous systems such as self-driving cars. Most of the current vision-based tracking methods perform tracking in the image domain. Other approaches, eg based on LIDAR and radar, track purely in 3D. While some vision-based tracking methods invoke 3D information in parts of their pipeline, and some 3D-based methods utilize image-based information in components of their approach, we propose to use image- and world-space information jointly throughout our method. We present our tracking pipeline as a 3D extension of image-based tracking. From enhancing the detections with 3D measurements to the reported positions of every tracked object, we use world-space 3D information at every stage of processing. We accomplish this by our novel coupled 2D-3D Kalman filter, combined with a conceptually clean and extendable hypothesize-and-select framework. Our approach matches the current state-of-the-art on the official KITTI benchmark, which performs evaluation in the 2D image domain only. Further experiments show significant improvements in 3D localization precision by enabling our coupled 2D-3D tracking.
Tasks	Self-Driving Cars
Published	2018-09-19
URL	http://arxiv.org/abs/1809.07357v1
PDF	http://arxiv.org/pdf/1809.07357v1.pdf
PWC	https://paperswithcode.com/paper/combined-image-and-world-space-tracking-in
Repo
Framework

Playing the Game of Universal Adversarial Perturbations


Title	Playing the Game of Universal Adversarial Perturbations
Authors	Julien Perolat, Mateusz Malinowski, Bilal Piot, Olivier Pietquin
Abstract	We study the problem of learning classifiers robust to universal adversarial perturbations. While prior work approaches this problem via robust optimization, adversarial training, or input transformation, we instead phrase it as a two-player zero-sum game. In this new formulation, both players simultaneously play the same game, where one player chooses a classifier that minimizes a classification loss whilst the other player creates an adversarial perturbation that increases the same loss when applied to every sample in the training set. By observing that performing a classification (respectively creating adversarial samples) is the best response to the other player, we propose a novel extension of a game-theoretic algorithm, namely fictitious play, to the domain of training robust classifiers. Finally, we empirically show the robustness and versatility of our approach in two defence scenarios where universal attacks are performed on several image classification datasets – CIFAR10, CIFAR100 and ImageNet.
Tasks	Image Classification
Published	2018-09-20
URL	http://arxiv.org/abs/1809.07802v2
PDF	http://arxiv.org/pdf/1809.07802v2.pdf
PWC	https://paperswithcode.com/paper/playing-the-game-of-universal-adversarial
Repo
Framework

Online Signature Verification using Deep Representation: A new Descriptor


Title	Online Signature Verification using Deep Representation: A new Descriptor
Authors	Mohammad Hajizadeh Saffar, Mohsen Fayyaz, Mohammad Sabokrou, Mahmood Fathy
Abstract	This paper presents an accurate method for verifying online signatures. The main difficulty of signature verification come from: (1) Lacking enough training samples (2) The methods must be spatial change invariant. To deal with these difficulties and modeling the signatures efficiently, we propose a method that a one-class classifier per each user is built on discriminative features. First, we pre-train a sparse auto-encoder using a large number of unlabeled signatures, then we applied the discriminative features, which are learned by auto-encoder to represent the training and testing signatures as a self-thought learning method (i.e. we have introduced a signature descriptor). Finally, user’s signatures are modeled and classified using a one-class classifier. The proposed method is independent on signature datasets thanks to self-taught learning. The experimental results indicate significant error reduction and accuracy enhancement in comparison with state-of-the-art methods on SVC2004 and SUSIG datasets.
Tasks	One-class classifier
Published	2018-06-24
URL	http://arxiv.org/abs/1806.09986v1
PDF	http://arxiv.org/pdf/1806.09986v1.pdf
PWC	https://paperswithcode.com/paper/online-signature-verification-using-deep
Repo
Framework

Ranking Recovery from Limited Comparisons using Low-Rank Matrix Completion


Title	Ranking Recovery from Limited Comparisons using Low-Rank Matrix Completion
Authors	Tal Levy, Alireza Vahid, Raja Giryes
Abstract	This paper proposes a new method for solving the well-known rank aggregation problem from pairwise comparisons using the method of low-rank matrix completion. The partial and noisy data of pairwise comparisons is transformed into a matrix form. We then use tools from matrix completion, which has served as a major component in the low-rank completion solution of the Netflix challenge, to construct the preference of the different objects. In our approach, the data of multiple comparisons is used to create an estimate of the probability of object i to win (or be chosen) over object j, where only a partial set of comparisons between N objects is known. The data is then transformed into a matrix form for which the noiseless solution has a known rank of one. An alternating minimization algorithm, in which the target matrix takes a bilinear form, is then used in combination with maximum likelihood estimation for both factors. The reconstructed matrix is used to obtain the true underlying preference intensity. This work demonstrates the improvement of our proposed algorithm over the current state-of-the-art in both simulated scenarios and real data.
Tasks	Low-Rank Matrix Completion, Matrix Completion
Published	2018-06-14
URL	http://arxiv.org/abs/1806.05419v1
PDF	http://arxiv.org/pdf/1806.05419v1.pdf
PWC	https://paperswithcode.com/paper/ranking-recovery-from-limited-comparisons
Repo
Framework

A Machine Learning Approach to Adaptive Covariance Localization


Title	A Machine Learning Approach to Adaptive Covariance Localization
Authors	Azam Moosavi, Ahmed Attia, Adrian Sandu
Abstract	Data assimilation plays a key role in large-scale atmospheric weather forecasting, where the state of the physical system is estimated from model outputs and observations, and is then used as initial condition to produce accurate future forecasts. The Ensemble Kalman Filter (EnKF) provides a practical implementation of the statistical solution of the data assimilation problem and has gained wide popularity as. This success can be attributed to its simple formulation and ease of implementation. EnKF is a Monte-Carlo algorithm that solves the data assimilation problem by sampling the probability distributions involved in Bayes theorem. Because of this, all flavors of EnKF are fundamentally prone to sampling errors when the ensemble size is small. In typical weather forecasting applications, the model state space has dimension $10^{9}-10^{12}$, while the ensemble size typically ranges between $30-100$ members. Sampling errors manifest themselves as long-range spurious correlations and have been shown to cause filter divergence. To alleviate this effect covariance localization dampens spurious correlations between state variables located at a large distance in the physical space, via an empirical distance-dependent function. The quality of the resulting analysis and forecast is greatly influenced by the choice of the localization function parameters, e.g., the radius of influence. The localization radius is generally tuned empirically to yield desirable results.This work, proposes two adaptive algorithms for covariance localization in the EnKF framework, both based on a machine learning approach. The first algorithm adapts the localization radius in time, while the second algorithm tunes the localization radius in both time and space. Numerical experiments carried out with the Lorenz-96 model, and a quasi-geostrophic model, reveal the potential of the proposed machine learning approaches.
Tasks	Weather Forecasting
Published	2018-01-02
URL	http://arxiv.org/abs/1801.00548v3
PDF	http://arxiv.org/pdf/1801.00548v3.pdf
PWC	https://paperswithcode.com/paper/a-machine-learning-approach-to-adaptive
Repo
Framework

Artificial Intelligence Enabled Software Defined Networking: A Comprehensive Overview


Title	Artificial Intelligence Enabled Software Defined Networking: A Comprehensive Overview
Authors	Majd Latah, Levent Toker
Abstract	Software defined networking (SDN) represents a promising networking architecture that combines central management and network programmability. SDN separates the control plane from the data plane and moves the network management to a central point, called the controller, that can be programmed and used as the brain of the network. Recently, the research community has showed an increased tendency to benefit from the recent advancements in the artificial intelligence (AI) field to provide learning abilities and better decision making in SDN. In this study, we provide a detailed overview of the recent efforts to include AI in SDN. Our study showed that the research efforts focused on three main sub-fields of AI namely: machine learning, meta-heuristics and fuzzy inference systems. Accordingly, in this work we investigate their different application areas and potential use, as well as the improvements achieved by including AI-based techniques in the SDN paradigm.
Tasks	Decision Making
Published	2018-03-19
URL	http://arxiv.org/abs/1803.06818v3
PDF	http://arxiv.org/pdf/1803.06818v3.pdf
PWC	https://paperswithcode.com/paper/artificial-intelligence-enabled-software
Repo
Framework

Evolutionary Self-Expressive Models for Subspace Clustering


Title	Evolutionary Self-Expressive Models for Subspace Clustering
Authors	Abolfazl Hashemi, Haris Vikalo
Abstract	The problem of organizing data that evolves over time into clusters is encountered in a number of practical settings. We introduce evolutionary subspace clustering, a method whose objective is to cluster a collection of evolving data points that lie on a union of low-dimensional evolving subspaces. To learn the parsimonious representation of the data points at each time step, we propose a non-convex optimization framework that exploits the self-expressiveness property of the evolving data while taking into account representation from the preceding time step. To find an approximate solution to the aforementioned non-convex optimization problem, we develop a scheme based on alternating minimization that both learns the parsimonious representation as well as adaptively tunes and infers a smoothing parameter reflective of the rate of data evolution. The latter addresses a fundamental challenge in evolutionary clustering – determining if and to what extent one should consider previous clustering solutions when analyzing an evolving data collection. Our experiments on both synthetic and real-world datasets demonstrate that the proposed framework outperforms state-of-the-art static subspace clustering algorithms and existing evolutionary clustering schemes in terms of both accuracy and running time, in a range of scenarios.
Tasks
Published	2018-10-29
URL	http://arxiv.org/abs/1810.11957v1
PDF	http://arxiv.org/pdf/1810.11957v1.pdf
PWC	https://paperswithcode.com/paper/evolutionary-self-expressive-models-for
Repo
Framework

Improving TSP Solutions Using GA with a New Hybrid Mutation Based on Knowledge and Randomness


Title	Improving TSP Solutions Using GA with a New Hybrid Mutation Based on Knowledge and Randomness
Authors	Esra’a Alkafaween, Ahmad B. A. Hassanat
Abstract	Genetic algorithm (GA) is an efficient tool for solving optimization problems by evolving solutions, as it mimics the Darwinian theory of natural evolution. The mutation operator is one of the key success factors in GA, as it is considered the exploration operator of GA. Various mutation operators exist to solve hard combinatorial problems such as the TSP. In this paper, we propose a hybrid mutation operator called “IRGIBNNM”, this mutation is a combination of two existing mutations, a knowledge-based mutation, and a random-based mutation. We also improve the existing “select best mutation” strategy using the proposed mutation. We conducted several experiments on twelve benchmark Symmetric traveling salesman problem (STSP) instances. The results of our experiments show the efficiency of the proposed mutation, particularly when we use it with some other mutations. Keyword: Knowledge-based mutation, Inversion mutation, Slide mutation, RGIBNNM, SBM.
Tasks
Published	2018-01-22
URL	http://arxiv.org/abs/1801.07233v1
PDF	http://arxiv.org/pdf/1801.07233v1.pdf
PWC	https://paperswithcode.com/paper/improving-tsp-solutions-using-ga-with-a-new
Repo
Framework

Anomaly Detection using Deep Learning based Image Completion


Title	Anomaly Detection using Deep Learning based Image Completion
Authors	Matthias Haselmann, Dieter P. Gruber, Paul Tabatabai
Abstract	Automated surface inspection is an important task in many manufacturing industries and often requires machine learning driven solutions. Supervised approaches, however, can be challenging, since it is often difficult to obtain large amounts of labeled training data. In this work, we instead perform one-class unsupervised learning on fault-free samples by training a deep convolutional neural network to complete images whose center regions are cut out. Since the network is trained exclusively on fault-free data, it completes the image patches with a fault-free version of the missing image region. The pixel-wise reconstruction error within the cut out region is an anomaly image which can be used for anomaly detection. Results on surface images of decorated plastic parts demonstrate that this approach is suitable for detection of visible anomalies and moreover surpasses all other tested methods.
Tasks	Anomaly Detection
Published	2018-11-16
URL	http://arxiv.org/abs/1811.06861v1
PDF	http://arxiv.org/pdf/1811.06861v1.pdf
PWC	https://paperswithcode.com/paper/anomaly-detection-using-deep-learning-based
Repo
Framework

Learning on the Edge: Explicit Boundary Handling in CNNs


Title	Learning on the Edge: Explicit Boundary Handling in CNNs
Authors	Carlo Innamorati, Tobias Ritschel, Tim Weyrich, Niloy J. Mitra
Abstract	Convolutional neural networks (CNNs) handle the case where filters extend beyond the image boundary using several heuristics, such as zero, repeat or mean padding. These schemes are applied in an ad-hoc fashion and, being weakly related to the image content and oblivious of the target task, result in low output quality at the boundary. In this paper, we propose a simple and effective improvement that learns the boundary handling itself. At training-time, the network is provided with a separate set of explicit boundary filters. At testing-time, we use these filters which have learned to extrapolate features at the boundary in an optimal way for the specific task. Our extensive evaluation, over a wide range of architectural changes (variations of layers, feature channels, or both), shows how the explicit filters result in improved boundary handling. Consequently, we demonstrate an improvement of 5% to 20% across the board of typical CNN applications (colorization, de-Bayering, optical flow, and disparity estimation).
Tasks	Colorization, Disparity Estimation, Optical Flow Estimation
Published	2018-05-08
URL	http://arxiv.org/abs/1805.03106v1
PDF	http://arxiv.org/pdf/1805.03106v1.pdf
PWC	https://paperswithcode.com/paper/learning-on-the-edge-explicit-boundary
Repo
Framework