April 2, 2020

# Paper Group ANR 265

Time Series Data Augmentation for Deep Learning: A Survey. Obstacle Tower Without Human Demonstrations: How Far a Deep Feed-Forward Network Goes with Reinforcement Learning. Assisted Learning and Imitation Privacy. LGVTON: A Landmark Guided Approach to Virtual Try-On. Statistical Queries and Statistical Algorithms: Foundations and Applications. Cor …

#### Time Series Data Augmentation for Deep Learning: A Survey

Title Time Series Data Augmentation for Deep Learning: A Survey
Authors Qingsong Wen, Liang Sun, Xiaomin Song, Jingkun Gao, Xue Wang, Huan Xu
Abstract Deep learning performs remarkably well on many time series analysis tasks recently. The superior performance of deep neural networks relies heavily on a large number of training data to avoid overfitting. However, the labeled data of many real-world time series applications may be limited such as classification in medical time series and anomaly detection in AIOps. As an effective way to enhance the size and quality of the training data, data augmentation is crucial to the successful application of deep learning models on time series data. In this paper, we systematically review different data augmentation methods for time series. We propose a taxonomy for the reviewed methods, and then provide a structured review for these methods by highlighting their strengths and limitations. We also empirically compare different data augmentation methods for different tasks including time series anomaly detection, classification and forecasting. Finally, we discuss and highlight future research directions, including data augmentation in time-frequency domain, augmentation combination, and data augmentation and weighting for imbalanced class.
Tasks Anomaly Detection, Data Augmentation, Time Series, Time Series Analysis
Published 2020-02-27
URL https://arxiv.org/abs/2002.12478v1
PDF https://arxiv.org/pdf/2002.12478v1.pdf
PWC https://paperswithcode.com/paper/time-series-data-augmentation-for-deep
Repo
Framework

#### Obstacle Tower Without Human Demonstrations: How Far a Deep Feed-Forward Network Goes with Reinforcement Learning

Title Obstacle Tower Without Human Demonstrations: How Far a Deep Feed-Forward Network Goes with Reinforcement Learning
Authors Marco Pleines, Jenia Jitsev, Mike Preuss, Frank Zimmer
Abstract The Obstacle Tower Challenge is the task to master a procedurally generated chain of levels that subsequently get harder to complete. Whereas the top 6 performing entries of last year’s competition all used human demonstrations to learn how to cope with the challenge, we present an approach that performed competitively (placed 7th) but starts completely from scratch by means of Deep Reinforcement Learning with a relatively simple feed-forward deep network structure. We especially look at the generalization performance of the taken approach concerning different seeds and various visual themes that have become available after the competition, and investigate where the agent fails and why. Note that our approach does not possess a short-term memory like employing recurrent hidden states. With this work, we hope to contribute to a better understanding of what is possible with a relatively simple, flexible solution that can be applied to learning in environments featuring complex 3D visual input where the abstract task structure itself is still fairly simple.
Published 2020-04-01
URL https://arxiv.org/abs/2004.00567v1
PDF https://arxiv.org/pdf/2004.00567v1.pdf
PWC https://paperswithcode.com/paper/obstacle-tower-without-human-demonstrations
Repo
Framework

#### Assisted Learning and Imitation Privacy

Title Assisted Learning and Imitation Privacy
Authors Xun Xian, Xinran Wang, Jie Ding, Reza Ghanadan
Abstract Motivated by the emerging needs of decentralized learners with personalized learning objectives, we present an Assisted Learning framework where a service provider Bob assists a learner Alice with supervised learning tasks without transmitting Bob’s private algorithm or data. Bob assists Alice either by building a predictive model using Alice’s labels, or by improving Alice’s private learning through iterative communications where only relevant statistics are transmitted. The proposed learning framework is naturally suitable for distributed, personalized, and privacy-aware scenarios. For example, it is shown in some scenarios that two suboptimal learners could achieve much better performance through Assisted Learning. Moreover, motivated by privacy concerns in Assisted Learning, we present a new notion of privacy to quantify the privacy leakage at learning level instead of data level. This new privacy, named imitation privacy, is particularly suitable for a market of statistical learners each holding private learning algorithms as well as data.
Published 2020-04-01
URL https://arxiv.org/abs/2004.00566v1
PDF https://arxiv.org/pdf/2004.00566v1.pdf
PWC https://paperswithcode.com/paper/assisted-learning-and-imitation-privacy
Repo
Framework

#### LGVTON: A Landmark Guided Approach to Virtual Try-On

Title LGVTON: A Landmark Guided Approach to Virtual Try-On
Authors Debapriya Roy, Sanchayan Santra, Bhabatosh Chanda
Abstract We address the problem of image based virtual try-on (VTON), where the goal is to synthesize an image of a person wearing the cloth of a model. An essential requirement for generating a perceptually convincing VTON result is preserving the characteristics of the cloth and the person. Keeping this in mind we propose \textit{LGVTON}, a novel self-supervised landmark guided approach to image based virtual try-on. The incorporation of self-supervision tackles the problem of lack of paired training data in model to person VTON scenario. LGVTON uses two types of landmarks to warp the model cloth according to the shape and pose of the person, one, human landmarks, the locations of anatomical keypoints of human, two, fashion landmarks, the structural keypoints of cloth. We introduce an unique way of using landmarks for warping which is more efficient and effective compared to existing warping based methods in current problem scenario. In addition to that, to make the method robust in cases of noisy landmark estimates that causes inaccurate warping, we propose a mask generator module that attempts to predict the true segmentation mask of the model cloth on the person, which in turn guides our image synthesizer module in tackling warping issues. Experimental results show the effectiveness of our method in comparison to the state-of-the-art VTON methods.
Published 2020-04-01
URL https://arxiv.org/abs/2004.00562v1
PDF https://arxiv.org/pdf/2004.00562v1.pdf
PWC https://paperswithcode.com/paper/lgvton-a-landmark-guided-approach-to-virtual
Repo
Framework

#### Statistical Queries and Statistical Algorithms: Foundations and Applications

Title Statistical Queries and Statistical Algorithms: Foundations and Applications
Authors Lev Reyzin
Abstract We give a survey of the foundations of statistical queries and their many applications to other areas. We introduce the model, give the main definitions, and we explore the fundamental theory statistical queries and how how it connects to various notions of learnability. We also give a detailed summary of some of the applications of statistical queries to other areas, including to optimization, to evolvability, and to differential privacy.
Published 2020-04-01
URL https://arxiv.org/abs/2004.00557v1
PDF https://arxiv.org/pdf/2004.00557v1.pdf
PWC https://paperswithcode.com/paper/statistical-queries-and-statistical
Repo
Framework

#### Coronavirus Covid-19 spreading in Italy: optimizing an epidemiological model with dynamic social distancing through Differential Evolution

Title Coronavirus Covid-19 spreading in Italy: optimizing an epidemiological model with dynamic social distancing through Differential Evolution
Authors I. De Falco, A. Della Cioppa, U. Scafuri, E. Tarantino
Abstract The aim of this paper consists in the application of a recent epidemiological model, namely SEIR with Social Distancing (SEIR–SD), extended here through the definition of a social distancing function varying over time, to assess the situation related to the spreading of the coronavirus Covid–19 in Italy and in two of its most important regions, i.e., Lombardy and Campania. To profitably use this model, the most suitable values of its parameters must be found. The estimation of the SEIR–SD model parameters takes place here through the use of Differential Evolution, a heuristic optimization technique. In this way, we are able to evaluate for each of the three above-mentioned scenarios the daily number of infectious cases from today until the end of virus spreading, the day(s) in which this number will be at its highest peak, and the day in which the infected cases will become very close to zero.
Published 2020-04-01
URL https://arxiv.org/abs/2004.00553v1
PDF https://arxiv.org/pdf/2004.00553v1.pdf
Repo
Framework

#### Development of swarm behavior in artificial learning agents that adapt to different foraging environments

Title Development of swarm behavior in artificial learning agents that adapt to different foraging environments
Authors Andrea López-Incera, Katja Ried, Thomas Müller, Hans J. Briegel
Abstract Collective behavior, and swarm formation in particular, has been studied from several perspectives within a large variety of fields, ranging from biology to physics. In this work, we apply Projective Simulation to model each individual as an artificial learning agent that interacts with its neighbors and surroundings in order to make decisions and learn from them. Within a reinforcement learning framework, we discuss one-dimensional learning scenarios where agents need to get to food resources to be rewarded. We observe how different types of collective motion emerge depending on the distance the agents need to travel to reach the resources. For instance, strongly aligned swarms emerge when the food source is placed far away from the region where agents are situated initially. In addition, we study the properties of the individual trajectories that occur within the different types of emergent collective dynamics. Agents trained to find distant resources exhibit individual trajectories with L'evy-like characteristics as a consequence of the collective motion, whereas agents trained to reach nearby resources present Brownian-like trajectories.
Published 2020-04-01
URL https://arxiv.org/abs/2004.00552v1
PDF https://arxiv.org/pdf/2004.00552v1.pdf
PWC https://paperswithcode.com/paper/development-of-swarm-behavior-in-artificial
Repo
Framework

#### Physically Realizable Adversarial Examples for LiDAR Object Detection

Title Physically Realizable Adversarial Examples for LiDAR Object Detection
Authors James Tu, Mengye Ren, Siva Manivasagam, Ming Liang, Bin Yang, Richard Du, Frank Cheng, Raquel Urtasun
Abstract Modern autonomous driving systems rely heavily on deep learning models to process point cloud sensory data; meanwhile, deep models have been shown to be susceptible to adversarial attacks with visually imperceptible perturbations. Despite the fact that this poses a security concern for the self-driving industry, there has been very little exploration in terms of 3D perception, as most adversarial attacks have only been applied to 2D flat images. In this paper, we address this issue and present a method to generate universal 3D adversarial objects to fool LiDAR detectors. In particular, we demonstrate that placing an adversarial object on the rooftop of any target vehicle to hide the vehicle entirely from LiDAR detectors with a success rate of 80%. We report attack results on a suite of detectors using various input representation of point clouds. We also conduct a pilot study on adversarial defense using data augmentation. This is one step closer towards safer self-driving under unseen conditions from limited training data.
Published 2020-04-01
URL https://arxiv.org/abs/2004.00543v1
PDF https://arxiv.org/pdf/2004.00543v1.pdf
Repo
Framework

#### Generation of Paths in a Maze using a Deep Network without Learning

Title Generation of Paths in a Maze using a Deep Network without Learning
Authors Tomas Kulvicius, Sebastian Herzog, Minija Tamosiunaite, Florentin Wörgötter
Abstract Trajectory- or path-planning is a fundamental issue in a wide variety of applications. Here we show that it is possible to solve path planning for multiple start- and end-points highly efficiently with a network that consists only of max pooling layers, for which no network training is needed. Different from competing approaches, very large mazes containing more than half a billion nodes with dense obstacle configuration and several thousand path end-points can this way be solved in very short time on parallel hardware.
Published 2020-04-01
URL https://arxiv.org/abs/2004.00540v1
PDF https://arxiv.org/pdf/2004.00540v1.pdf
PWC https://paperswithcode.com/paper/generation-of-paths-in-a-maze-using-a-deep
Repo
Framework

#### Learning Sparse Rewarded Tasks from Sub-Optimal Demonstrations

Title Learning Sparse Rewarded Tasks from Sub-Optimal Demonstrations
Authors Zhuangdi Zhu, Kaixiang Lin, Bo Dai, Jiayu Zhou
Abstract Model-free deep reinforcement learning (RL) has demonstrated its superiority on many complex sequential decision-making problems. However, heavy dependence on dense rewards and high sample-complexity impedes the wide adoption of these methods in real-world scenarios. On the other hand, imitation learning (IL) learns effectively in sparse-rewarded tasks by leveraging the existing expert demonstrations. In practice, collecting a sufficient amount of expert demonstrations can be prohibitively expensive, and the quality of demonstrations typically limits the performance of the learning policy. In this work, we propose Self-Adaptive Imitation Learning (SAIL) that can achieve (near) optimal performance given only a limited number of sub-optimal demonstrations for highly challenging sparse reward tasks. SAIL bridges the advantages of IL and RL to reduce the sample complexity substantially, by effectively exploiting sup-optimal demonstrations and efficiently exploring the environment to surpass the demonstrated performance. Extensive empirical results show that not only does SAIL significantly improve the sample-efficiency but also leads to much better final performance across different continuous control tasks, comparing to the state-of-the-art.
Tasks Continuous Control, Decision Making, Imitation Learning
Published 2020-04-01
URL https://arxiv.org/abs/2004.00530v1
PDF https://arxiv.org/pdf/2004.00530v1.pdf
Repo
Framework

#### Statistical Optimal Transport posed as Learning Kernel Embedding

Title Statistical Optimal Transport posed as Learning Kernel Embedding
Authors J. Saketha Nath, Pratik Jawanpuria
Abstract This work takes the novel approach of posing the statistical Optimal Transport (OT) problem as that of learning the transport plan’s kernel mean embedding. The key advantage is that the estimates for the embeddings of the marginals can now be employed directly, leading to a dimension-free sample complexity for the proposed transport plan and transport map estimators. Also, because of the implicit smoothing in the kernel embeddings, the proposed estimators can perform out-of-sample estimation. Interestingly, the proposed formulation employs an MMD based regularization to avoid overfitting, which is complementary to existing $\phi$-divergence (entropy) based regularization techniques. An appropriate representer theorem is presented that leads to a fully kernelized formulation and hence the same formulation can be used to perform continuous/semi-discrete/discrete OT in any non-standard domain (as long as universal kernels in those domains are known). Finally, an ADMM based algorithm is presented for solving the kernelized formulation efficiently. Empirical results show that the proposed estimator outperforms discrete OT based estimator in terms of transport map accuracy.
Published 2020-02-08
URL https://arxiv.org/abs/2002.03179v2
PDF https://arxiv.org/pdf/2002.03179v2.pdf
PWC https://paperswithcode.com/paper/statistical-optimal-transport-posed-as
Repo
Framework

#### On a scalable entropic breaching of the overfitting barrier in machine learning

Title On a scalable entropic breaching of the overfitting barrier in machine learning
Authors Illia Horenko
Abstract Overfitting and treatment of “small data” are among the most challenging problems in the machine learning (ML), when a relatively small data statistics size $T$ is not enough to provide a robust ML fit for a relatively large data feature dimension $D$. Deploying a massively-parallel ML analysis of generic classification problems for different $D$ and $T$, existence of statistically-significant linear overfitting barriers for common ML methods is demonstrated. For example, these results reveal that for a robust classification of bioinformatics-motivated generic problems with the Long Short-Term Memory deep learning classifier (LSTM) one needs in a best case a statistics $T$ that is at least 13.8 times larger then the feature dimension $D$. It is shown that this overfitting barrier can be breached at a $10^{-12}$ fraction of the computational cost by means of the entropy-optimal Scalable Probabilistic Approximations algorithm (eSPA), performing a joint solution of the entropy-optimal Bayesian network inference and feature space segmentation problems. Application of eSPA to experimental single cell RNA sequencing data exhibits a 30-fold classification performance boost when compared to standard bioinformatics tools - and a 7-fold boost when compared to the deep learning LSTM classifier.
Published 2020-02-08
URL https://arxiv.org/abs/2002.03176v1
PDF https://arxiv.org/pdf/2002.03176v1.pdf
PWC https://paperswithcode.com/paper/on-a-scalable-entropic-breaching-of-the
Repo
Framework

#### Cross-Shape Graph Convolutional Networks

Title Cross-Shape Graph Convolutional Networks
Authors Dmitry Petrov, Evangelos Kalogerakis
Abstract We present a method that processes 3D point clouds by performing graph convolution operations across shapes. In this manner, point descriptors are learned by allowing interaction and propagation of feature representations within a shape collection. To enable this form of non-local, cross-shape graph convolution, our method learns a pairwise point attention mechanism indicating the degree of interaction between points on different shapes. Our method also learns to create a graph over shapes of an input collection whose edges connect shapes deemed as useful for performing cross-shape convolution. The edges are also equipped with learned weights indicating the compatibility of each shape pair for cross-shape convolution. Our experiments demonstrate that this interaction and propagation of point representations across shapes make them more discriminative. In particular, our results show significantly improved performance for 3D point cloud semantic segmentation compared to conventional approaches, especially in cases with the limited number of training examples.
Published 2020-03-20
URL https://arxiv.org/abs/2003.09053v2
PDF https://arxiv.org/pdf/2003.09053v2.pdf
PWC https://paperswithcode.com/paper/cross-shape-graph-convolutional-networks
Repo
Framework

#### Improved RawNet with Filter-wise Rescaling for Text-independent Speaker Verification using Raw Waveforms

Title Improved RawNet with Filter-wise Rescaling for Text-independent Speaker Verification using Raw Waveforms
Authors Jee-weon Jung, Seung-bin Kim, Hye-jin Shim, Ju-ho Kim, Ha-Jin Yu
Abstract Recent advances in deep learning have facilitated the design of speaker verification systems that directly input raw waveforms. For example, RawNet extracts speaker embeddings from raw waveforms, which simplifies the process pipeline and demonstrates competitive performance. In this study, we improve RawNet by rescaling feature maps using various methods. The proposed mechanism utilizes a filter-wise rescale map that adopts a sigmoid non-linear function. It refers to a vector with dimensionality equal to the number of filters in a given feature map. Using a filter-wise rescale map, we propose to rescale the feature map multiplicatively, additively, or both. In addition, we investigate replacing the first convolution layer with the sinc-convolution layer of SincNet. Experiments performed on the VoxCeleb1 evaluation dataset demonstrate that the proposed methods are effective, and the best performing system reduces the equal error rate by half compared to the original RawNet. Expanded evaluation results obtained using the VoxCeleb1-E and VoxCeleb-H protocols marginally outperform existing state-of-the-art systems.
Tasks Speaker Verification, Text-Independent Speaker Verification
Published 2020-04-01
URL https://arxiv.org/abs/2004.00526v1
PDF https://arxiv.org/pdf/2004.00526v1.pdf
PWC https://paperswithcode.com/paper/improved-rawnet-with-filter-wise-rescaling
Repo
Framework

#### Stochastic gradient descent with random learning rate

Title Stochastic gradient descent with random learning rate
Authors Daniele Musso
Abstract We propose to optimize neural networks with a uniformly-distributed random learning rate. The associated stochastic gradient descent algorithm can be approximated by continuous stochastic equations and analyzed with the Fokker-Planck formalism. In the small learning rate approximation, the training process is characterized by an effective temperature which depends on the average learning rate, the mini-batch size and the momentum of the optimization algorithm. By comparing the random learning rate protocol with cyclic and constant protocols, we suggest that the random choice is generically the best strategy in the small learning rate regime, yielding better regularization without extra computational cost. We provide supporting evidence through experiments on both shallow, fully-connected and deep, convolutional neural networks for image classification on the MNIST and CIFAR10 datasets.