Paper Group ANR 265
Time Series Data Augmentation for Deep Learning: A Survey. Obstacle Tower Without Human Demonstrations: How Far a Deep Feed-Forward Network Goes with Reinforcement Learning. Assisted Learning and Imitation Privacy. LGVTON: A Landmark Guided Approach to Virtual Try-On. Statistical Queries and Statistical Algorithms: Foundations and Applications. Cor …
Time Series Data Augmentation for Deep Learning: A Survey
| Title | Time Series Data Augmentation for Deep Learning: A Survey |
| Authors | Qingsong Wen, Liang Sun, Xiaomin Song, Jingkun Gao, Xue Wang, Huan Xu |
| Abstract | Deep learning performs remarkably well on many time series analysis tasks recently. The superior performance of deep neural networks relies heavily on a large number of training data to avoid overfitting. However, the labeled data of many real-world time series applications may be limited such as classification in medical time series and anomaly detection in AIOps. As an effective way to enhance the size and quality of the training data, data augmentation is crucial to the successful application of deep learning models on time series data. In this paper, we systematically review different data augmentation methods for time series. We propose a taxonomy for the reviewed methods, and then provide a structured review for these methods by highlighting their strengths and limitations. We also empirically compare different data augmentation methods for different tasks including time series anomaly detection, classification and forecasting. Finally, we discuss and highlight future research directions, including data augmentation in time-frequency domain, augmentation combination, and data augmentation and weighting for imbalanced class. |
| Tasks | Anomaly Detection, Data Augmentation, Time Series, Time Series Analysis |
| Published | 2020-02-27 |
| URL | https://arxiv.org/abs/2002.12478v1 |
| https://arxiv.org/pdf/2002.12478v1.pdf | |
| PWC | https://paperswithcode.com/paper/time-series-data-augmentation-for-deep |
| Repo | |
| Framework | |
Obstacle Tower Without Human Demonstrations: How Far a Deep Feed-Forward Network Goes with Reinforcement Learning
| Title | Obstacle Tower Without Human Demonstrations: How Far a Deep Feed-Forward Network Goes with Reinforcement Learning |
| Authors | Marco Pleines, Jenia Jitsev, Mike Preuss, Frank Zimmer |
| Abstract | The Obstacle Tower Challenge is the task to master a procedurally generated chain of levels that subsequently get harder to complete. Whereas the top 6 performing entries of last year’s competition all used human demonstrations to learn how to cope with the challenge, we present an approach that performed competitively (placed 7th) but starts completely from scratch by means of Deep Reinforcement Learning with a relatively simple feed-forward deep network structure. We especially look at the generalization performance of the taken approach concerning different seeds and various visual themes that have become available after the competition, and investigate where the agent fails and why. Note that our approach does not possess a short-term memory like employing recurrent hidden states. With this work, we hope to contribute to a better understanding of what is possible with a relatively simple, flexible solution that can be applied to learning in environments featuring complex 3D visual input where the abstract task structure itself is still fairly simple. |
| Tasks | |
| Published | 2020-04-01 |
| URL | https://arxiv.org/abs/2004.00567v1 |
| https://arxiv.org/pdf/2004.00567v1.pdf | |
| PWC | https://paperswithcode.com/paper/obstacle-tower-without-human-demonstrations |
| Repo | |
| Framework | |
Assisted Learning and Imitation Privacy
| Title | Assisted Learning and Imitation Privacy |
| Authors | Xun Xian, Xinran Wang, Jie Ding, Reza Ghanadan |
| Abstract | Motivated by the emerging needs of decentralized learners with personalized learning objectives, we present an Assisted Learning framework where a service provider Bob assists a learner Alice with supervised learning tasks without transmitting Bob’s private algorithm or data. Bob assists Alice either by building a predictive model using Alice’s labels, or by improving Alice’s private learning through iterative communications where only relevant statistics are transmitted. The proposed learning framework is naturally suitable for distributed, personalized, and privacy-aware scenarios. For example, it is shown in some scenarios that two suboptimal learners could achieve much better performance through Assisted Learning. Moreover, motivated by privacy concerns in Assisted Learning, we present a new notion of privacy to quantify the privacy leakage at learning level instead of data level. This new privacy, named imitation privacy, is particularly suitable for a market of statistical learners each holding private learning algorithms as well as data. |
| Tasks | |
| Published | 2020-04-01 |
| URL | https://arxiv.org/abs/2004.00566v1 |
| https://arxiv.org/pdf/2004.00566v1.pdf | |
| PWC | https://paperswithcode.com/paper/assisted-learning-and-imitation-privacy |
| Repo | |
| Framework | |
LGVTON: A Landmark Guided Approach to Virtual Try-On
| Title | LGVTON: A Landmark Guided Approach to Virtual Try-On |
| Authors | Debapriya Roy, Sanchayan Santra, Bhabatosh Chanda |
| Abstract | We address the problem of image based virtual try-on (VTON), where the goal is to synthesize an image of a person wearing the cloth of a model. An essential requirement for generating a perceptually convincing VTON result is preserving the characteristics of the cloth and the person. Keeping this in mind we propose \textit{LGVTON}, a novel self-supervised landmark guided approach to image based virtual try-on. The incorporation of self-supervision tackles the problem of lack of paired training data in model to person VTON scenario. LGVTON uses two types of landmarks to warp the model cloth according to the shape and pose of the person, one, human landmarks, the locations of anatomical keypoints of human, two, fashion landmarks, the structural keypoints of cloth. We introduce an unique way of using landmarks for warping which is more efficient and effective compared to existing warping based methods in current problem scenario. In addition to that, to make the method robust in cases of noisy landmark estimates that causes inaccurate warping, we propose a mask generator module that attempts to predict the true segmentation mask of the model cloth on the person, which in turn guides our image synthesizer module in tackling warping issues. Experimental results show the effectiveness of our method in comparison to the state-of-the-art VTON methods. |
| Tasks | |
| Published | 2020-04-01 |
| URL | https://arxiv.org/abs/2004.00562v1 |
| https://arxiv.org/pdf/2004.00562v1.pdf | |
| PWC | https://paperswithcode.com/paper/lgvton-a-landmark-guided-approach-to-virtual |
| Repo | |
| Framework | |
Statistical Queries and Statistical Algorithms: Foundations and Applications
| Title | Statistical Queries and Statistical Algorithms: Foundations and Applications |
| Authors | Lev Reyzin |
| Abstract | We give a survey of the foundations of statistical queries and their many applications to other areas. We introduce the model, give the main definitions, and we explore the fundamental theory statistical queries and how how it connects to various notions of learnability. We also give a detailed summary of some of the applications of statistical queries to other areas, including to optimization, to evolvability, and to differential privacy. |
| Tasks | |
| Published | 2020-04-01 |
| URL | https://arxiv.org/abs/2004.00557v1 |
| https://arxiv.org/pdf/2004.00557v1.pdf | |
| PWC | https://paperswithcode.com/paper/statistical-queries-and-statistical |
| Repo | |
| Framework | |
Coronavirus Covid-19 spreading in Italy: optimizing an epidemiological model with dynamic social distancing through Differential Evolution
| Title | Coronavirus Covid-19 spreading in Italy: optimizing an epidemiological model with dynamic social distancing through Differential Evolution |
| Authors | I. De Falco, A. Della Cioppa, U. Scafuri, E. Tarantino |
| Abstract | The aim of this paper consists in the application of a recent epidemiological model, namely SEIR with Social Distancing (SEIR–SD), extended here through the definition of a social distancing function varying over time, to assess the situation related to the spreading of the coronavirus Covid–19 in Italy and in two of its most important regions, i.e., Lombardy and Campania. To profitably use this model, the most suitable values of its parameters must be found. The estimation of the SEIR–SD model parameters takes place here through the use of Differential Evolution, a heuristic optimization technique. In this way, we are able to evaluate for each of the three above-mentioned scenarios the daily number of infectious cases from today until the end of virus spreading, the day(s) in which this number will be at its highest peak, and the day in which the infected cases will become very close to zero. |
| Tasks | |
| Published | 2020-04-01 |
| URL | https://arxiv.org/abs/2004.00553v1 |
| https://arxiv.org/pdf/2004.00553v1.pdf | |
| PWC | https://paperswithcode.com/paper/coronavirus-covid-19-spreading-in-italy |
| Repo | |
| Framework | |
Development of swarm behavior in artificial learning agents that adapt to different foraging environments
| Title | Development of swarm behavior in artificial learning agents that adapt to different foraging environments |
| Authors | Andrea López-Incera, Katja Ried, Thomas Müller, Hans J. Briegel |
| Abstract | Collective behavior, and swarm formation in particular, has been studied from several perspectives within a large variety of fields, ranging from biology to physics. In this work, we apply Projective Simulation to model each individual as an artificial learning agent that interacts with its neighbors and surroundings in order to make decisions and learn from them. Within a reinforcement learning framework, we discuss one-dimensional learning scenarios where agents need to get to food resources to be rewarded. We observe how different types of collective motion emerge depending on the distance the agents need to travel to reach the resources. For instance, strongly aligned swarms emerge when the food source is placed far away from the region where agents are situated initially. In addition, we study the properties of the individual trajectories that occur within the different types of emergent collective dynamics. Agents trained to find distant resources exhibit individual trajectories with L'evy-like characteristics as a consequence of the collective motion, whereas agents trained to reach nearby resources present Brownian-like trajectories. |
| Tasks | |
| Published | 2020-04-01 |
| URL | https://arxiv.org/abs/2004.00552v1 |
| https://arxiv.org/pdf/2004.00552v1.pdf | |
| PWC | https://paperswithcode.com/paper/development-of-swarm-behavior-in-artificial |
| Repo | |
| Framework | |
Physically Realizable Adversarial Examples for LiDAR Object Detection
| Title | Physically Realizable Adversarial Examples for LiDAR Object Detection |
| Authors | James Tu, Mengye Ren, Siva Manivasagam, Ming Liang, Bin Yang, Richard Du, Frank Cheng, Raquel Urtasun |
| Abstract | Modern autonomous driving systems rely heavily on deep learning models to process point cloud sensory data; meanwhile, deep models have been shown to be susceptible to adversarial attacks with visually imperceptible perturbations. Despite the fact that this poses a security concern for the self-driving industry, there has been very little exploration in terms of 3D perception, as most adversarial attacks have only been applied to 2D flat images. In this paper, we address this issue and present a method to generate universal 3D adversarial objects to fool LiDAR detectors. In particular, we demonstrate that placing an adversarial object on the rooftop of any target vehicle to hide the vehicle entirely from LiDAR detectors with a success rate of 80%. We report attack results on a suite of detectors using various input representation of point clouds. We also conduct a pilot study on adversarial defense using data augmentation. This is one step closer towards safer self-driving under unseen conditions from limited training data. |
| Tasks | Adversarial Defense, Autonomous Driving, Data Augmentation, Object Detection |
| Published | 2020-04-01 |
| URL | https://arxiv.org/abs/2004.00543v1 |
| https://arxiv.org/pdf/2004.00543v1.pdf | |
| PWC | https://paperswithcode.com/paper/physically-realizable-adversarial-examples |
| Repo | |
| Framework | |
Generation of Paths in a Maze using a Deep Network without Learning
| Title | Generation of Paths in a Maze using a Deep Network without Learning |
| Authors | Tomas Kulvicius, Sebastian Herzog, Minija Tamosiunaite, Florentin Wörgötter |
| Abstract | Trajectory- or path-planning is a fundamental issue in a wide variety of applications. Here we show that it is possible to solve path planning for multiple start- and end-points highly efficiently with a network that consists only of max pooling layers, for which no network training is needed. Different from competing approaches, very large mazes containing more than half a billion nodes with dense obstacle configuration and several thousand path end-points can this way be solved in very short time on parallel hardware. |
| Tasks | |
| Published | 2020-04-01 |
| URL | https://arxiv.org/abs/2004.00540v1 |
| https://arxiv.org/pdf/2004.00540v1.pdf | |
| PWC | https://paperswithcode.com/paper/generation-of-paths-in-a-maze-using-a-deep |
| Repo | |
| Framework | |
Learning Sparse Rewarded Tasks from Sub-Optimal Demonstrations
| Title | Learning Sparse Rewarded Tasks from Sub-Optimal Demonstrations |
| Authors | Zhuangdi Zhu, Kaixiang Lin, Bo Dai, Jiayu Zhou |
| Abstract | Model-free deep reinforcement learning (RL) has demonstrated its superiority on many complex sequential decision-making problems. However, heavy dependence on dense rewards and high sample-complexity impedes the wide adoption of these methods in real-world scenarios. On the other hand, imitation learning (IL) learns effectively in sparse-rewarded tasks by leveraging the existing expert demonstrations. In practice, collecting a sufficient amount of expert demonstrations can be prohibitively expensive, and the quality of demonstrations typically limits the performance of the learning policy. In this work, we propose Self-Adaptive Imitation Learning (SAIL) that can achieve (near) optimal performance given only a limited number of sub-optimal demonstrations for highly challenging sparse reward tasks. SAIL bridges the advantages of IL and RL to reduce the sample complexity substantially, by effectively exploiting sup-optimal demonstrations and efficiently exploring the environment to surpass the demonstrated performance. Extensive empirical results show that not only does SAIL significantly improve the sample-efficiency but also leads to much better final performance across different continuous control tasks, comparing to the state-of-the-art. |
| Tasks | Continuous Control, Decision Making, Imitation Learning |
| Published | 2020-04-01 |
| URL | https://arxiv.org/abs/2004.00530v1 |
| https://arxiv.org/pdf/2004.00530v1.pdf | |
| PWC | https://paperswithcode.com/paper/learning-sparse-rewarded-tasks-from-sub |
| Repo | |
| Framework | |
Statistical Optimal Transport posed as Learning Kernel Embedding
| Title | Statistical Optimal Transport posed as Learning Kernel Embedding |
| Authors | J. Saketha Nath, Pratik Jawanpuria |
| Abstract | This work takes the novel approach of posing the statistical Optimal Transport (OT) problem as that of learning the transport plan’s kernel mean embedding. The key advantage is that the estimates for the embeddings of the marginals can now be employed directly, leading to a dimension-free sample complexity for the proposed transport plan and transport map estimators. Also, because of the implicit smoothing in the kernel embeddings, the proposed estimators can perform out-of-sample estimation. Interestingly, the proposed formulation employs an MMD based regularization to avoid overfitting, which is complementary to existing $\phi$-divergence (entropy) based regularization techniques. An appropriate representer theorem is presented that leads to a fully kernelized formulation and hence the same formulation can be used to perform continuous/semi-discrete/discrete OT in any non-standard domain (as long as universal kernels in those domains are known). Finally, an ADMM based algorithm is presented for solving the kernelized formulation efficiently. Empirical results show that the proposed estimator outperforms discrete OT based estimator in terms of transport map accuracy. |
| Tasks | |
| Published | 2020-02-08 |
| URL | https://arxiv.org/abs/2002.03179v2 |
| https://arxiv.org/pdf/2002.03179v2.pdf | |
| PWC | https://paperswithcode.com/paper/statistical-optimal-transport-posed-as |
| Repo | |
| Framework | |
On a scalable entropic breaching of the overfitting barrier in machine learning
| Title | On a scalable entropic breaching of the overfitting barrier in machine learning |
| Authors | Illia Horenko |
| Abstract | Overfitting and treatment of “small data” are among the most challenging problems in the machine learning (ML), when a relatively small data statistics size $T$ is not enough to provide a robust ML fit for a relatively large data feature dimension $D$. Deploying a massively-parallel ML analysis of generic classification problems for different $D$ and $T$, existence of statistically-significant linear overfitting barriers for common ML methods is demonstrated. For example, these results reveal that for a robust classification of bioinformatics-motivated generic problems with the Long Short-Term Memory deep learning classifier (LSTM) one needs in a best case a statistics $T$ that is at least 13.8 times larger then the feature dimension $D$. It is shown that this overfitting barrier can be breached at a $10^{-12}$ fraction of the computational cost by means of the entropy-optimal Scalable Probabilistic Approximations algorithm (eSPA), performing a joint solution of the entropy-optimal Bayesian network inference and feature space segmentation problems. Application of eSPA to experimental single cell RNA sequencing data exhibits a 30-fold classification performance boost when compared to standard bioinformatics tools - and a 7-fold boost when compared to the deep learning LSTM classifier. |
| Tasks | |
| Published | 2020-02-08 |
| URL | https://arxiv.org/abs/2002.03176v1 |
| https://arxiv.org/pdf/2002.03176v1.pdf | |
| PWC | https://paperswithcode.com/paper/on-a-scalable-entropic-breaching-of-the |
| Repo | |
| Framework | |
Cross-Shape Graph Convolutional Networks
| Title | Cross-Shape Graph Convolutional Networks |
| Authors | Dmitry Petrov, Evangelos Kalogerakis |
| Abstract | We present a method that processes 3D point clouds by performing graph convolution operations across shapes. In this manner, point descriptors are learned by allowing interaction and propagation of feature representations within a shape collection. To enable this form of non-local, cross-shape graph convolution, our method learns a pairwise point attention mechanism indicating the degree of interaction between points on different shapes. Our method also learns to create a graph over shapes of an input collection whose edges connect shapes deemed as useful for performing cross-shape convolution. The edges are also equipped with learned weights indicating the compatibility of each shape pair for cross-shape convolution. Our experiments demonstrate that this interaction and propagation of point representations across shapes make them more discriminative. In particular, our results show significantly improved performance for 3D point cloud semantic segmentation compared to conventional approaches, especially in cases with the limited number of training examples. |
| Tasks | Semantic Segmentation |
| Published | 2020-03-20 |
| URL | https://arxiv.org/abs/2003.09053v2 |
| https://arxiv.org/pdf/2003.09053v2.pdf | |
| PWC | https://paperswithcode.com/paper/cross-shape-graph-convolutional-networks |
| Repo | |
| Framework | |
Improved RawNet with Filter-wise Rescaling for Text-independent Speaker Verification using Raw Waveforms
| Title | Improved RawNet with Filter-wise Rescaling for Text-independent Speaker Verification using Raw Waveforms |
| Authors | Jee-weon Jung, Seung-bin Kim, Hye-jin Shim, Ju-ho Kim, Ha-Jin Yu |
| Abstract | Recent advances in deep learning have facilitated the design of speaker verification systems that directly input raw waveforms. For example, RawNet extracts speaker embeddings from raw waveforms, which simplifies the process pipeline and demonstrates competitive performance. In this study, we improve RawNet by rescaling feature maps using various methods. The proposed mechanism utilizes a filter-wise rescale map that adopts a sigmoid non-linear function. It refers to a vector with dimensionality equal to the number of filters in a given feature map. Using a filter-wise rescale map, we propose to rescale the feature map multiplicatively, additively, or both. In addition, we investigate replacing the first convolution layer with the sinc-convolution layer of SincNet. Experiments performed on the VoxCeleb1 evaluation dataset demonstrate that the proposed methods are effective, and the best performing system reduces the equal error rate by half compared to the original RawNet. Expanded evaluation results obtained using the VoxCeleb1-E and VoxCeleb-H protocols marginally outperform existing state-of-the-art systems. |
| Tasks | Speaker Verification, Text-Independent Speaker Verification |
| Published | 2020-04-01 |
| URL | https://arxiv.org/abs/2004.00526v1 |
| https://arxiv.org/pdf/2004.00526v1.pdf | |
| PWC | https://paperswithcode.com/paper/improved-rawnet-with-filter-wise-rescaling |
| Repo | |
| Framework | |
Stochastic gradient descent with random learning rate
| Title | Stochastic gradient descent with random learning rate |
| Authors | Daniele Musso |
| Abstract | We propose to optimize neural networks with a uniformly-distributed random learning rate. The associated stochastic gradient descent algorithm can be approximated by continuous stochastic equations and analyzed with the Fokker-Planck formalism. In the small learning rate approximation, the training process is characterized by an effective temperature which depends on the average learning rate, the mini-batch size and the momentum of the optimization algorithm. By comparing the random learning rate protocol with cyclic and constant protocols, we suggest that the random choice is generically the best strategy in the small learning rate regime, yielding better regularization without extra computational cost. We provide supporting evidence through experiments on both shallow, fully-connected and deep, convolutional neural networks for image classification on the MNIST and CIFAR10 datasets. |
| Tasks | Image Classification |
| Published | 2020-03-15 |
| URL | https://arxiv.org/abs/2003.06926v2 |
| https://arxiv.org/pdf/2003.06926v2.pdf | |
| PWC | https://paperswithcode.com/paper/stochastic-gradient-descent-with-random |
| Repo | |
| Framework | |
