Paper Group NANR 73
Learning to Reach Goals Without Reinforcement Learning. HOW IMPORTANT ARE NETWORK WEIGHTS? TO WHAT EXTENT DO THEY NEED AN UPDATE?. Universal Adversarial Attack Using Very Few Test Examples. THE EFFECT OF ADVERSARIAL TRAINING: A THEORETICAL CHARACTERIZATION. Perturbations are not Enough: Generating Adversarial Examples with Spatial Distortions. Unre …
Learning to Reach Goals Without Reinforcement Learning
| Title | Learning to Reach Goals Without Reinforcement Learning |
| Authors | Anonymous |
| Abstract | Imitation learning algorithms provide a simple and straightforward approach for training control policies via standard supervised learning methods. By maximizing the likelihood of good actions provided by an expert demonstrator, supervised imitation learning can produce effective policies without the algorithmic complexities and optimization challenges of reinforcement learning, at the cost of requiring an expert demonstrator – typically a person – to provide the demonstrations. In this paper, we ask: can we use imitation learning to train effective policies without any expert demonstrations? The key observation that makes this possible is that, in the multi-task setting, trajectories that are generated by a suboptimal policy can still serve as optimal examples for other tasks. In particular, in the setting where the tasks correspond to different goals, every trajectory is a successful demonstration for the state that it actually reaches. Informed by this observation, we propose a very simple algorithm for learning behaviors without any demonstrations, user-provided reward functions, or complex reinforcement learning methods. Our method simply maximizes the likelihood of actions the agent actually took in its own previous rollouts, conditioned on the goal being the state that it actually reached. Although related variants of this approach have been proposed previously in imitation learning settings with example demonstrations, we present the first instance of this approach as a method for learning goal-reaching policies entirely from scratch. We present a theoretical result linking self-supervised imitation learning and reinforcement learning, and empirical results showing that it performs competitively with more complex reinforcement learning methods on a range of challenging goal reaching problems. |
| Tasks | Imitation Learning |
| Published | 2020-01-01 |
| URL | https://openreview.net/forum?id=ByxoqJrtvr |
| https://openreview.net/pdf?id=ByxoqJrtvr | |
| PWC | https://paperswithcode.com/paper/learning-to-reach-goals-without-reinforcement |
| Repo | |
| Framework | |
HOW IMPORTANT ARE NETWORK WEIGHTS? TO WHAT EXTENT DO THEY NEED AN UPDATE?
| Title | HOW IMPORTANT ARE NETWORK WEIGHTS? TO WHAT EXTENT DO THEY NEED AN UPDATE? |
| Authors | Fawaz Sammani, Mahmoud Elsayed, Abdelsalam Hamdi |
| Abstract | In the context of optimization, a gradient of a neural network indicates the amount a specific weight should change with respect to the loss. Therefore, small gradients indicate a good value of the weight that requires no change and can be kept frozen during training. This paper provides an experimental study on the importance of a neural network weights, and to which extent do they need to be updated. We wish to show that starting from the third epoch, freezing weights which have no informative gradient and are less likely to be changed during training, results in a very slight drop in the overall accuracy (and in sometimes better). We experiment on the MNIST, CIFAR10 and Flickr8k datasets using several architectures (VGG19, ResNet-110 and DenseNet-121). On CIFAR10, we show that freezing 80% of the VGG19 network parameters from the third epoch onwards results in 0.24% drop in accuracy, while freezing 50% of Resnet-110 parameters results in 0.9% drop in accuracy and finally freezing 70% of Densnet-121 parameters results in 0.57% drop in accuracy. Furthermore, to experiemnt with real-life applications, we train an image captioning model with attention mechanism on the Flickr8k dataset using LSTM networks, freezing 60% of the parameters from the third epoch onwards, resulting in a better BLEU-4 score than the fully trained model. Our source code can be found in the appendix. |
| Tasks | Image Captioning |
| Published | 2020-01-01 |
| URL | https://openreview.net/forum?id=rkg6PhNKDr |
| https://openreview.net/pdf?id=rkg6PhNKDr | |
| PWC | https://paperswithcode.com/paper/how-important-are-network-weights-to-what |
| Repo | |
| Framework | |
Universal Adversarial Attack Using Very Few Test Examples
| Title | Universal Adversarial Attack Using Very Few Test Examples |
| Authors | Anonymous |
| Abstract | Adversarial attacks such as Gradient-based attacks, Fast Gradient Sign Method (FGSM) by Goodfellow et al.(2015) and DeepFool by Moosavi-Dezfooli et al. (2016) are input-dependent, small pixel-wise perturbations of images which fool state of the art neural networks into misclassifying images but are unlikely to fool any human. On the other hand a universal adversarial attack is an input-agnostic perturbation. The same perturbation is applied to all inputs and yet the neural network is fooled on a large fraction of the inputs. In this paper, we show that multiple known input-dependent pixel-wise perturbations share a common spectral property. Using this spectral property, we show that the top singular vector of input-dependent adversarial attack directions can be used as a very simple universal adversarial attack on neural networks. We evaluate the error rates and fooling rates of three universal attacks, SVD-Gradient, SVD-DeepFool and SVD-FGSM, on state of the art neural networks. We show that these universal attack vectors can be computed using a small sample of test inputs. We establish our results both theoretically and empirically. On VGG19 and VGG16, the fooling rate of SVD-DeepFool and SVD-Gradient perturbations constructed from observing less than 0.2% of the validation set of ImageNet is as good as the universal attack of Moosavi-Dezfooli et al. (2017a). To prove our theoretical results, we use matrix concentration inequalities and spectral perturbation bounds. For completeness, we also discuss another recent approach to universal adversarial perturbations based on (p, q)-singular vectors, proposed independently by Khrulkov & Oseledets (2018), and point out the simplicity and efficiency of our universal attack as the key difference. |
| Tasks | Adversarial Attack |
| Published | 2020-01-01 |
| URL | https://openreview.net/forum?id=HygS91rYvH |
| https://openreview.net/pdf?id=HygS91rYvH | |
| PWC | https://paperswithcode.com/paper/universal-adversarial-attack-using-very-few |
| Repo | |
| Framework | |
THE EFFECT OF ADVERSARIAL TRAINING: A THEORETICAL CHARACTERIZATION
| Title | THE EFFECT OF ADVERSARIAL TRAINING: A THEORETICAL CHARACTERIZATION |
| Authors | Anonymous |
| Abstract | It has widely shown that adversarial training (Madry et al., 2018) is effective in defending adversarial attack empirically. However, the theoretical understanding of the difference between the solution of adversarial training and that of standard training is limited. In this paper, we characterize the solution of adversarial training for linear classification problem for a full range of adversarial radius “. Specifically, we show that if the data themselves are ”-strongly linearly-separable”, adversarial training with radius smaller than " converges to the hard margin solution of SVM with a faster rate than standard training. If the data themselves are not ”-strongly linearly-separable”, we show that adversarial training with radius " is stable to outliers while standard training is not. Moreover, we prove that the classifier returned by adversarial training with a large radius " has low confidence in each data point. Experiments corroborate our theoretical finding well. |
| Tasks | Adversarial Attack |
| Published | 2020-01-01 |
| URL | https://openreview.net/forum?id=B1eXvyHKwS |
| https://openreview.net/pdf?id=B1eXvyHKwS | |
| PWC | https://paperswithcode.com/paper/the-effect-of-adversarial-training-a |
| Repo | |
| Framework | |
Perturbations are not Enough: Generating Adversarial Examples with Spatial Distortions
| Title | Perturbations are not Enough: Generating Adversarial Examples with Spatial Distortions |
| Authors | Anonymous |
| Abstract | Deep neural network image classifiers are reported to be susceptible to adversarial evasion attacks, which use carefully crafted images created to mislead a classifier. Recently, various kinds of adversarial attack methods have been proposed, most of which focus on adding small perturbations to input images. Despite the success of existing approaches, the way to generate realistic adversarial images with small perturbations remains a challenging problem. In this paper, we aim to address this problem by proposing a novel adversarial method, which generates adversarial examples by imposing not only perturbations but also spatial distortions on input images, including scaling, rotation, shear, and translation. As humans are less susceptible to small spatial distortions, the proposed approach can produce visually more realistic attacks with smaller perturbations, able to deceive classifiers without affecting human predictions. We learn our method by amortized techniques with neural networks and generate adversarial examples efficiently by a forward pass of the networks. Extensive experiments on attacking different types of non-robustified classifiers and robust classifiers with defence show that our method has state-of-the-art performance in comparison with advanced attack parallels. |
| Tasks | Adversarial Attack |
| Published | 2020-01-01 |
| URL | https://openreview.net/forum?id=HJg3HyStwB |
| https://openreview.net/pdf?id=HJg3HyStwB | |
| PWC | https://paperswithcode.com/paper/perturbations-are-not-enough-generating |
| Repo | |
| Framework | |
Unrestricted Adversarial Attacks For Semantic Segmentation
| Title | Unrestricted Adversarial Attacks For Semantic Segmentation |
| Authors | Anonymous |
| Abstract | Despite the rapid development of adversarial attacks for machine learning models, many types of new adversarial examples still remain unknown. Uncovered types of adversarial attacks pose serious concern for the safety of the models, which raise the question about the effectiveness of current adversarial robustness evaluation. Semantic segmentation is one of the most impactful applications of machine learning; however, their robustness under adversarial attack is not well studied. In this paper, we focus on generating unrestricted adversarial examples for semantic segmentation models. We demonstrate a simple yet effective method to generate unrestricted adversarial examples using conditional generative adversarial networks (CGAN) without any hand-crafted metric. The naive implementation of CGAN, however, yields inferior image quality and low attack success rate. Instead, we leverage the SPADE (Spatially-adaptive denormalization) structure with an additional loss item, which is able to generate effective adversarial attacks in a single step. We validate our approach on the well studied Cityscapes andADE20K datasets, and demonstrate that our synthetic adversarial examples are not only realistic, but also improves the attack success rate by up to 41.0% compared with the state of the art adversarial attack methods including PGD attack. |
| Tasks | Adversarial Attack, Semantic Segmentation |
| Published | 2020-01-01 |
| URL | https://openreview.net/forum?id=rkx6MJSFPH |
| https://openreview.net/pdf?id=rkx6MJSFPH | |
| PWC | https://paperswithcode.com/paper/unrestricted-adversarial-attacks-for-semantic |
| Repo | |
| Framework | |
Accelerating Monte Carlo Bayesian Inference via Approximating Predictive Uncertainty over the Simplex
| Title | Accelerating Monte Carlo Bayesian Inference via Approximating Predictive Uncertainty over the Simplex |
| Authors | Anonymous |
| Abstract | Estimating the predictive uncertainty of a Bayesian learning model is critical in various decision-making problems, e.g., reinforcement learning, detecting adversarial attack, self-driving car. As the model posterior is almost always intractable, most efforts were made on finding an accurate approximation the true posterior. Even though a decent estimation of the model posterior is obtained, another approximation is required to compute the predictive distribution over the desired output. A common accurate solution is to use Monte Carlo (MC) integration. However, it needs to maintain a large number of samples, evaluate the model repeatedly and average multiple model outputs. In many real-world cases, this is computationally prohibitive. In this work, assuming that the exact posterior or a decent approximation is obtained, we propose a generic framework to approximate the output probability distribution induced by model posterior with a parameterized model and in an amortized fashion. The aim is to approximate the true uncertainty of a specific Bayesian model, meanwhile alleviating the heavy workload of MC integration at testing time. The proposed method is universally applicable to Bayesian classification models that allow for posterior sampling. Theoretically, we show that the idea of amortization incurs no additional costs on approximation performance. Empirical results validate the strong practical performance of our approach. |
| Tasks | Adversarial Attack, Bayesian Inference, Decision Making |
| Published | 2020-01-01 |
| URL | https://openreview.net/forum?id=HJlHzJBFwB |
| https://openreview.net/pdf?id=HJlHzJBFwB | |
| PWC | https://paperswithcode.com/paper/accelerating-monte-carlo-bayesian-inference-1 |
| Repo | |
| Framework | |
Enhancing Transformation-Based Defenses Against Adversarial Attacks with a Distribution Classifier
| Title | Enhancing Transformation-Based Defenses Against Adversarial Attacks with a Distribution Classifier |
| Authors | Anonymous |
| Abstract | Adversarial attacks on convolutional neural networks (CNN) have gained significant attention and there have been active research efforts on defense mechanisms. Stochastic input transformation methods have been proposed, where the idea is to recover the image from adversarial attack by random transformation, and to take the majority vote as consensus among the random samples. However, the transformation improves the accuracy on adversarial images at the expense of the accuracy on clean images. While it is intuitive that the accuracy on clean images would deteriorate, the exact mechanism in which how this occurs is unclear. In this paper, we study the distribution of softmax induced by stochastic transformations. We observe that with random transformations on the clean images, although the mass of the softmax distribution could shift to the wrong class, the resulting distribution of softmax could be used to correct the prediction. Furthermore, on the adversarial counterparts, with the image transformation, the resulting shapes of the distribution of softmax are similar to the distributions from the clean images. With these observations, we propose a method to improve existing transformation-based defenses. We train a separate lightweight distribution classifier to recognize distinct features in the distributions of softmax outputs of transformed images. Our empirical studies show that our distribution classifier, by training on distributions obtained from clean images only, outperforms majority voting for both clean and adversarial images. Our method is generic and can be integrated with existing transformation-based defenses. |
| Tasks | Adversarial Attack |
| Published | 2020-01-01 |
| URL | https://openreview.net/forum?id=BkgWahEFvr |
| https://openreview.net/pdf?id=BkgWahEFvr | |
| PWC | https://paperswithcode.com/paper/enhancing-transformation-based-defenses |
| Repo | |
| Framework | |
Sign-OPT: A Query-Efficient Hard-label Adversarial Attack
| Title | Sign-OPT: A Query-Efficient Hard-label Adversarial Attack |
| Authors | Anonymous |
| Abstract | We study the most practical problem setup for evaluating adversarial robustness of a machine learning system with limited access: the hard-label black-box attack setting for generating adversarial examples, where limited model queries are allowed and only the decision is provided to a queried data input. Several algorithms have been proposed for this problem but they typically require huge amount (>20,000) of queries for attacking one example. Among them, one of the state-of-the-art approaches (Cheng et al., 2019) showed that hard-label attack can be modeled as an optimization problem where the objective function can be evaluated by binary search with additional model queries, thereby a zeroth order optimization algorithm can be applied. In this paper, we adopt the same optimization formulation but propose to directly estimate the sign of gradient at any direction instead of the gradient itself, which enjoys the benefit of single query. Using this single query oracle for retrieving sign of directional derivative, we develop a novel query-efficient Sign-OPT approach for hard-label black-box attack. We provide a convergence analysis of the new algorithm and conduct experiments on several models on MNIST, CIFAR-10 and ImageNet. We find that Sign-OPT attack consistently requires 5X to 10X fewer queries when compared to the current state-of-the-art approaches, and usually converges to an adversarial example with smaller perturbation. |
| Tasks | Adversarial Attack |
| Published | 2020-01-01 |
| URL | https://openreview.net/forum?id=SklTQCNtvS |
| https://openreview.net/pdf?id=SklTQCNtvS | |
| PWC | https://paperswithcode.com/paper/sign-opt-a-query-efficient-hard-label-1 |
| Repo | |
| Framework | |
Simple and Effective Stochastic Neural Networks
| Title | Simple and Effective Stochastic Neural Networks |
| Authors | Anonymous |
| Abstract | Stochastic neural networks (SNNs) are currently topical, with several paradigms being actively investigated including dropout, Bayesian neural networks, variational information bottleneck (VIB) and noise regularized learning. These neural network variants impact several major considerations, including generalization, network compression, and robustness against adversarial attack and label noise. However, many existing networks are complicated and expensive to train, and/or only address one or two of these practical considerations. In this paper we propose a simple and effective stochastic neural network (SE-SNN) architecture for discriminative learning by directly modeling activation uncertainty and encouraging high activation variability. Compared to existing SNNs, our SE-SNN is simpler to implement and faster to train, and produces state of the art results on network compression by pruning, adversarial defense and learning with label noise. |
| Tasks | Adversarial Attack, Adversarial Defense |
| Published | 2020-01-01 |
| URL | https://openreview.net/forum?id=SJxeI6EYwS |
| https://openreview.net/pdf?id=SJxeI6EYwS | |
| PWC | https://paperswithcode.com/paper/simple-and-effective-stochastic-neural |
| Repo | |
| Framework | |
R2D2: Reuse & Reduce via Dynamic Weight Diffusion for Training Efficient NLP Models
| Title | R2D2: Reuse & Reduce via Dynamic Weight Diffusion for Training Efficient NLP Models |
| Authors | Anonymous |
| Abstract | We propose R2D2 layers, a new neural block for training efficient NLP models. Our proposed method is characterized by a dynamic weight diffusion mechanism which learns to reuse and reduce parameters in the conventional transformation layer, commonly found in popular Transformer/LSTMs models. Our method is inspired by recent Quaternion methods which share parameters via the Hamilton product. This can be interpreted as a neural and learned approximation of the Hamilton product which imbues our method with increased flexibility and expressiveness, i.e., we are no longer restricted by the 4D nature of Quaternion weight sharing. We conduct extensive experiments in the NLP domain, showing that R2D2 (i) enables a parameter savings of up to 2 times to 16 times with minimal degradation of performance and (ii) outperforms other parameter savings alternative such as low-rank factorization and Quaternion methods. |
| Tasks | |
| Published | 2020-01-01 |
| URL | https://openreview.net/forum?id=H1lTQ1rFvS |
| https://openreview.net/pdf?id=H1lTQ1rFvS | |
| PWC | https://paperswithcode.com/paper/r2d2-reuse-reduce-via-dynamic-weight |
| Repo | |
| Framework | |
RATE-DISTORTION OPTIMIZATION GUIDED AUTOENCODER FOR GENERATIVE APPROACH
| Title | RATE-DISTORTION OPTIMIZATION GUIDED AUTOENCODER FOR GENERATIVE APPROACH |
| Authors | Anonymous |
| Abstract | In the generative model approach of machine learning, it is essential to acquire an accurate probabilistic model and compress the dimension of data for easy treatment. However, in the conventional deep-autoencoder based generative model such as VAE, the probability of the real space cannot be obtained correctly from that of in the latent space, because the scaling between both spaces is not controlled. This has also been an obstacle to quantifying the impact of the variation of latent variables on data. In this paper, we propose a method to learn parametric probability distribution and autoencoder simultaneously based on Rate-Distortion Optimization to support scaling control. It is proved theoretically and experimentally that (i) the probability distribution of the latent space obtained by this model is proportional to the probability distribution of the real space because Jacobian between two spaces is constant: (ii) our model behaves as non-linear PCA, which enables to evaluate the influence of latent variables on data. Furthermore, to verify the usefulness on the practical application, we evaluate its performance in unsupervised anomaly detection and outperform current state-of-the-art methods. |
| Tasks | Anomaly Detection, Unsupervised Anomaly Detection |
| Published | 2020-01-01 |
| URL | https://openreview.net/forum?id=BJxeHyrKPB |
| https://openreview.net/pdf?id=BJxeHyrKPB | |
| PWC | https://paperswithcode.com/paper/rate-distortion-optimization-guided |
| Repo | |
| Framework | |
Revisit Knowledge Distillation: a Teacher-free Framework
| Title | Revisit Knowledge Distillation: a Teacher-free Framework |
| Authors | Anonymous |
| Abstract | Knowledge Distillation (KD) aims to distill the knowledge of a cumbersome teacher model into a lightweight student model. Its success is generally attributed to the privileged information on similarities among categories provided by the teacher model, and in this sense, only strong teacher models are deployed to teach weaker students in practice. In this work, we challenge this common belief by following experimental observations: 1) beyond the acknowledgment that the teacher can improve the student, the student can also enhance the teacher significantly by reversing the KD procedure; 2) a poorly-trained teacher with much lower accuracy than the student can still improve the latter significantly. To explain these observations, we provide a theoretical analysis of the relationships between KD and label smoothing regularization. We prove that 1) KD is a type of learned label smoothing regularization and 2) label smoothing regularization provides a virtual teacher model for KD. From these results, we argue that the success of KD is not fully due to the similarity information between categories, but also to the regularization of soft targets, which is equally or even more important. Based on these analyses, we further propose a novel Teacher-free Knowledge Distillation (Tf-KD) framework, where a student model learns from itself or manually-designed regularization distribution. The Tf-KD achieves comparable performance with normal KD from a superior teacher. It is generic and can be directly deployed for training deep neural networks. Without any extra computation cost, Tf-KD achieves up to 0.65% improvement on ImageNet over well-established baseline models, which is superior to label smoothing regularization. |
| Tasks | |
| Published | 2020-01-01 |
| URL | https://openreview.net/forum?id=BJxA924YvS |
| https://openreview.net/pdf?id=BJxA924YvS | |
| PWC | https://paperswithcode.com/paper/revisit-knowledge-distillation-a-teacher-free-1 |
| Repo | |
| Framework | |
Imagine That! Leveraging Emergent Affordances for Tool Synthesis in Reaching Tasks
| Title | Imagine That! Leveraging Emergent Affordances for Tool Synthesis in Reaching Tasks |
| Authors | Anonymous |
| Abstract | In this paper we investigate an artificial agent’s ability to perform task-focused tool synthesis via imagination. Our motivation is to explore the richness of information captured by the latent space of an object-centric generative model - and how to exploit it. In particular, our approach employs activation maximisation of a task-based performance predictor to optimise the latent variable of a structured latent-space model in order to generate tool geometries appropriate for the task at hand. We evaluate our model using a novel dataset of synthetic reaching tasks inspired by the cognitive sciences and behavioural ecology. In doing so we examine the model’s ability to imagine tools for increasingly complex scenario types, beyond those seen during training. Our experiments demonstrate that the synthesis process modifies emergent, task-relevant object affordances in a targeted and deliberate way: the agents often specifically modify aspects of the tools which relate to meaningful (yet implicitly learned) concepts such as a tool’s length, width and configuration. Our results therefore suggest, that task relevant object affordances are implicitly encoded as directions in a structured latent space shaped by experience. |
| Tasks | |
| Published | 2020-01-01 |
| URL | https://openreview.net/forum?id=BkeyOxrYwH |
| https://openreview.net/pdf?id=BkeyOxrYwH | |
| PWC | https://paperswithcode.com/paper/imagine-that-leveraging-emergent-affordances |
| Repo | |
| Framework | |
Multi-Agent Hierarchical Reinforcement Learning for Humanoid Navigation
| Title | Multi-Agent Hierarchical Reinforcement Learning for Humanoid Navigation |
| Authors | Anonymous |
| Abstract | Multi-agent reinforcement learning is a particularly challenging problem. Current methods have made progress on cooperative and competitive environments with particle-based agents. Little progress has been made on solutions that could op- erate in the real world with interaction, dynamics, and humanoid robots. In this work, we make a significant step in multi-agent models on simulated humanoid robot navigation by combining Multi-Agent Reinforcement Learning (MARL) with Hierarchical Reinforcement Learning (HRL). We build on top of founda- tional prior work in learning low-level physical controllers for locomotion and add a layer to learn decentralized policies for multi-agent goal-directed collision avoidance systems. A video of our results on a multi-agent pursuit environment can be seen here |
| Tasks | Hierarchical Reinforcement Learning, Multi-agent Reinforcement Learning, Robot Navigation |
| Published | 2020-01-01 |
| URL | https://openreview.net/forum?id=B1ldb6NKDr |
| https://openreview.net/pdf?id=B1ldb6NKDr | |
| PWC | https://paperswithcode.com/paper/multi-agent-hierarchical-reinforcement-1 |
| Repo | |
| Framework | |
