Paper Group NANR 122
Optimal Strategies Against Generative Attacks. Versatile Anomaly Detection with Outlier Preserving Distribution Mapping Autoencoders. Deep RL for Blood Glucose Control: Lessons, Challenges, and Opportunities. Towards Certified Defense for Unrestricted Adversarial Attacks. Minimally distorted Adversarial Examples with a Fast Adaptive Boundary Attack …
Optimal Strategies Against Generative Attacks
| Title | Optimal Strategies Against Generative Attacks |
| Authors | Anonymous |
| Abstract | Generative neural models have improved dramatically recently. With this progress comes the risk that such models will be used to attack systems that rely on sensor data for authentication and anomaly detection. Many such learning systems are installed worldwide, protecting critical infrastructure or private data against malfunction and cyber attacks. We formulate the scenario of such an authentication system facing generative impersonation attacks, characterize it from a theoretical perspective and explore its practical implications. In particular, we ask fundamental theoretical questions in learning, statistics and information theory: How hard is it to detect a ``fake reality’'? How much data does the attacker need to collect before it can reliably generate nominally-looking artificial data? Are there optimal strategies for the attacker or the authenticator? We cast the problem as a maximin game, characterize the optimal strategy for both attacker and authenticator in the general case, and provide the optimal strategies in closed form for the case of Gaussian source distributions. Our analysis reveals the structure of the optimal attack and the relative importance of data collection for both authenticator and attacker. Based on these insights we design practical learning approaches, and show that they result in models that are more robust to various attacks on real-world data. Code will be made publicly available upon publication. | |
| Tasks | Anomaly Detection |
| Published | 2020-01-01 |
| URL | https://openreview.net/forum?id=BkgzMCVtPB |
| https://openreview.net/pdf?id=BkgzMCVtPB | |
| PWC | https://paperswithcode.com/paper/optimal-strategies-against-generative-attacks |
| Repo | |
| Framework | |
Versatile Anomaly Detection with Outlier Preserving Distribution Mapping Autoencoders
| Title | Versatile Anomaly Detection with Outlier Preserving Distribution Mapping Autoencoders |
| Authors | Anonymous |
| Abstract | State-of-the-art deep learning methods for outlier detection make the assumption that anomalies will appear far away from inlier data in the latent space produced by distribution mapping deep networks. However, this assumption fails in practice, because the divergence penalty adopted for this purpose encourages mapping outliers into the same high-probability regions as inliers. To overcome this shortcoming, we introduce a novel deep learning outlier detection method, called Outlier Preserving Distribution Mapping Autoencoder (OP-DMA), which succeeds to map outliers to low probability regions in the latent space of an autoencoder. For this we leverage the insight that outliers are likely to have a higher reconstruction error than inliers. We thus achieve outlier-preserving distribution mapping through weighting the reconstruction error of individual points by the value of a multivariate Gaussian probability density function evaluated at those points. This weighting implies that outliers will result overall penalty if they are mapped to low-probability regions. We show that if the global minimum of our newly proposed loss function is achieved, then our OP-DMA maps inliers to regions with a Mahalanobis distance less than delta, and outliers to regions past this delta, delta being the inverse Chi Squared CDF evaluated at (1-alpha) with alpha the percentage of outliers in the dataset. Our experiments confirm that OP-DMA consistently outperforms the state-of-art methods on a rich variety of outlier detection benchmark datasets. |
| Tasks | Anomaly Detection, Outlier Detection |
| Published | 2020-01-01 |
| URL | https://openreview.net/forum?id=SkxV0RVYDH |
| https://openreview.net/pdf?id=SkxV0RVYDH | |
| PWC | https://paperswithcode.com/paper/versatile-anomaly-detection-with-outlier |
| Repo | |
| Framework | |
Deep RL for Blood Glucose Control: Lessons, Challenges, and Opportunities
| Title | Deep RL for Blood Glucose Control: Lessons, Challenges, and Opportunities |
| Authors | Anonymous |
| Abstract | Individuals with type 1 diabetes (T1D) lack the ability to produce the insulin their bodies need. As a result, they must continually make decisions about how much insulin to self-administer in order to adequately control their blood glucose levels. Longitudinal data streams captured from wearables, like continuous glucose monitors, can help these individuals manage their health, but currently the majority of the decision burden remains on the user. To relieve this burden, researchers are working on closed-loop solutions that combine a continuous glucose monitor and an insulin pump with a control algorithm in an `artificial pancreas.’ Such systems aim to estimate and deliver the appropriate amount of insulin. Here, we develop reinforcement learning (RL) techniques for automated blood glucose control. Through a series of experiments, we compare the performance of different deep RL approaches to non-RL approaches. We highlight the flexibility of RL approaches, demonstrating how they can adapt to new individuals with little additional data. On over 21k hours of simulated data across 30 patients, RL approaches outperform baseline control algorithms (increasing time spent in normal glucose range from 71% to 75%) without requiring meal announcements. Moreover, these approaches are adept at leveraging latent behavioral patterns (increasing time in range from 58% to 70%). This work demonstrates the potential of deep RL for controlling complex physiological systems with minimal expert knowledge. | |
| Tasks | |
| Published | 2020-01-01 |
| URL | https://openreview.net/forum?id=ryeN5aEYDH |
| https://openreview.net/pdf?id=ryeN5aEYDH | |
| PWC | https://paperswithcode.com/paper/deep-rl-for-blood-glucose-control-lessons |
| Repo | |
| Framework | |
Towards Certified Defense for Unrestricted Adversarial Attacks
| Title | Towards Certified Defense for Unrestricted Adversarial Attacks |
| Authors | Anonymous |
| Abstract | Certified defenses against adversarial examples are very important in safety-critical applications of machine learning. However, existing certified defense strategies only safeguard against perturbation-based adversarial attacks, where the attacker is only allowed to modify normal data points by adding small perturbations. In this paper, we provide certified defenses under the more general threat model of unrestricted adversarial attacks. We allow the attacker to generate arbitrary inputs to fool the classifier, and assume the attacker knows everything except the classifiers’ parameters and the training dataset used to learn it. Lack of knowledge about the classifiers parameters prevents an attacker from generating adversarial examples successfully. Our defense draws inspiration from differential privacy, and is based on intentionally adding noise to the classifier’s outputs to limit the attacker’s knowledge about the parameters. We prove concrete bounds on the minimum number of queries required for any attacker to generate a successful adversarial attack. For a simple linear classifiers we prove that the bound is asymptotically optimal up to a constant by exhibiting an attack algorithm that achieves this lower bound. We empirically show the success of our defense strategy against strong black box attack algorithms. |
| Tasks | Adversarial Attack |
| Published | 2020-01-01 |
| URL | https://openreview.net/forum?id=S1lBVgHYvr |
| https://openreview.net/pdf?id=S1lBVgHYvr | |
| PWC | https://paperswithcode.com/paper/towards-certified-defense-for-unrestricted |
| Repo | |
| Framework | |
Minimally distorted Adversarial Examples with a Fast Adaptive Boundary Attack
| Title | Minimally distorted Adversarial Examples with a Fast Adaptive Boundary Attack |
| Authors | Anonymous |
| Abstract | The evaluation of robustness against adversarial manipulations of neural networks-based classifiers is mainly tested with empirical attacks as the methods for the exact computation, even when available, do not scale to large networks. We propose in this paper a new white-box adversarial attack wrt the $l_p$-norms for $p \in {1,2,\infty}$ aiming at finding the minimal perturbation necessary to change the class of a given input. It has an intuitive geometric meaning, yields quickly high quality results, minimizes the size of the perturbation (so that it returns the robust accuracy at every threshold with a single run). It performs better or similarly to state-of-the-art attacks which are partially specialized to one $l_p$-norm. |
| Tasks | Adversarial Attack |
| Published | 2020-01-01 |
| URL | https://openreview.net/forum?id=HJlzxgBtwH |
| https://openreview.net/pdf?id=HJlzxgBtwH | |
| PWC | https://paperswithcode.com/paper/minimally-distorted-adversarial-examples-with-1 |
| Repo | |
| Framework | |
Generalized Inner Loop Meta-Learning
| Title | Generalized Inner Loop Meta-Learning |
| Authors | Anonymous |
| Abstract | Many (but not all) approaches self-qualifying as “meta-learning” in deep learning and reinforcement learning fit a common pattern of approximating the solution to a nested optimization problem. In this paper, we give a formalization of this shared pattern, which we call GIMLI, prove its general requirements, and derive a general-purpose algorithm for implementing similar approaches. Based on this analysis and algorithm, we describe a library of our design, unnamedlib, which we share with the community to assist and enable future research into these kinds of meta-learning approaches. We end the paper by showcasing the practical applications of this framework and library through illustrative experiments and ablation studies which they facilitate. |
| Tasks | Meta-Learning |
| Published | 2020-01-01 |
| URL | https://openreview.net/forum?id=BygWRaVYwH |
| https://openreview.net/pdf?id=BygWRaVYwH | |
| PWC | https://paperswithcode.com/paper/generalized-inner-loop-meta-learning-1 |
| Repo | |
| Framework | |
Efficient and Robust Asynchronous Federated Learning with Stragglers
| Title | Efficient and Robust Asynchronous Federated Learning with Stragglers |
| Authors | Anonymous |
| Abstract | We address the efficiency issues caused by the straggler effect in the recently emerged federated learning, which collaboratively trains a model on decentralized non-i.i.d. (non-independent and identically distributed) data across massive worker devices without exchanging training data in the unreliable and heterogeneous networks. We propose a novel two-stage analysis on the error bounds of general federated learning, which provides practical insights into optimization. As a result, we propose a novel easy-to-implement federated learning algorithm that uses asynchronous settings and strategies to control discrepancies between the global model and delayed models and adjust the number of local epochs with the estimation of staleness to accelerate convergence and resist performance deterioration caused by stragglers. Experiment results show that our algorithm converges fast and robust on the existence of massive stragglers. |
| Tasks | |
| Published | 2020-01-01 |
| URL | https://openreview.net/forum?id=B1lL9grYDS |
| https://openreview.net/pdf?id=B1lL9grYDS | |
| PWC | https://paperswithcode.com/paper/efficient-and-robust-asynchronous-federated |
| Repo | |
| Framework | |
LEARNING TO IMPUTE: A GENERAL FRAMEWORK FOR SEMI-SUPERVISED LEARNING
| Title | LEARNING TO IMPUTE: A GENERAL FRAMEWORK FOR SEMI-SUPERVISED LEARNING |
| Authors | Anonymous |
| Abstract | Recent semi-supervised learning methods have shown to achieve comparable results to their supervised counterparts while using only a small portion of labels in image classification tasks thanks to their regularization strategies. In this paper, we take a more direct approach for semi-supervised learning and propose learning to impute the labels of unlabeled samples such that a network achieves better generalization when it is trained on these labels. We pose the problem in a learning-to-learn formulation which can easily be incorporated to the state-of-the-art semi-supervised techniques and boost their performance especially when the labels are limited. We demonstrate that our method is applicable to both classification and regression problems including image classification and facial landmark detection tasks. |
| Tasks | Facial Landmark Detection, Image Classification |
| Published | 2020-01-01 |
| URL | https://openreview.net/forum?id=SkxHRySFvr |
| https://openreview.net/pdf?id=SkxHRySFvr | |
| PWC | https://paperswithcode.com/paper/learning-to-impute-a-general-framework-for |
| Repo | |
| Framework | |
B-Spline CNNs on Lie groups
| Title | B-Spline CNNs on Lie groups |
| Authors | Anonymous |
| Abstract | Group convolutional neural networks (G-CNNs) can be used to improve classical CNNs by equipping them with the geometric structure of groups. Central in the success of G-CNNs is the lifting of feature maps to higher dimensional disentangled representations, in which data characteristics are effectively learned, geometric data-augmentations are made obsolete, and predictable behavior under geometric transformations (equivariance) is guaranteed via group theory. Currently, however, the practical implementations of G-CNNs are limited to either discrete groups (that leave the grid intact) or continuous compact groups such as rotations (that enable the use of Fourier theory). In this paper we lift these limitations and propose a modular framework for the design and implementation of G-CNNs for arbitrary Lie groups. In our approach the differential structure of Lie groups is used to expand convolution kernels in a generic basis of B-splines that is defined on the Lie algebra. This leads to a flexible framework that enables localized, atrous, and deformable convolutions in G-CNNs by means of respectively localized, sparse and non-uniform B-spline expansions. The impact and potential of our approach is studied on two benchmark datasets: cancer detection in histopathology slides (PCam dataset) in which rotation equivariance plays a key role and facial landmark localization (CelebA dataset) in which scale equivariance is important. In both cases, G-CNN architectures outperform their classical 2D counterparts and the added value of atrous and localized group convolutions is studied in detail. |
| Tasks | Face Alignment |
| Published | 2020-01-01 |
| URL | https://openreview.net/forum?id=H1gBhkBFDH |
| https://openreview.net/pdf?id=H1gBhkBFDH | |
| PWC | https://paperswithcode.com/paper/b-spline-cnns-on-lie-groups |
| Repo | |
| Framework | |
Fast Sparse ConvNets
| Title | Fast Sparse ConvNets |
| Authors | Anonymous |
| Abstract | Historically, the pursuit of efficient inference has been one of the driving forces be-hind the research into new deep learning architectures and building blocks. Some of the recent examples include: the squeeze-and-excitation module of (Hu et al.,2018), depthwise separable convolutions in Xception (Chollet, 2017), and the inverted bottleneck in MobileNet v2 (Sandler et al., 2018). Notably, in all of these cases, the resulting building blocks enabled not only higher efficiency, but also higher accuracy, and found wide adoption in the field. In this work, we further expand the arsenal of efficient building blocks for neural network architectures; but instead of combining standard primitives (such as convolution), we advocate for the replacement of these dense primitives with their sparse counterparts. While the idea of using sparsity to decrease the parameter count is not new (Mozer & Smolensky, 1989), the conventional wisdom is that this reduction in theoretical FLOPs does not translate into real-world efficiency gains. We aim to correct this misconception by introducing a family of efficient sparse kernels for several hardware platforms, which we plan to open-source for the benefit of the community. Equipped with our efficient implementation of sparse primitives, we show that sparse versions of MobileNet v1 and MobileNet v2 architectures substantially outperform strong dense baselines on the efficiency-accuracy curve. On Snapdragon 835 our sparse networks outperform their dense equivalents by 1.1−2.2x – equivalent to approximately one entire generation of improvement. We hope that our findings will facilitate wider adoption of sparsity as a tool for creating efficient and accurate deep learning architectures. |
| Tasks | |
| Published | 2020-01-01 |
| URL | https://openreview.net/forum?id=SkgHtkrYPH |
| https://openreview.net/pdf?id=SkgHtkrYPH | |
| PWC | https://paperswithcode.com/paper/fast-sparse-convnets |
| Repo | |
| Framework | |
The problem with DDPG: understanding failures in deterministic environments with sparse rewards
| Title | The problem with DDPG: understanding failures in deterministic environments with sparse rewards |
| Authors | Anonymous |
| Abstract | In environments with continuous state and action spaces, state-of-the-art actor-critic reinforcement learning algorithms can solve very complex problems, yet can also fail in environments that seem trivial, but the reason for such failures is still poorly understood. In this paper, we contribute a formal explanation of these failures in the particular case of sparse reward and deterministic environments. First, using a very elementary control problem, we illustrate that the learning process can get stuck into a fixed point corresponding to a poor solution. Then, generalizing from the studied example, we provide a detailed analysis of the underlying mechanisms which results in a new understanding of one of the convergence regimes of these algorithms. The resulting perspective casts a new light on already existing solutions to the issues we have highlighted, and suggests other potential approaches. |
| Tasks | |
| Published | 2020-01-01 |
| URL | https://openreview.net/forum?id=HyxnH64KwS |
| https://openreview.net/pdf?id=HyxnH64KwS | |
| PWC | https://paperswithcode.com/paper/the-problem-with-ddpg-understanding-failures |
| Repo | |
| Framework | |
Semi-supervised Semantic Segmentation using Auxiliary Network
| Title | Semi-supervised Semantic Segmentation using Auxiliary Network |
| Authors | Anonymous |
| Abstract | Recently, the convolutional neural networks (CNNs) have shown great success on semantic segmentation task. However, for practical applications such as autonomous driving, the popular supervised learning method faces two challenges: the demand of low computational complexity and the need of huge training dataset accompanied by ground truth. Our focus in this paper is semi-supervised learning. We wish to use both labeled and unlabeled data in the training process. A highly efficient semantic segmentation network is our platform, which achieves high segmentation accuracy at low model size and high inference speed. We propose a semi-supervised learning approach to improve segmentation accuracy by including extra images without labels. While most existing semi-supervised learning methods are designed based on the adversarial learning techniques, we present a new and different approach, which trains an auxiliary CNN network that validates labels (ground-truth) on the unlabeled images. Therefore, in the supervised training phase, both the segmentation network and the auxiliary network are trained using labeled images. Then, in the unsupervised training phase, the unlabeled images are segmented and a subset of image pixels are picked up by the auxiliary network; and then they are used as ground truth to train the segmentation network. Thus, at the end, all dataset images can be used for retraining the segmentation network to improve the segmentation results. We use Cityscapes and CamVid datasets to verify the effectiveness of our semi-supervised scheme, and our experimental results show that it can improve the mean IoU for about 1.2% to 2.9% on the challenging Cityscapes dataset. |
| Tasks | Autonomous Driving, Semantic Segmentation, Semi-Supervised Semantic Segmentation |
| Published | 2020-01-01 |
| URL | https://openreview.net/forum?id=BkxFi2VYvS |
| https://openreview.net/pdf?id=BkxFi2VYvS | |
| PWC | https://paperswithcode.com/paper/semi-supervised-semantic-segmentation-using-1 |
| Repo | |
| Framework | |
AdvectiveNet: An Eulerian-Lagrangian Fluidic Reservoir for Point Cloud Processing
| Title | AdvectiveNet: An Eulerian-Lagrangian Fluidic Reservoir for Point Cloud Processing |
| Authors | Anonymous |
| Abstract | This paper presents a novel physics-inspired deep learning approach for point cloud processing motivated by the natural flow phenomena in fluid mechanics. Our learning architecture jointly defines data in an Eulerian world space, using a static background grid, and a Lagrangian material space, using moving particles. By introducing this Eulerian-Lagrangian representation, we are able to naturally evolve and accumulate particle features using flow velocities generated from a generalized, high-dimensional force field. We demonstrate the efficacy of this system by solving various point cloud classification and segmentation problems with state-of-the-art performance. The entire geometric reservoir and data flow mimic the pipeline of the classic PIC/FLIP scheme in modeling natural flow, bridging the disciplines of geometric machine learning and physical simulation. |
| Tasks | |
| Published | 2020-01-01 |
| URL | https://openreview.net/forum?id=H1eqQeHFDS |
| https://openreview.net/pdf?id=H1eqQeHFDS | |
| PWC | https://paperswithcode.com/paper/advectivenet-an-eulerian-lagrangian-fluidic |
| Repo | |
| Framework | |
Defective Convolutional Layers Learn Robust CNNs
| Title | Defective Convolutional Layers Learn Robust CNNs |
| Authors | Anonymous |
| Abstract | Robustness of convolutional neural networks has recently been highlighted by the adversarial examples, i.e., inputs added with well-designed perturbations which are imperceptible to humans but can cause the network to give incorrect outputs. Recent research suggests that the noises in adversarial examples break the textural structure, which eventually leads to wrong predictions by convolutional neural networks. To help a convolutional neural network make predictions relying less on textural information, we propose defective convolutional layers which contain defective neurons whose activations are set to be a constant function. As the defective neurons contain no information and are far different from the standard neurons in its spatial neighborhood, the textural features cannot be accurately extracted and the model has to seek for other features for classification, such as the shape. We first show that predictions made by the defective CNN are less dependent on textural information, but more on shape information, and further find that adversarial examples generated by the defective CNN appear to have semantic shapes. Experimental results demonstrate the defective CNN has higher defense ability than the standard CNN against various types of attack. In particular, it achieves state-of-the-art performance against transfer-based attacks without applying any adversarial training. |
| Tasks | |
| Published | 2020-01-01 |
| URL | https://openreview.net/forum?id=ryeQmCVYPS |
| https://openreview.net/pdf?id=ryeQmCVYPS | |
| PWC | https://paperswithcode.com/paper/defective-convolutional-layers-learn-robust |
| Repo | |
| Framework | |
Random Partition Relaxation for Training Binary and Ternary Weight Neural Network
| Title | Random Partition Relaxation for Training Binary and Ternary Weight Neural Network |
| Authors | Anonymous |
| Abstract | We present Random Partition Relaxation (RPR), a method for strong quantization of the parameters of convolutional neural networks to binary (+1/-1) and ternary (+1/0/-1) values. Starting from a pretrained model, we first quantize the weights and then relax random partitions of them to their continuous values for retraining before quantizing them again and switching to another weight partition for further adaptation. We empirically evaluate the performance of RPR with ResNet-18, ResNet-50 and GoogLeNet on the ImageNet classification task for binary and ternary weight networks. We show accuracies beyond the state-of-the-art for binary- and ternary-weight GoogLeNet and competitive performance for ResNet-18 and ResNet-50 using a SGD-based training method that can easily be integrated into existing frameworks. |
| Tasks | Quantization |
| Published | 2020-01-01 |
| URL | https://openreview.net/forum?id=S1lvWeBFwB |
| https://openreview.net/pdf?id=S1lvWeBFwB | |
| PWC | https://paperswithcode.com/paper/random-partition-relaxation-for-training |
| Repo | |
| Framework | |
