April 2, 2020

3334 words 16 mins read

Paper Group ANR 336

Uncertainty Principle for Communication Compression in Distributed and Federated Learning and the Search for an Optimal Compressor. End-To-End Graph-based Deep Semi-Supervised Learning. Adversarial Camouflage: Hiding Physical-World Attacks with Natural Styles. Unshuffling Data for Improved Generalization. Dynamic Backdoor Attacks Against Machine Le …

Uncertainty Principle for Communication Compression in Distributed and Federated Learning and the Search for an Optimal Compressor


Title	Uncertainty Principle for Communication Compression in Distributed and Federated Learning and the Search for an Optimal Compressor
Authors	Mher Safaryan, Egor Shulgin, Peter Richtárik
Abstract	In order to mitigate the high communication cost in distributed and federated learning, various vector compression schemes, such as quantization, sparsification and dithering, have become very popular. In designing a compression method, one aims to communicate as few bits as possible, which minimizes the cost per communication round, while at the same time attempting to impart as little distortion (variance) to the communicated messages as possible, which minimizes the adverse effect of the compression on the overall number of communication rounds. However, intuitively, these two goals are fundamentally in conflict: the more compression we allow, the more distorted the messages become. We formalize this intuition and prove an {\em uncertainty principle} for randomized compression operators, thus quantifying this limitation mathematically, and {\em effectively providing lower bounds on what might be achievable with communication compression}. Motivated by these developments, we call for the search for the optimal compression operator. In an attempt to take a first step in this direction, we construct a new unbiased compression method inspired by the Kashin representation of vectors, which we call {\em Kashin compression (KC)}. In contrast to all previously proposed compression mechanisms, we prove that KC enjoys a {\em dimension independent} variance bound with an explicit formula even in the regime when only a few bits need to be communicate per each vector entry. We show how KC can be provably and efficiently combined with several existing optimization algorithms, in all cases leading to communication complexity improvements on previous state of the art.
Tasks	Quantization
Published	2020-02-20
URL	https://arxiv.org/abs/2002.08958v1
PDF	https://arxiv.org/pdf/2002.08958v1.pdf
PWC	https://paperswithcode.com/paper/uncertainty-principle-for-communication
Repo
Framework

End-To-End Graph-based Deep Semi-Supervised Learning


Title	End-To-End Graph-based Deep Semi-Supervised Learning
Authors	Zihao Wang, Enmei Tu, Zhou Meng
Abstract	The quality of a graph is determined jointly by three key factors of the graph: nodes, edges and similarity measure (or edge weights), and is very crucial to the success of graph-based semi-supervised learning (SSL) approaches. Recently, dynamic graph, which means part/all its factors are dynamically updated during the training process, has demonstrated to be promising for graph-based semi-supervised learning. However, existing approaches only update part of the three factors and keep the rest manually specified during the learning stage. In this paper, we propose a novel graph-based semi-supervised learning approach to optimize all three factors simultaneously in an end-to-end learning fashion. To this end, we concatenate two neural networks (feature network and similarity network) together to learn the categorical label and semantic similarity, respectively, and train the networks to minimize a unified SSL objective function. We also introduce an extended graph Laplacian regularization term to increase training efficiency. Extensive experiments on several benchmark datasets demonstrate the effectiveness of our approach.
Tasks	Semantic Similarity, Semantic Textual Similarity
Published	2020-02-23
URL	https://arxiv.org/abs/2002.09891v1
PDF	https://arxiv.org/pdf/2002.09891v1.pdf
PWC	https://paperswithcode.com/paper/end-to-end-graph-based-deep-semi-supervised
Repo
Framework

Adversarial Camouflage: Hiding Physical-World Attacks with Natural Styles


Title	Adversarial Camouflage: Hiding Physical-World Attacks with Natural Styles
Authors	Ranjie Duan, Xingjun Ma, Yisen Wang, James Bailey, A. K. Qin, Yun Yang
Abstract	Deep neural networks (DNNs) are known to be vulnerable to adversarial examples. Existing works have mostly focused on either digital adversarial examples created via small and imperceptible perturbations, or physical-world adversarial examples created with large and less realistic distortions that are easily identified by human observers. In this paper, we propose a novel approach, called Adversarial Camouflage (\emph{AdvCam}), to craft and camouflage physical-world adversarial examples into natural styles that appear legitimate to human observers. Specifically, \emph{AdvCam} transfers large adversarial perturbations into customized styles, which are then “hidden” on-target object or off-target background. Experimental evaluation shows that, in both digital and physical-world scenarios, adversarial examples crafted by \emph{AdvCam} are well camouflaged and highly stealthy, while remaining effective in fooling state-of-the-art DNN image classifiers. Hence, \emph{AdvCam} is a flexible approach that can help craft stealthy attacks to evaluate the robustness of DNNs. \emph{AdvCam} can also be used to protect private information from being detected by deep learning systems.
Tasks
Published	2020-03-08
URL	https://arxiv.org/abs/2003.08757v1
PDF	https://arxiv.org/pdf/2003.08757v1.pdf
PWC	https://paperswithcode.com/paper/adversarial-camouflage-hiding-physical-world
Repo
Framework

Unshuffling Data for Improved Generalization


Title	Unshuffling Data for Improved Generalization
Authors	Damien Teney, Ehsan Abbasnejad, Anton van den Hengel
Abstract	The inability to generalize beyond the distribution of a training set is at the core of practical limits of machine learning. We show that the common practice of mixing and shuffling training examples when training deep neural networks is not optimal. On the opposite, partitioning the training data into non-i.i.d. subsets can serve to guide the model to rely on reliable statistical patterns while ignoring spurious correlations in the training data. We demonstrate multiple use cases where these subsets are built using unsupervised clustering, prior knowledge, or other meta-data from existing datasets. The approach is supported by recent results on a causal view of generalization, it is simple to apply, and it demonstrably improves generalization. Applied to the task of visual question answering, we obtain state-of-the-art performance on VQA-CP. We also show improvements over data augmentation using equivalent questions on GQA. Finally, we show a small improvement when training a model simultaneously on VQA v2 and Visual Genome, treating them as two distinct environments rather than one aggregated training set.
Tasks	Data Augmentation, Question Answering, Visual Question Answering
Published	2020-02-27
URL	https://arxiv.org/abs/2002.11894v2
PDF	https://arxiv.org/pdf/2002.11894v2.pdf
PWC	https://paperswithcode.com/paper/unshuffling-data-for-improved-generalization
Repo
Framework

Dynamic Backdoor Attacks Against Machine Learning Models


Title	Dynamic Backdoor Attacks Against Machine Learning Models
Authors	Ahmed Salem, Rui Wen, Michael Backes, Shiqing Ma, Yang Zhang
Abstract	Machine learning (ML) has made tremendous progress during the past decade and is being adopted in various critical real-world applications. However, recent research has shown that ML models are vulnerable to multiple security and privacy attacks. In particular, backdoor attacks against ML models that have recently raised a lot of awareness. A successful backdoor attack can cause severe consequences, such as allowing an adversary to bypass critical authentication systems. Current backdooring techniques rely on adding static triggers (with fixed patterns and locations) on ML model inputs. In this paper, we propose the first class of dynamic backdooring techniques: Random Backdoor, Backdoor Generating Network (BaN), and conditional Backdoor Generating Network (c-BaN). Triggers generated by our techniques can have random patterns and locations, which reduce the efficacy of the current backdoor detection mechanisms. In particular, BaN and c-BaN are the first two schemes that algorithmically generate triggers, which rely on a novel generative network. Moreover, c-BaN is the first conditional backdooring technique, that given a target label, it can generate a target-specific trigger. Both BaN and c-BaN are essentially a general framework which renders the adversary the flexibility for further customizing backdoor attacks. We extensively evaluate our techniques on three benchmark datasets: MNIST, CelebA, and CIFAR-10. Our techniques achieve almost perfect attack performance on backdoored data with a negligible utility loss. We further show that our techniques can bypass current state-of-the-art defense mechanisms against backdoor attacks, including Neural Cleanse, ABS, and STRIP.
Tasks
Published	2020-03-07
URL	https://arxiv.org/abs/2003.03675v1
PDF	https://arxiv.org/pdf/2003.03675v1.pdf
PWC	https://paperswithcode.com/paper/dynamic-backdoor-attacks-against-machine
Repo
Framework

Adversarial Machine Learning – Industry Perspectives


Title	Adversarial Machine Learning – Industry Perspectives
Authors	Ram Shankar Siva Kumar, Magnus Nyström, John Lambert, Andrew Marshall, Mario Goertzel, Andi Comissoneru, Matt Swann, Sharon Xia
Abstract	Based on interviews with 28 organizations, we found that industry practitioners are not equipped with tactical and strategic tools to protect, detect and respond to attacks on their Machine Learning (ML) systems. We leverage the insights from the interviews and we enumerate the gaps in perspective in securing machine learning systems when viewed in the context of traditional software security development. We write this paper from the perspective of two personas: developers/ML engineers and security incident responders who are tasked with securing ML systems as they are designed, developed and deployed ML systems. The goal of this paper is to engage researchers to revise and amend the Security Development Lifecycle for industrial-grade software in the adversarial ML era.
Tasks
Published	2020-02-04
URL	https://arxiv.org/abs/2002.05646v1
PDF	https://arxiv.org/pdf/2002.05646v1.pdf
PWC	https://paperswithcode.com/paper/adversarial-machine-learning-industry
Repo
Framework

Fast Symmetric Diffeomorphic Image Registration with Convolutional Neural Networks


Title	Fast Symmetric Diffeomorphic Image Registration with Convolutional Neural Networks
Authors	Tony C. W. Mok, Albert C. S. Chung
Abstract	Diffeomorphic deformable image registration is crucial in many medical image studies, as it offers unique, special properties including topology preservation and invertibility of the transformation. Recent deep learning-based deformable image registration methods achieve fast image registration by leveraging a convolutional neural network (CNN) to learn the spatial transformation from the synthetic ground truth or the similarity metric. However, these approaches often ignore the topology preservation of the transformation and the smoothness of the transformation which is enforced by a global smoothing energy function alone. Moreover, deep learning-based approaches often estimate the displacement field directly, which cannot guarantee the existence of the inverse transformation. In this paper, we present a novel, efficient unsupervised symmetric image registration method which maximizes the similarity between images within the space of diffeomorphic maps and estimates both forward and inverse transformations simultaneously. We evaluate our method on 3D image registration with a large scale brain image dataset. Our method achieves state-of-the-art registration accuracy and running time while maintaining desirable diffeomorphic properties.
Tasks	Image Registration
Published	2020-03-20
URL	https://arxiv.org/abs/2003.09514v1
PDF	https://arxiv.org/pdf/2003.09514v1.pdf
PWC	https://paperswithcode.com/paper/fast-symmetric-diffeomorphic-image
Repo
Framework

TanksWorld: A Multi-Agent Environment for AI Safety Research


Title	TanksWorld: A Multi-Agent Environment for AI Safety Research
Authors	Corban G. Rivera, Olivia Lyons, Arielle Summitt, Ayman Fatima, Ji Pak, William Shao, Robert Chalmers, Aryeh Englander, Edward W. Staley, I-Jeng Wang, Ashley J. Llorens
Abstract	The ability to create artificial intelligence (AI) capable of performing complex tasks is rapidly outpacing our ability to ensure the safe and assured operation of AI-enabled systems. Fortunately, a landscape of AI safety research is emerging in response to this asymmetry and yet there is a long way to go. In particular, recent simulation environments created to illustrate AI safety risks are relatively simple or narrowly-focused on a particular issue. Hence, we see a critical need for AI safety research environments that abstract essential aspects of complex real-world applications. In this work, we introduce the AI safety TanksWorld as an environment for AI safety research with three essential aspects: competing performance objectives, human-machine teaming, and multi-agent competition. The AI safety TanksWorld aims to accelerate the advancement of safe multi-agent decision-making algorithms by providing a software framework to support competitions with both system performance and safety objectives. As a work in progress, this paper introduces our research objectives and learning environment with reference code and baseline performance metrics to follow in a future work.
Tasks	Decision Making
Published	2020-02-25
URL	https://arxiv.org/abs/2002.11174v1
PDF	https://arxiv.org/pdf/2002.11174v1.pdf
PWC	https://paperswithcode.com/paper/tanksworld-a-multi-agent-environment-for-ai
Repo
Framework

LEAP System for SRE19 Challenge – Improvements and Error Analysis


Title	LEAP System for SRE19 Challenge – Improvements and Error Analysis
Authors	Shreyas Ramoji, Prashant Krishnan, Bhargavram Mysore, Prachi Singh, Sriram Ganapathy
Abstract	The NIST Speaker Recognition Evaluation - Conversational Telephone Speech (CTS) challenge 2019 was an open evaluation for the task of speaker verification in challenging conditions. In this paper, we provide a detailed account of the LEAP SRE system submitted to the CTS challenge focusing on the novel components in the back-end system modeling. All the systems used the time-delay neural network (TDNN) based x-vector embeddings. The x-vector system in our SRE19 submission used a large pool of training speakers (about 14k speakers). Following the x-vector extraction, we explored a neural network approach to backend score computation that was optimized for a speaker verification cost. The system combination of generative and neural PLDA models resulted in significant improvements for the SRE evaluation dataset. We also found additional gains for the SRE systems based on score normalization and calibration. Subsequent to the evaluations, we have performed a detailed analysis of the submitted systems. The analysis revealed the incremental gains obtained for different training dataset combinations as well as the modeling methods.
Tasks	Calibration, Speaker Recognition, Speaker Verification
Published	2020-02-07
URL	https://arxiv.org/abs/2002.02735v1
PDF	https://arxiv.org/pdf/2002.02735v1.pdf
PWC	https://paperswithcode.com/paper/leap-system-for-sre19-challenge-improvements
Repo
Framework

High Performance Logistic Regression for Privacy-Preserving Genome Analysis


Title	High Performance Logistic Regression for Privacy-Preserving Genome Analysis
Authors	Martine De Cock, Rafael Dowsley, Anderson C. A. Nascimento, Davis Railsback, Jianwei Shen, Ariel Todoki
Abstract	In this paper, we present a secure logistic regression training protocol and its implementation, with a new subprotocol to securely compute the activation function. To the best of our knowledge, we present the fastest existing secure Multi-Party Computation implementation for training logistic regression models on high dimensional genome data distributed across a local area network.
Tasks
Published	2020-02-13
URL	https://arxiv.org/abs/2002.05377v2
PDF	https://arxiv.org/pdf/2002.05377v2.pdf
PWC	https://paperswithcode.com/paper/high-performance-logistic-regression-for
Repo
Framework

A Question-Centric Model for Visual Question Answering in Medical Imaging


Title	A Question-Centric Model for Visual Question Answering in Medical Imaging
Authors	Minh H. Vu, Tommy Löfstedt, Tufve Nyholm, Raphael Sznitman
Abstract	Deep learning methods have proven extremely effective at performing a variety of medical image analysis tasks. With their potential use in clinical routine, their lack of transparency has however been one of their few weak points, raising concerns regarding their behavior and failure modes. While most research to infer model behavior has focused on indirect strategies that estimate prediction uncertainties and visualize model support in the input image space, the ability to explicitly query a prediction model regarding its image content offers a more direct way to determine the behavior of trained models. To this end, we present a novel Visual Question Answering approach that allows an image to be queried by means of a written question. Experiments on a variety of medical and natural image datasets show that by fusing image and question features in a novel way, the proposed approach achieves an equal or higher accuracy compared to current methods.
Tasks	Question Answering, Visual Question Answering
Published	2020-03-02
URL	https://arxiv.org/abs/2003.08760v1
PDF	https://arxiv.org/pdf/2003.08760v1.pdf
PWC	https://paperswithcode.com/paper/a-question-centric-model-for-visual-question
Repo
Framework

Stereotypical Bias Removal for Hate Speech Detection Task using Knowledge-based Generalizations


Title	Stereotypical Bias Removal for Hate Speech Detection Task using Knowledge-based Generalizations
Authors	Pinkesh Badjatiya, Manish Gupta, Vasudeva Varma
Abstract	With the ever-increasing cases of hate spread on social media platforms, it is critical to design abuse detection mechanisms to proactively avoid and control such incidents. While there exist methods for hate speech detection, they stereotype words and hence suffer from inherently biased training. Bias removal has been traditionally studied for structured datasets, but we aim at bias mitigation from unstructured text data. In this paper, we make two important contributions. First, we systematically design methods to quantify the bias for any model and propose algorithms for identifying the set of words which the model stereotypes. Second, we propose novel methods leveraging knowledge-based generalizations for bias-free learning. Knowledge-based generalization provides an effective way to encode knowledge because the abstraction they provide not only generalizes content but also facilitates retraction of information from the hate speech detection classifier, thereby reducing the imbalance. We experiment with multiple knowledge generalization policies and analyze their effect on general performance and in mitigating bias. Our experiments with two real-world datasets, a Wikipedia Talk Pages dataset (WikiDetox) of size ~96k and a Twitter dataset of size ~24k, show that the use of knowledge-based generalizations results in better performance by forcing the classifier to learn from generalized content. Our methods utilize existing knowledge-bases and can easily be extended to other tasks
Tasks	Abuse Detection, Hate Speech Detection
Published	2020-01-15
URL	https://arxiv.org/abs/2001.05495v1
PDF	https://arxiv.org/pdf/2001.05495v1.pdf
PWC	https://paperswithcode.com/paper/stereotypical-bias-removal-for-hate-speech
Repo
Framework

A Robust Imbalanced SAR Image Change Detection Approach Based on Deep Difference Image and PCANet


Title	A Robust Imbalanced SAR Image Change Detection Approach Based on Deep Difference Image and PCANet
Authors	Xinzheng Zhang, Hang Su, Ce Zhang, Peter M. Atkinson, Xiaoheng Tan, Xiaoping Zeng, Xin Jian
Abstract	In this research, a novel robust change detection approach is presented for imbalanced multi-temporal synthetic aperture radar (SAR) image based on deep learning. Our main contribution is to develop a novel method for generating difference image and a parallel fuzzy c-means (FCM) clustering method. The main steps of our proposed approach are as follows: 1) Inspired by convolution and pooling in deep learning, a deep difference image (DDI) is obtained based on parameterized pooling leading to better speckle suppression and feature enhancement than traditional difference images. 2) Two different parameter Sigmoid nonlinear mapping are applied to the DDI to get two mapped DDIs. Parallel FCM are utilized on these two mapped DDIs to obtain three types of pseudo-label pixels, namely, changed pixels, unchanged pixels, and intermediate pixels. 3) A PCANet with support vector machine (SVM) are trained to classify intermediate pixels to be changed or unchanged. Three imbalanced multi-temporal SAR image sets are used for change detection experiments. The experimental results demonstrate that the proposed approach is effective and robust for imbalanced SAR data, and achieve up to 99.52% change detection accuracy superior to most state-of-the-art methods.
Tasks
Published	2020-03-03
URL	https://arxiv.org/abs/2003.01768v1
PDF	https://arxiv.org/pdf/2003.01768v1.pdf
PWC	https://paperswithcode.com/paper/a-robust-imbalanced-sar-image-change
Repo
Framework

Deep reinforcement learning for large-scale epidemic control


Title	Deep reinforcement learning for large-scale epidemic control
Authors	Pieter Libin, Arno Moonens, Timothy Verstraeten, Fabian Perez-Sanjines, Niel Hens, Philippe Lemey, Ann Nowé
Abstract	Epidemics of infectious diseases are an important threat to public health and global economies. Yet, the development of prevention strategies remains a challenging process, as epidemics are non-linear and complex processes. For this reason, we investigate a deep reinforcement learning approach to automatically learn prevention strategies in the context of pandemic influenza. Firstly, we construct a new epidemiological meta-population model, with 379 patches (one for each administrative district in Great Britain), that adequately captures the infection process of pandemic influenza. Our model balances complexity and computational efficiency such that the use of reinforcement learning techniques becomes attainable. Secondly, we set up a ground truth such that we can evaluate the performance of the ‘Proximal Policy Optimization’ algorithm to learn in a single district of this epidemiological model. Finally, we consider a large-scale problem, by conducting an experiment where we aim to learn a joint policy to control the districts in a community of 11 tightly coupled districts, for which no ground truth can be established. This experiment shows that deep reinforcement learning can be used to learn mitigation policies in complex epidemiological models with a large state space. Moreover, through this experiment, we demonstrate that there can be an advantage to consider collaboration between districts when designing prevention strategies.
Tasks
Published	2020-03-30
URL	https://arxiv.org/abs/2003.13676v1
PDF	https://arxiv.org/pdf/2003.13676v1.pdf
PWC	https://paperswithcode.com/paper/deep-reinforcement-learning-for-large-scale
Repo
Framework

Stable Training of DNN for Speech Enhancement based on Perceptually-Motivated Black-Box Cost Function


Title	Stable Training of DNN for Speech Enhancement based on Perceptually-Motivated Black-Box Cost Function
Authors	Masaki Kawanaka, Yuma Koizumi, Ryoichi Miyazaki, Kohei Yatabe
Abstract	Improving subjective sound quality of enhanced signals is one of the most important missions in speech enhancement. For evaluating the subjective quality, several methods related to perceptually-motivated objective sound quality assessment (OSQA) have been proposed such as PESQ (perceptual evaluation of speech quality). However, direct use of such measures for training deep neural network (DNN) is not allowed in most cases because popular OSQAs are non-differentiable with respect to DNN parameters. Therefore, the previous study has proposed to approximate the score of OSQAs by an auxiliary DNN so that its gradient can be used for training the primary DNN. One problem with this approach is instability of the training caused by the approximation error of the score. To overcome this problem, we propose to use stabilization techniques borrowed from reinforcement learning. The experiments, aimed to increase the score of PESQ as an example, show that the proposed method (i) can stably train a DNN to increase PESQ, (ii) achieved the state-of-the-art PESQ score on a public dataset, and (iii) resulted in better sound quality than conventional methods based on subjective evaluation.
Tasks	Speech Enhancement
Published	2020-02-14
URL	https://arxiv.org/abs/2002.05879v1
PDF	https://arxiv.org/pdf/2002.05879v1.pdf
PWC	https://paperswithcode.com/paper/stable-training-of-dnn-for-speech-enhancement
Repo
Framework