October 20, 2019

3032 words 15 mins read

Paper Group AWR 255

With Friends Like These, Who Needs Adversaries?. Local Gradients Smoothing: Defense against localized adversarial attacks. Unsupervised Sense-Aware Hypernymy Extraction. Task Agnostic Continual Learning Using Online Variational Bayes. Knowledge Distillation with Adversarial Samples Supporting Decision Boundary. Autoconj: Recognizing and Exploiting …

With Friends Like These, Who Needs Adversaries?


Title	With Friends Like These, Who Needs Adversaries?
Authors	Saumya Jetley, Nicholas A. Lord, Philip H. S. Torr
Abstract	The vulnerability of deep image classification networks to adversarial attack is now well known, but less well understood. Via a novel experimental analysis, we illustrate some facts about deep convolutional networks for image classification that shed new light on their behaviour and how it connects to the problem of adversaries. In short, the celebrated performance of these networks and their vulnerability to adversarial attack are simply two sides of the same coin: the input image-space directions along which the networks are most vulnerable to attack are the same directions which they use to achieve their classification performance in the first place. We develop this result in two main steps. The first uncovers the fact that classes tend to be associated with specific image-space directions. This is shown by an examination of the class-score outputs of nets as functions of 1D movements along these directions. This provides a novel perspective on the existence of universal adversarial perturbations. The second is a clear demonstration of the tight coupling between classification performance and vulnerability to adversarial attack within the spaces spanned by these directions. Thus, our analysis resolves the apparent contradiction between accuracy and vulnerability. It provides a new perspective on much of the prior art and reveals profound implications for efforts to construct neural nets that are both accurate and robust to adversarial attack.
Tasks	Adversarial Attack, Image Classification
Published	2018-07-11
URL	http://arxiv.org/abs/1807.04200v4
PDF	http://arxiv.org/pdf/1807.04200v4.pdf
PWC	https://paperswithcode.com/paper/with-friends-like-these-who-needs-adversaries
Repo	https://github.com/torrvision/whoneedsadversaries
Framework	none

Local Gradients Smoothing: Defense against localized adversarial attacks


Title	Local Gradients Smoothing: Defense against localized adversarial attacks
Authors	Muzammal Naseer, Salman H. Khan, Fatih Porikli
Abstract	Deep neural networks (DNNs) have shown vulnerability to adversarial attacks, i.e., carefully perturbed inputs designed to mislead the network at inference time. Recently introduced localized attacks, Localized and Visible Adversarial Noise (LaVAN) and Adversarial patch, pose a new challenge to deep learning security by adding adversarial noise only within a specific region without affecting the salient objects in an image. Driven by the observation that such attacks introduce concentrated high-frequency changes at a particular image location, we have developed an effective method to estimate noise location in gradient domain and transform those high activation regions caused by adversarial noise in image domain while having minimal effect on the salient object that is important for correct classification. Our proposed Local Gradients Smoothing (LGS) scheme achieves this by regularizing gradients in the estimated noisy region before feeding the image to DNN for inference. We have shown the effectiveness of our method in comparison to other defense methods including Digital Watermarking, JPEG compression, Total Variance Minimization (TVM) and Feature squeezing on ImageNet dataset. In addition, we systematically study the robustness of the proposed defense mechanism against Back Pass Differentiable Approximation (BPDA), a state of the art attack recently developed to break defenses that transform an input sample to minimize the adversarial effect. Compared to other defense mechanisms, LGS is by far the most resistant to BPDA in localized adversarial attack setting.
Tasks	Adversarial Attack
Published	2018-07-03
URL	http://arxiv.org/abs/1807.01216v2
PDF	http://arxiv.org/pdf/1807.01216v2.pdf
PWC	https://paperswithcode.com/paper/local-gradients-smoothing-defense-against
Repo	https://github.com/Ping-C/certifiedpatchdefense
Framework	pytorch

Unsupervised Sense-Aware Hypernymy Extraction


Title	Unsupervised Sense-Aware Hypernymy Extraction
Authors	Dmitry Ustalov, Alexander Panchenko, Chris Biemann, Simone Paolo Ponzetto
Abstract	In this paper, we show how unsupervised sense representations can be used to improve hypernymy extraction. We present a method for extracting disambiguated hypernymy relationships that propagates hypernyms to sets of synonyms (synsets), constructs embeddings for these sets, and establishes sense-aware relationships between matching synsets. Evaluation on two gold standard datasets for English and Russian shows that the method successfully recognizes hypernymy relationships that cannot be found with standard Hearst patterns and Wiktionary datasets for the respective languages.
Tasks
Published	2018-09-17
URL	http://arxiv.org/abs/1809.06223v1
PDF	http://arxiv.org/pdf/1809.06223v1.pdf
PWC	https://paperswithcode.com/paper/unsupervised-sense-aware-hypernymy-extraction
Repo	https://github.com/dustalov/watlink
Framework	none

Task Agnostic Continual Learning Using Online Variational Bayes


Title	Task Agnostic Continual Learning Using Online Variational Bayes
Authors	Chen Zeno, Itay Golan, Elad Hoffer, Daniel Soudry
Abstract	Catastrophic forgetting is the notorious vulnerability of neural networks to the change of the data distribution while learning. This phenomenon has long been considered a major obstacle for allowing the use of learning agents in realistic continual learning settings. A large body of continual learning research assumes that task boundaries are known during training. However, research for scenarios in which task boundaries are unknown during training has been lacking. In this paper we present, for the first time, a method for preventing catastrophic forgetting (BGD) for scenarios with task boundaries that are unknown during training — task-agnostic continual learning. Code of our algorithm is available at https://github.com/igolan/bgd.
Tasks	Continual Learning
Published	2018-03-27
URL	http://arxiv.org/abs/1803.10123v3
PDF	http://arxiv.org/pdf/1803.10123v3.pdf
PWC	https://paperswithcode.com/paper/task-agnostic-continual-learning-using-online
Repo	https://github.com/taldatech/tf-bgd
Framework	tf

Knowledge Distillation with Adversarial Samples Supporting Decision Boundary


Title	Knowledge Distillation with Adversarial Samples Supporting Decision Boundary
Authors	Byeongho Heo, Minsik Lee, Sangdoo Yun, Jin Young Choi
Abstract	Many recent works on knowledge distillation have provided ways to transfer the knowledge of a trained network for improving the learning process of a new one, but finding a good technique for knowledge distillation is still an open problem. In this paper, we provide a new perspective based on a decision boundary, which is one of the most important component of a classifier. The generalization performance of a classifier is closely related to the adequacy of its decision boundary, so a good classifier bears a good decision boundary. Therefore, transferring information closely related to the decision boundary can be a good attempt for knowledge distillation. To realize this goal, we utilize an adversarial attack to discover samples supporting a decision boundary. Based on this idea, to transfer more accurate information about the decision boundary, the proposed algorithm trains a student classifier based on the adversarial samples supporting the decision boundary. Experiments show that the proposed method indeed improves knowledge distillation and achieves the state-of-the-arts performance.
Tasks	Adversarial Attack
Published	2018-05-15
URL	http://arxiv.org/abs/1805.05532v4
PDF	http://arxiv.org/pdf/1805.05532v4.pdf
PWC	https://paperswithcode.com/paper/knowledge-distillation-with-adversarial
Repo	https://github.com/bhheo/BSS_distillation
Framework	pytorch

Autoconj: Recognizing and Exploiting Conjugacy Without a Domain-Specific Language


Title	Autoconj: Recognizing and Exploiting Conjugacy Without a Domain-Specific Language
Authors	Matthew D. Hoffman, Matthew J. Johnson, Dustin Tran
Abstract	Deriving conditional and marginal distributions using conjugacy relationships can be time consuming and error prone. In this paper, we propose a strategy for automating such derivations. Unlike previous systems which focus on relationships between pairs of random variables, our system (which we call Autoconj) operates directly on Python functions that compute log-joint distribution functions. Autoconj provides support for conjugacy-exploiting algorithms in any Python embedded PPL. This paves the way for accelerating development of novel inference algorithms and structure-exploiting modeling strategies.
Tasks
Published	2018-11-29
URL	http://arxiv.org/abs/1811.11926v1
PDF	http://arxiv.org/pdf/1811.11926v1.pdf
PWC	https://paperswithcode.com/paper/autoconj-recognizing-and-exploiting-conjugacy
Repo	https://github.com/pyro-ppl/funsor
Framework	pytorch

Learning Representations of Ultrahigh-dimensional Data for Random Distance-based Outlier Detection


Title	Learning Representations of Ultrahigh-dimensional Data for Random Distance-based Outlier Detection
Authors	Guansong Pang, Longbing Cao, Ling Chen, Huan Liu
Abstract	Learning expressive low-dimensional representations of ultrahigh-dimensional data, e.g., data with thousands/millions of features, has been a major way to enable learning methods to address the curse of dimensionality. However, existing unsupervised representation learning methods mainly focus on preserving the data regularity information and learning the representations independently of subsequent outlier detection methods, which can result in suboptimal and unstable performance of detecting irregularities (i.e., outliers). This paper introduces a ranking model-based framework, called RAMODO, to address this issue. RAMODO unifies representation learning and outlier detection to learn low-dimensional representations that are tailored for a state-of-the-art outlier detection approach - the random distance-based approach. This customized learning yields more optimal and stable representations for the targeted outlier detectors. Additionally, RAMODO can leverage little labeled data as prior knowledge to learn more expressive and application-relevant representations. We instantiate RAMODO to an efficient method called REPEN to demonstrate the performance of RAMODO. Extensive empirical results on eight real-world ultrahigh dimensional data sets show that REPEN (i) enables a random distance-based detector to obtain significantly better AUC performance and two orders of magnitude speedup; (ii) performs substantially better and more stably than four state-of-the-art representation learning methods; and (iii) leverages less than 1% labeled data to achieve up to 32% AUC improvement.
Tasks	Anomaly Detection, Disease Prediction, Network Intrusion Detection, Outlier Detection, Representation Learning, Unsupervised Representation Learning
Published	2018-06-13
URL	http://arxiv.org/abs/1806.04808v1
PDF	http://arxiv.org/pdf/1806.04808v1.pdf
PWC	https://paperswithcode.com/paper/learning-representations-of-ultrahigh
Repo	https://github.com/GuansongPang/deep-outlier-detection
Framework	tf

MRPC: An R package for accurate inference of causal graphs


Title	MRPC: An R package for accurate inference of causal graphs
Authors	Md. Bahadur Badsha, Evan A Martin, Audrey Qiuyan Fu
Abstract	We present MRPC, an R package that learns causal graphs with improved accuracy over existing packages, such as pcalg and bnlearn. Our algorithm builds on the powerful PC algorithm, the canonical algorithm in computer science for learning directed acyclic graphs. The improvement in accuracy results from online control of the false discovery rate (FDR) that reduces false positive edges, a more accurate approach to identifying v-structures (i.e., $T_1 \rightarrow T_2 \leftarrow T_3$), and robust estimation of the correlation matrix among nodes. For genomic data that contain genotypes and gene expression for each sample, MRPC incorporates the principle of Mendelian randomization to orient the edges. Our package can be applied to continuous and discrete data.
Tasks
Published	2018-06-05
URL	http://arxiv.org/abs/1806.01899v1
PDF	http://arxiv.org/pdf/1806.01899v1.pdf
PWC	https://paperswithcode.com/paper/mrpc-an-r-package-for-accurate-inference-of
Repo	https://github.com/audreyqyfu/mrpc
Framework	none

Investigation of enhanced Tacotron text-to-speech synthesis systems with self-attention for pitch accent language


Title	Investigation of enhanced Tacotron text-to-speech synthesis systems with self-attention for pitch accent language
Authors	Yusuke Yasuda, Xin Wang, Shinji Takaki, Junichi Yamagishi
Abstract	End-to-end speech synthesis is a promising approach that directly converts raw text to speech. Although it was shown that Tacotron2 outperforms classical pipeline systems with regards to naturalness in English, its applicability to other languages is still unknown. Japanese could be one of the most difficult languages for which to achieve end-to-end speech synthesis, largely due to its character diversity and pitch accents. Therefore, state-of-the-art systems are still based on a traditional pipeline framework that requires a separate text analyzer and duration model. Towards end-to-end Japanese speech synthesis, we extend Tacotron to systems with self-attention to capture long-term dependencies related to pitch accents and compare their audio quality with classical pipeline systems under various conditions to show their pros and cons. In a large-scale listening test, we investigated the impacts of the presence of accentual-type labels, the use of force or predicted alignments, and acoustic features used as local condition parameters of the Wavenet vocoder. Our results reveal that although the proposed systems still do not match the quality of a top-line pipeline system for Japanese, we show important stepping stones towards end-to-end Japanese speech synthesis.
Tasks	Speech Synthesis, Text-To-Speech Synthesis
Published	2018-10-29
URL	http://arxiv.org/abs/1810.11960v2
PDF	http://arxiv.org/pdf/1810.11960v2.pdf
PWC	https://paperswithcode.com/paper/investigation-of-enhanced-tacotron-text-to
Repo	https://github.com/nii-yamagishilab/self-attention-tacotron
Framework	tf

A Language for Function Signature Representations


Title	A Language for Function Signature Representations
Authors	Kyle Richardson
Abstract	Recent work by (Richardson and Kuhn, 2017a,b; Richardson et al., 2018) looks at semantic parser induction and question answering in the domain of source code libraries and APIs. In this brief note, we formalize the representations being learned in these studies and introduce a simple domain specific language and a systematic translation from this language to first-order logic. By recasting the target representations in terms of classical logic, we aim to broaden the applicability of existing code datasets for investigating more complex natural language understanding and reasoning problems in the software domain.
Tasks	Question Answering
Published	2018-03-31
URL	http://arxiv.org/abs/1804.00987v2
PDF	http://arxiv.org/pdf/1804.00987v2.pdf
PWC	https://paperswithcode.com/paper/a-language-for-function-signature
Repo	https://github.com/yakazimir/Code-Datasets
Framework	none

Response Generation by Context-aware Prototype Editing


Title	Response Generation by Context-aware Prototype Editing
Authors	Yu Wu, Furu Wei, Shaohan Huang, Yunli Wang, Zhoujun Li, Ming Zhou
Abstract	Open domain response generation has achieved remarkable progress in recent years, but sometimes yields short and uninformative responses. We propose a new paradigm for response generation, that is response generation by editing, which significantly increases the diversity and informativeness of the generation results. Our assumption is that a plausible response can be generated by slightly revising an existing response prototype. The prototype is retrieved from a pre-defined index and provides a good start-point for generation because it is grammatical and informative. We design a response editing model, where an edit vector is formed by considering differences between a prototype context and a current context, and then the edit vector is fed to a decoder to revise the prototype response for the current context. Experiment results on a large scale dataset demonstrate that the response editing model outperforms generative and retrieval-based models on various aspects.
Tasks
Published	2018-06-19
URL	http://arxiv.org/abs/1806.07042v4
PDF	http://arxiv.org/pdf/1806.07042v4.pdf
PWC	https://paperswithcode.com/paper/response-generation-by-context-aware
Repo	https://github.com/jimth001/Bi-Seq2Seq
Framework	tf

A 2D laser rangefinder scans dataset of standard EUR pallets


Title	A 2D laser rangefinder scans dataset of standard EUR pallets
Authors	Ihab S. Mohamed, Alessio Capitanelli, Fulvio Mastrogiovanni, Stefano Rovetta, Renato Zaccaria
Abstract	In the past few years, the technology of automated guided vehicles (AGVs) has notably advanced. In particular, in the context of factory and warehouse automation, different approaches have been presented for detecting and localizing pallets inside warehouses and shop-floor environments. In a related research paper [1], we show that an AGVs can detect, localize, and track pallets using machine learning techniques based only on the data of an on-board 2D laser rangefinder. Such sensor is very common in industrial scenarios due to its simplicity and robustness, but it can only provide a limited amount of data. Therefore, it has been neglected in the past in favor of more complex solutions. In this paper, we release to the community the data we collected in [1] for further research activities in the field of pallet localization and tracking. The dataset comprises a collection of 565 2D scans from real-world environments, which are divided into 340 samples where pallets are present, and 225 samples where they are not. The data have been manually labelled and are provided in different formats.
Tasks
Published	2018-05-22
URL	http://arxiv.org/abs/1805.08564v2
PDF	http://arxiv.org/pdf/1805.08564v2.pdf
PWC	https://paperswithcode.com/paper/a-2d-laser-rangefinder-scans-dataset-of
Repo	https://github.com/EMAROLab/PDT
Framework	none

Deep Co-Training for Semi-Supervised Image Recognition


Title	Deep Co-Training for Semi-Supervised Image Recognition
Authors	Siyuan Qiao, Wei Shen, Zhishuai Zhang, Bo Wang, Alan Yuille
Abstract	In this paper, we study the problem of semi-supervised image recognition, which is to learn classifiers using both labeled and unlabeled images. We present Deep Co-Training, a deep learning based method inspired by the Co-Training framework. The original Co-Training learns two classifiers on two views which are data from different sources that describe the same instances. To extend this concept to deep learning, Deep Co-Training trains multiple deep neural networks to be the different views and exploits adversarial examples to encourage view difference, in order to prevent the networks from collapsing into each other. As a result, the co-trained networks provide different and complementary information about the data, which is necessary for the Co-Training framework to achieve good results. We test our method on SVHN, CIFAR-10/100 and ImageNet datasets, and our method outperforms the previous state-of-the-art methods by a large margin.
Tasks
Published	2018-03-15
URL	http://arxiv.org/abs/1803.05984v1
PDF	http://arxiv.org/pdf/1803.05984v1.pdf
PWC	https://paperswithcode.com/paper/deep-co-training-for-semi-supervised-image
Repo	https://github.com/AlanChou/Deep-Co-Training-for-Semi-Supervised-Image-Recognition
Framework	pytorch

Enhancing Underwater Imagery using Generative Adversarial Networks


Title	Enhancing Underwater Imagery using Generative Adversarial Networks
Authors	Cameron Fabbri, Md Jahidul Islam, Junaed Sattar
Abstract	Autonomous underwater vehicles (AUVs) rely on a variety of sensors - acoustic, inertial and visual - for intelligent decision making. Due to its non-intrusive, passive nature, and high information content, vision is an attractive sensing modality, particularly at shallower depths. However, factors such as light refraction and absorption, suspended particles in the water, and color distortion affect the quality of visual data, resulting in noisy and distorted images. AUVs that rely on visual sensing thus face difficult challenges, and consequently exhibit poor performance on vision-driven tasks. This paper proposes a method to improve the quality of visual underwater scenes using Generative Adversarial Networks (GANs), with the goal of improving input to vision-driven behaviors further down the autonomy pipeline. Furthermore, we show how recently proposed methods are able to generate a dataset for the purpose of such underwater image restoration. For any visually-guided underwater robots, this improvement can result in increased safety and reliability through robust visual perception. To that effect, we present quantitative and qualitative data which demonstrates that images corrected through the proposed approach generate more visually appealing images, and also provide increased accuracy for a diver tracking algorithm.
Tasks	Decision Making, Image Restoration
Published	2018-01-11
URL	http://arxiv.org/abs/1801.04011v1
PDF	http://arxiv.org/pdf/1801.04011v1.pdf
PWC	https://paperswithcode.com/paper/enhancing-underwater-imagery-using-generative
Repo	https://github.com/cameronfabbri/Underwater-Color-Correction
Framework	tf

Deep learning cardiac motion analysis for human survival prediction


Title	Deep learning cardiac motion analysis for human survival prediction
Authors	Ghalib A. Bello, Timothy J. W. Dawes, Jinming Duan, Carlo Biffi, Antonio de Marvao, Luke S. G. E. Howard, J. Simon R. Gibbs, Martin R. Wilkins, Stuart A. Cook, Daniel Rueckert, Declan P. O’Regan
Abstract	Motion analysis is used in computer vision to understand the behaviour of moving objects in sequences of images. Optimising the interpretation of dynamic biological systems requires accurate and precise motion tracking as well as efficient representations of high-dimensional motion trajectories so that these can be used for prediction tasks. Here we use image sequences of the heart, acquired using cardiac magnetic resonance imaging, to create time-resolved three-dimensional segmentations using a fully convolutional network trained on anatomical shape priors. This dense motion model formed the input to a supervised denoising autoencoder (4Dsurvival), which is a hybrid network consisting of an autoencoder that learns a task-specific latent code representation trained on observed outcome data, yielding a latent representation optimised for survival prediction. To handle right-censored survival outcomes, our network used a Cox partial likelihood loss function. In a study of 302 patients the predictive accuracy (quantified by Harrell’s C-index) was significantly higher (p < .0001) for our model C=0.73 (95$%$ CI: 0.68 - 0.78) than the human benchmark of C=0.59 (95$%$ CI: 0.53 - 0.65). This work demonstrates how a complex computer vision task using high-dimensional medical image data can efficiently predict human survival.
Tasks	Denoising
Published	2018-10-08
URL	http://arxiv.org/abs/1810.03382v1
PDF	http://arxiv.org/pdf/1810.03382v1.pdf
PWC	https://paperswithcode.com/paper/deep-learning-cardiac-motion-analysis-for
Repo	https://github.com/UK-Digital-Heart-Project/4Dsurvival
Framework	tf