October 18, 2019

3208 words 16 mins read

Paper Group ANR 411

Spectral Embedding Norm: Looking Deep into the Spectrum of the Graph Laplacian. Data Association with Gaussian Processes. Fault Tolerance in Iterative-Convergent Machine Learning. Direct Uncertainty Prediction for Medical Second Opinions. Minimum spanning tree release under differential privacy constraints. Breast Tumor Segmentation and Shape Class …

Spectral Embedding Norm: Looking Deep into the Spectrum of the Graph Laplacian


Title	Spectral Embedding Norm: Looking Deep into the Spectrum of the Graph Laplacian
Authors	Xiuyuan Cheng, Gal Mishne
Abstract	The extraction of clusters from a dataset which includes multiple clusters and a significant background component is a non-trivial task of practical importance. In image analysis this manifests for example in anomaly detection and target detection. The traditional spectral clustering algorithm, which relies on the leading $K$ eigenvectors to detect $K$ clusters, fails in such cases. In this paper we propose the {\it spectral embedding norm} which sums the squared values of the first $I$ normalized eigenvectors, where $I$ can be significantly larger than $K$. We prove that this quantity can be used to separate clusters from the background in unbalanced settings, including extreme cases such as outlier detection. The performance of the algorithm is not sensitive to the choice of $I$, and we demonstrate its application on synthetic and real-world remote sensing and neuroimaging datasets.
Tasks	Anomaly Detection, Outlier Detection
Published	2018-10-25
URL	https://arxiv.org/abs/1810.10695v2
PDF	https://arxiv.org/pdf/1810.10695v2.pdf
PWC	https://paperswithcode.com/paper/spectral-embedding-norm-looking-deep-into-the
Repo
Framework

Data Association with Gaussian Processes


Title	Data Association with Gaussian Processes
Authors	Markus Kaiser, Clemens Otte, Thomas Runkler, Carl Henrik Ek
Abstract	The data association problem is concerned with separating data coming from different generating processes, for example when data come from different data sources, contain significant noise, or exhibit multimodality. We present a fully Bayesian approach to this problem. Our model is capable of simultaneously solving the data association problem and the induced supervised learning problems. Underpinning our approach is the use of Gaussian process priors to encode the structure of both the data and the data associations. We present an efficient learning scheme based on doubly stochastic variational inference and discuss how it can be applied to deep Gaussian process priors.
Tasks	Gaussian Processes
Published	2018-10-16
URL	https://arxiv.org/abs/1810.07158v3
PDF	https://arxiv.org/pdf/1810.07158v3.pdf
PWC	https://paperswithcode.com/paper/data-association-with-gaussian-processes
Repo
Framework

Fault Tolerance in Iterative-Convergent Machine Learning


Title	Fault Tolerance in Iterative-Convergent Machine Learning
Authors	Aurick Qiao, Bryon Aragam, Bingjing Zhang, Eric P. Xing
Abstract	Machine learning (ML) training algorithms often possess an inherent self-correcting behavior due to their iterative-convergent nature. Recent systems exploit this property to achieve adaptability and efficiency in unreliable computing environments by relaxing the consistency of execution and allowing calculation errors to be self-corrected during training. However, the behavior of such systems are only well understood for specific types of calculation errors, such as those caused by staleness, reduced precision, or asynchronicity, and for specific types of training algorithms, such as stochastic gradient descent. In this paper, we develop a general framework to quantify the effects of calculation errors on iterative-convergent algorithms and use this framework to design new strategies for checkpoint-based fault tolerance. Our framework yields a worst-case upper bound on the iteration cost of arbitrary perturbations to model parameters during training. Our system, SCAR, employs strategies which reduce the iteration cost upper bound due to perturbations incurred when recovering from checkpoints. We show that SCAR can reduce the iteration cost of partial failures by 78% - 95% when compared with traditional checkpoint-based fault tolerance across a variety of ML models and training algorithms.
Tasks
Published	2018-10-17
URL	http://arxiv.org/abs/1810.07354v1
PDF	http://arxiv.org/pdf/1810.07354v1.pdf
PWC	https://paperswithcode.com/paper/fault-tolerance-in-iterative-convergent
Repo
Framework

Direct Uncertainty Prediction for Medical Second Opinions


Title	Direct Uncertainty Prediction for Medical Second Opinions
Authors	Maithra Raghu, Katy Blumer, Rory Sayres, Ziad Obermeyer, Robert Kleinberg, Sendhil Mullainathan, Jon Kleinberg
Abstract	The issue of disagreements amongst human experts is a ubiquitous one in both machine learning and medicine. In medicine, this often corresponds to doctor disagreements on a patient diagnosis. In this work, we show that machine learning models can be trained to give uncertainty scores to data instances that might result in high expert disagreements. In particular, they can identify patient cases that would benefit most from a medical second opinion. Our central methodological finding is that Direct Uncertainty Prediction (DUP), training a model to predict an uncertainty score directly from the raw patient features, works better than Uncertainty Via Classification, the two-step process of training a classifier and postprocessing the output distribution to give an uncertainty score. We show this both with a theoretical result, and on extensive evaluations on a large scale medical imaging application.
Tasks
Published	2018-07-04
URL	https://arxiv.org/abs/1807.01771v4
PDF	https://arxiv.org/pdf/1807.01771v4.pdf
PWC	https://paperswithcode.com/paper/direct-uncertainty-prediction-for-medical
Repo
Framework

Minimum spanning tree release under differential privacy constraints


Title	Minimum spanning tree release under differential privacy constraints
Authors	Rafael Pinot
Abstract	We investigate the problem of nodes clustering under privacy constraints when representing a dataset as a graph. Our contribution is threefold. First we formally define the concept of differential privacy for structured databases such as graphs, and give an alternative definition based on a new neighborhood notion between graphs. This definition is adapted to particular frameworks that can be met in various application fields such as genomics, world wide web, population survey, etc. Second, we introduce a new algorithm to tackle the issue of privately releasing an approximated minimum spanning tree topology for a simple-undirected-weighted graph. It provides a simple way of producing the topology of a private almost minimum spanning tree which outperforms, in most cases, the state of the art “Laplace mechanism” in terms of weight-approximation error. Finally, we propose a theoretically motivated method combining a sanitizing mechanism (such as Laplace or our new algorithm) with a Minimum Spanning Tree (MST)-based clustering algorithm. It provides an accurate method for nodes clustering in a graph while keeping the sensitive information contained in the edges weights of the private graph. We provide some theoretical results on the robustness of an almost minimum spanning tree construction for Laplace sanitizing mechanisms. These results exhibit which conditions the graph weights should respect in order to consider that the nodes form well separated clusters both for Laplace and our algorithm as sanitizing mechanism. The method has been experimentally evaluated on simulated data, and preliminary results show the good behavior of the algorithm while identifying well separated clusters.
Tasks
Published	2018-01-19
URL	http://arxiv.org/abs/1801.06423v1
PDF	http://arxiv.org/pdf/1801.06423v1.pdf
PWC	https://paperswithcode.com/paper/minimum-spanning-tree-release-under
Repo
Framework

Breast Tumor Segmentation and Shape Classification in Mammograms using Generative Adversarial and Convolutional Neural Network


Title	Breast Tumor Segmentation and Shape Classification in Mammograms using Generative Adversarial and Convolutional Neural Network
Authors	Vivek Kumar Singh, Hatem A. Rashwan, Santiago Romani, Farhan Akram, Nidhi Pandey, Md. Mostafa Kamal Sarker, Adel Saleh, Meritexell Arenas, Miguel Arquez, Domenec Puig, Jordina Torrents-Barrena
Abstract	Mammogram inspection in search of breast tumors is a tough assignment that radiologists must carry out frequently. Therefore, image analysis methods are needed for the detection and delineation of breast masses, which portray crucial morphological information that will support reliable diagnosis. In this paper, we proposed a conditional Generative Adversarial Network (cGAN) devised to segment a breast mass within a region of interest (ROI) in a mammogram. The generative network learns to recognize the breast mass area and to create the binary mask that outlines the breast mass. In turn, the adversarial network learns to distinguish between real (ground truth) and synthetic segmentations, thus enforcing the generative network to create binary masks as realistic as possible. The cGAN works well even when the number of training samples are limited. Therefore, the proposed method outperforms several state-of-the-art approaches. This hypothesis is corroborated by diverse experiments performed on two datasets, the public INbreast and a private in-house dataset. The proposed segmentation model provides a high Dice coefficient and Intersection over Union (IoU) of 94% and 87%, respectively. In addition, a shape descriptor based on a Convolutional Neural Network (CNN) is proposed to classify the generated masks into four mass shapes: irregular, lobular, oval and round. The proposed shape descriptor was trained on Digital Database for Screening Mammography (DDSM) yielding an overall accuracy of 80%, which outperforms the current state-of-the-art.
Tasks
Published	2018-09-05
URL	http://arxiv.org/abs/1809.01687v3
PDF	http://arxiv.org/pdf/1809.01687v3.pdf
PWC	https://paperswithcode.com/paper/breast-tumor-segmentation-and-shape
Repo
Framework

What can linguistics and deep learning contribute to each other?


Title	What can linguistics and deep learning contribute to each other?
Authors	Tal Linzen
Abstract	Joe Pater’s target article calls for greater interaction between neural network research and linguistics. I expand on this call and show how such interaction can benefit both fields. Linguists can contribute to research on neural networks for language technologies by clearly delineating the linguistic capabilities that can be expected of such systems, and by constructing controlled experimental paradigms that can determine whether those desiderata have been met. In the other direction, neural networks can benefit the scientific study of language by providing infrastructure for modeling human sentence processing and for evaluating the necessity of particular innate constraints on language acquisition.
Tasks	Language Acquisition
Published	2018-09-11
URL	http://arxiv.org/abs/1809.04179v2
PDF	http://arxiv.org/pdf/1809.04179v2.pdf
PWC	https://paperswithcode.com/paper/what-can-linguistics-and-deep-learning
Repo
Framework

Stochastic Training of Residual Networks: a Differential Equation Viewpoint


Title	Stochastic Training of Residual Networks: a Differential Equation Viewpoint
Authors	Qi Sun, Yunzhe Tao, Qiang Du
Abstract	During the last few years, significant attention has been paid to the stochastic training of artificial neural networks, which is known as an effective regularization approach that helps improve the generalization capability of trained models. In this work, the method of modified equations is applied to show that the residual network and its variants with noise injection can be regarded as weak approximations of stochastic differential equations. Such observations enable us to bridge the stochastic training processes with the optimal control of backward Kolmogorov’s equations. This not only offers a novel perspective on the effects of regularization from the loss landscape viewpoint but also sheds light on the design of more reliable and efficient stochastic training strategies. As an example, we propose a new way to utilize Bernoulli dropout within the plain residual network architecture and conduct experiments on a real-world image classification task to substantiate our theoretical findings.
Tasks	Image Classification
Published	2018-12-01
URL	http://arxiv.org/abs/1812.00174v1
PDF	http://arxiv.org/pdf/1812.00174v1.pdf
PWC	https://paperswithcode.com/paper/stochastic-training-of-residual-networks-a
Repo
Framework

Detecting Backdoor Attacks on Deep Neural Networks by Activation Clustering


Title	Detecting Backdoor Attacks on Deep Neural Networks by Activation Clustering
Authors	Bryant Chen, Wilka Carvalho, Nathalie Baracaldo, Heiko Ludwig, Benjamin Edwards, Taesung Lee, Ian Molloy, Biplav Srivastava
Abstract	While machine learning (ML) models are being increasingly trusted to make decisions in different and varying areas, the safety of systems using such models has become an increasing concern. In particular, ML models are often trained on data from potentially untrustworthy sources, providing adversaries with the opportunity to manipulate them by inserting carefully crafted samples into the training set. Recent work has shown that this type of attack, called a poisoning attack, allows adversaries to insert backdoors or trojans into the model, enabling malicious behavior with simple external backdoor triggers at inference time and only a blackbox perspective of the model itself. Detecting this type of attack is challenging because the unexpected behavior occurs only when a backdoor trigger, which is known only to the adversary, is present. Model users, either direct users of training data or users of pre-trained model from a catalog, may not guarantee the safe operation of their ML-based system. In this paper, we propose a novel approach to backdoor detection and removal for neural networks. Through extensive experimental results, we demonstrate its effectiveness for neural networks classifying text and images. To the best of our knowledge, this is the first methodology capable of detecting poisonous data crafted to insert backdoors and repairing the model that does not require a verified and trusted dataset.
Tasks
Published	2018-11-09
URL	http://arxiv.org/abs/1811.03728v1
PDF	http://arxiv.org/pdf/1811.03728v1.pdf
PWC	https://paperswithcode.com/paper/detecting-backdoor-attacks-on-deep-neural
Repo
Framework

ComDefend: An Efficient Image Compression Model to Defend Adversarial Examples


Title	ComDefend: An Efficient Image Compression Model to Defend Adversarial Examples
Authors	Xiaojun Jia, Xingxing Wei, Xiaochun Cao, Hassan Foroosh
Abstract	Deep neural networks (DNNs) have been demonstrated to be vulnerable to adversarial examples. Specifically, adding imperceptible perturbations to clean images can fool the well trained deep neural networks. In this paper, we propose an end-to-end image compression model to defend adversarial examples: \textbf{ComDefend}. The proposed model consists of a compression convolutional neural network (ComCNN) and a reconstruction convolutional neural network (ResCNN). The ComCNN is used to maintain the structure information of the original image and purify adversarial perturbations. And the ResCNN is used to reconstruct the original image with high quality. In other words, ComDefend can transform the adversarial image to its clean version, which is then fed to the trained classifier. Our method is a pre-processing module, and does not modify the classifier’s structure during the whole process. Therefore, it can be combined with other model-specific defense models to jointly improve the classifier’s robustness. A series of experiments conducted on MNIST, CIFAR10 and ImageNet show that the proposed method outperforms the state-of-the-art defense methods, and is consistently effective to protect classifiers against adversarial attacks.
Tasks	Image Compression
Published	2018-11-30
URL	https://arxiv.org/abs/1811.12673v3
PDF	https://arxiv.org/pdf/1811.12673v3.pdf
PWC	https://paperswithcode.com/paper/comdefend-an-efficient-image-compression
Repo
Framework

Self-Organizing Maps as a Storage and Transfer Mechanism in Reinforcement Learning


Title	Self-Organizing Maps as a Storage and Transfer Mechanism in Reinforcement Learning
Authors	Thommen George Karimpanal, Roland Bouffanais
Abstract	The idea of reusing information from previously learned tasks (source tasks) for the learning of new tasks (target tasks) has the potential to significantly improve the sample efficiency reinforcement learning agents. In this work, we describe an approach to concisely store and represent learned task knowledge, and reuse it by allowing it to guide the exploration of an agent while it learns new tasks. In order to do so, we use a measure of similarity that is defined directly in the space of parameterized representations of the value functions. This similarity measure is also used as a basis for a variant of the growing self-organizing map algorithm, which is simultaneously used to enable the storage of previously acquired task knowledge in an adaptive and scalable manner.We empirically validate our approach in a simulated navigation environment and discuss possible extensions to this approach along with potential applications where it could be particularly useful.
Tasks
Published	2018-07-19
URL	http://arxiv.org/abs/1807.07530v1
PDF	http://arxiv.org/pdf/1807.07530v1.pdf
PWC	https://paperswithcode.com/paper/self-organizing-maps-as-a-storage-and
Repo
Framework

Replication study: Development and validation of deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs


Title	Replication study: Development and validation of deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs
Authors	Mike Voets, Kajsa Møllersen, Lars Ailo Bongo
Abstract	Replication studies are essential for validation of new methods, and are crucial to maintain the high standards of scientific publications, and to use the results in practice. We have attempted to replicate the main method in ‘Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs’ published in JAMA 2016; 316(22). We re-implemented the method since the source code is not available, and we used publicly available data sets. The original study used non-public fundus images from EyePACS and three hospitals in India for training. We used a different EyePACS data set from Kaggle. The original study used the benchmark data set Messidor-2 to evaluate the algorithm’s performance. We used the same data set. In the original study, ophthalmologists re-graded all images for diabetic retinopathy, macular edema, and image gradability. There was one diabetic retinopathy grade per image for our data sets, and we assessed image gradability ourselves. Hyper-parameter settings were not described in the original study. But some of these were later published. We were not able to replicate the original study. Our algorithm’s area under the receiver operating curve (AUC) of 0.94 on the Kaggle EyePACS test set and 0.80 on Messidor-2 did not come close to the reported AUC of 0.99 in the original study. This may be caused by the use of a single grade per image, different data, or different not described hyper-parameter settings. This study shows the challenges of replicating deep learning, and the need for more replication studies to validate deep learning methods, especially for medical image analysis. Our source code and instructions are available at: https://github.com/mikevoets/jama16-retina-replication
Tasks	Diabetic Retinopathy Detection, Medical Image Segmentation, Mitosis Detection
Published	2018-03-12
URL	http://arxiv.org/abs/1803.04337v3
PDF	http://arxiv.org/pdf/1803.04337v3.pdf
PWC	https://paperswithcode.com/paper/replication-study-development-and-validation
Repo
Framework

Adding Neural Network Controllers to Behavior Trees without Destroying Performance Guarantees


Title	Adding Neural Network Controllers to Behavior Trees without Destroying Performance Guarantees
Authors	Christopher Iliffe Sprague, Petter Ögren
Abstract	In this paper, we show how Behavior Trees that have performance guarantees, in terms of safety and goal convergence, can be extended with components that were designed using machine learning, without destroying those performance guarantees. Machine learning approaches such as reinforcement learning or learning from demonstration can be very appealing to AI designers that want efficient and realistic behaviors in their agents. However, those algorithms seldom provide guarantees for solving the given task in all different situations while keeping the agent safe. Instead, such guarantees are often easier to find for manually designed model based approaches. In this paper we exploit the modularity of Behavior trees to extend a given design with an efficient, but possibly unreliable, machine learning component in a way that preserves the guarantees. The approach is illustrated with an inverted pendulum example.
Tasks
Published	2018-09-26
URL	https://arxiv.org/abs/1809.10283v2
PDF	https://arxiv.org/pdf/1809.10283v2.pdf
PWC	https://paperswithcode.com/paper/adding-neural-network-controllers-to-behavior
Repo
Framework

Robust Neural Machine Translation with Joint Textual and Phonetic Embedding


Title	Robust Neural Machine Translation with Joint Textual and Phonetic Embedding
Authors	Hairong Liu, Mingbo Ma, Liang Huang, Hao Xiong, Zhongjun He
Abstract	Neural machine translation (NMT) is notoriously sensitive to noises, but noises are almost inevitable in practice. One special kind of noise is the homophone noise, where words are replaced by other words with similar pronunciations. We propose to improve the robustness of NMT to homophone noises by 1) jointly embedding both textual and phonetic information of source sentences, and 2) augmenting the training dataset with homophone noises. Interestingly, to achieve better translation quality and more robustness, we found that most (though not all) weights should be put on the phonetic rather than textual information. Experiments show that our method not only significantly improves the robustness of NMT to homophone noises, but also surprisingly improves the translation quality on some clean test sets.
Tasks	Machine Translation, Speech Recognition
Published	2018-10-15
URL	https://arxiv.org/abs/1810.06729v2
PDF	https://arxiv.org/pdf/1810.06729v2.pdf
PWC	https://paperswithcode.com/paper/robust-neural-machine-translation-with-joint
Repo
Framework

Towards Automatic & Personalised Mobile Health Interventions: An Interactive Machine Learning Perspective


Title	Towards Automatic & Personalised Mobile Health Interventions: An Interactive Machine Learning Perspective
Authors	Ahmed Fadhil
Abstract	Machine learning (ML) is the fastest growing field in computer science and healthcare, providing future benefits in improved medical diagnoses, disease analyses and prevention. In this paper, we introduce an application of interactive machine learning (iML) in a telemedicine system, to enable automatic and personalised interventions for lifestyle promotion. We first present the high level architecture of the system and the components forming the overall architecture. We then illustrate the interactive machine learning process design. Prediction models are expected to be trained through the participants’ profiles, activity performance, and feedback from the caregiver. Finally, we show some preliminary results during the system implementation and discuss future directions. We envisage the proposed system to be digitally implemented, and behaviourally designed to promote healthy lifestyle and activities, and hence prevent users from the risk of chronic diseases.
Tasks
Published	2018-03-03
URL	http://arxiv.org/abs/1803.01842v1
PDF	http://arxiv.org/pdf/1803.01842v1.pdf
PWC	https://paperswithcode.com/paper/towards-automatic-personalised-mobile-health
Repo
Framework