October 19, 2019

3159 words 15 mins read

Paper Group ANR 295

Adversarially Robust Generalization Requires More Data. A Group-Theoretic Approach to Computational Abstraction: Symmetry-Driven Hierarchical Clustering. Generalised Differential Privacy for Text Document Processing. Combating Uncertainty with Novel Losses for Automatic Left Atrium Segmentation. Convergence of the Expectation-Maximization Algorithm …

Adversarially Robust Generalization Requires More Data


Title	Adversarially Robust Generalization Requires More Data
Authors	Ludwig Schmidt, Shibani Santurkar, Dimitris Tsipras, Kunal Talwar, Aleksander Mądry
Abstract	Machine learning models are often susceptible to adversarial perturbations of their inputs. Even small perturbations can cause state-of-the-art classifiers with high “standard” accuracy to produce an incorrect prediction with high confidence. To better understand this phenomenon, we study adversarially robust learning from the viewpoint of generalization. We show that already in a simple natural data model, the sample complexity of robust learning can be significantly larger than that of “standard” learning. This gap is information theoretic and holds irrespective of the training algorithm or the model family. We complement our theoretical results with experiments on popular image classification datasets and show that a similar gap exists here as well. We postulate that the difficulty of training robust classifiers stems, at least partially, from this inherently larger sample complexity.
Tasks	Image Classification
Published	2018-04-30
URL	http://arxiv.org/abs/1804.11285v2
PDF	http://arxiv.org/pdf/1804.11285v2.pdf
PWC	https://paperswithcode.com/paper/adversarially-robust-generalization-requires
Repo
Framework

A Group-Theoretic Approach to Computational Abstraction: Symmetry-Driven Hierarchical Clustering


Title	A Group-Theoretic Approach to Computational Abstraction: Symmetry-Driven Hierarchical Clustering
Authors	Haizi Yu, Igor Mineyev, Lav R. Varshney
Abstract	Abstraction plays a key role in concept learning and knowledge discovery; this paper is concerned with computational abstraction. In particular, we study the nature of abstraction through a group-theoretic approach, formalizing it as symmetry-driven—as opposed to data-driven—hierarchical clustering. Thus, the resulting clustering framework is data-free, feature-free, similarity-free, and globally hierarchical—the four key features that distinguish it from common data clustering models such as $k$-means. Beyond a theoretical foundation for abstraction, we also present a top-down and a bottom-up approach to establish an algorithmic foundation for practical abstraction-generating methods. Lastly, via both a theoretical explanation and a real-world application, we illustrate that further coupling of our abstraction framework with statistics realizes Shannon’s information lattice and even further, brings learning into the picture. This not only presents one use case of our proposed computational abstraction, but also gives a first step towards a principled and cognitive way of automatic concept learning and knowledge discovery.
Tasks
Published	2018-07-30
URL	https://arxiv.org/abs/1807.11167v2
PDF	https://arxiv.org/pdf/1807.11167v2.pdf
PWC	https://paperswithcode.com/paper/a-group-theoretic-approach-to-abstraction
Repo
Framework

Generalised Differential Privacy for Text Document Processing


Title	Generalised Differential Privacy for Text Document Processing
Authors	Natasha Fernandes, Mark Dras, Annabelle McIver
Abstract	We address the problem of how to “obfuscate” texts by removing stylistic clues which can identify authorship, whilst preserving (as much as possible) the content of the text. In this paper we combine ideas from “generalised differential privacy” and machine learning techniques for text processing to model privacy for text documents. We define a privacy mechanism that operates at the level of text documents represented as “bags-of-words” - these representations are typical in machine learning and contain sufficient information to carry out many kinds of classification tasks including topic identification and authorship attribution (of the original documents). We show that our mechanism satisfies privacy with respect to a metric for semantic similarity, thereby providing a balance between utility, defined by the semantic content of texts, with the obfuscation of stylistic clues. We demonstrate our implementation on a “fan fiction” dataset, confirming that it is indeed possible to disguise writing style effectively whilst preserving enough information and variation for accurate content classification tasks.
Tasks	Semantic Similarity, Semantic Textual Similarity
Published	2018-11-26
URL	http://arxiv.org/abs/1811.10256v2
PDF	http://arxiv.org/pdf/1811.10256v2.pdf
PWC	https://paperswithcode.com/paper/generalised-differential-privacy-for-text
Repo
Framework

Combating Uncertainty with Novel Losses for Automatic Left Atrium Segmentation


Title	Combating Uncertainty with Novel Losses for Automatic Left Atrium Segmentation
Authors	Xin Yang, Na Wang, Yi Wang, Xu Wang, Reza Nezafat, Dong Ni, Pheng-Ann Heng
Abstract	Segmenting left atrium in MR volume holds great potentials in promoting the treatment of atrial fibrillation. However, the varying anatomies, artifacts and low contrasts among tissues hinder the advance of both manual and automated solutions. In this paper, we propose a fully-automated framework to segment left atrium in gadolinium-enhanced MR volumes. The region of left atrium is firstly automatically localized by a detection module. Our framework then originates with a customized 3D deep neural network to fully explore the spatial dependency in the region for segmentation. To alleviate the risk of low training efficiency and potential overfitting, we enhance our deep network with the transfer learning and deep supervision strategy. Main contribution of our network design lies in the composite loss function to combat the boundary ambiguity and hard examples. We firstly adopt the Overlap loss to encourage network reduce the overlap between the foreground and background and thus sharpen the predictions on boundary. We then propose a novel Focal Positive loss to guide the learning of voxel-specific threshold and emphasize the foreground to improve classification sensitivity. Further improvement is obtained with an recursive training scheme. With ablation studies, all the introduced modules prove to be effective. The proposed framework achieves an average Dice of 92.24 in segmenting left atrium with pulmonary veins on 20 testing volumes.
Tasks	Transfer Learning
Published	2018-12-14
URL	http://arxiv.org/abs/1812.05807v1
PDF	http://arxiv.org/pdf/1812.05807v1.pdf
PWC	https://paperswithcode.com/paper/combating-uncertainty-with-novel-losses-for
Repo
Framework

Convergence of the Expectation-Maximization Algorithm Through Discrete-Time Lyapunov Stability Theory


Title	Convergence of the Expectation-Maximization Algorithm Through Discrete-Time Lyapunov Stability Theory
Authors	Orlando Romero, Sarthak Chatterjee, Sérgio Pequito
Abstract	In this paper, we propose a dynamical systems perspective of the Expectation-Maximization (EM) algorithm. More precisely, we can analyze the EM algorithm as a nonlinear state-space dynamical system. The EM algorithm is widely adopted for data clustering and density estimation in statistics, control systems, and machine learning. This algorithm belongs to a large class of iterative algorithms known as proximal point methods. In particular, we re-interpret limit points of the EM algorithm and other local maximizers of the likelihood function it seeks to optimize as equilibria in its dynamical system representation. Furthermore, we propose to assess its convergence as asymptotic stability in the sense of Lyapunov. As a consequence, we proceed by leveraging recent results regarding discrete-time Lyapunov stability theory in order to establish asymptotic stability (and thus, convergence) in the dynamical system representation of the EM algorithm.
Tasks	Density Estimation
Published	2018-10-04
URL	http://arxiv.org/abs/1810.02022v1
PDF	http://arxiv.org/pdf/1810.02022v1.pdf
PWC	https://paperswithcode.com/paper/convergence-of-the-expectation-maximization
Repo
Framework

Networking the Boids is More Robust Against Adversarial Learning


Title	Networking the Boids is More Robust Against Adversarial Learning
Authors	Jiangjun Tang, George Leu, Hussein Abbass
Abstract	Swarm behavior using Boids-like models has been studied primarily using close-proximity spatial sensory information (e.g. vision range). In this study, we propose a novel approach in which the classic definition of boids\textquoteright \ neighborhood that relies on sensory perception and Euclidian space locality is replaced with graph-theoretic network-based proximity mimicking communication and social networks. We demonstrate that networking the boids leads to faster swarming and higher quality of the formation. We further investigate the effect of adversarial learning, whereby an observer attempts to reverse engineer the dynamics of the swarm through observing its behavior. The results show that networking the swarm demonstrated a more robust approach against adversarial learning than a local-proximity neighborhood structure.
Tasks
Published	2018-02-27
URL	http://arxiv.org/abs/1802.10206v1
PDF	http://arxiv.org/pdf/1802.10206v1.pdf
PWC	https://paperswithcode.com/paper/networking-the-boids-is-more-robust-against
Repo
Framework

Semantic-Aware Generative Adversarial Nets for Unsupervised Domain Adaptation in Chest X-ray Segmentation


Title	Semantic-Aware Generative Adversarial Nets for Unsupervised Domain Adaptation in Chest X-ray Segmentation
Authors	Cheng Chen, Qi Dou, Hao Chen, Pheng-Ann Heng
Abstract	In spite of the compelling achievements that deep neural networks (DNNs) have made in medical image computing, these deep models often suffer from degraded performance when being applied to new test datasets with domain shift. In this paper, we present a novel unsupervised domain adaptation approach for segmentation tasks by designing semantic-aware generative adversarial networks (GANs). Specifically, we transform the test image into the appearance of source domain, with the semantic structural information being well preserved, which is achieved by imposing a nested adversarial learning in semantic label space. In this way, the segmentation DNN learned from the source domain is able to be directly generalized to the transformed test image, eliminating the need of training a new model for every new target dataset. Our domain adaptation procedure is unsupervised, without using any target domain labels. The adversarial learning of our network is guided by a GAN loss for mapping data distributions, a cycle-consistency loss for retaining pixel-level content, and a semantic-aware loss for enhancing structural information. We validated our method on two different chest X-ray public datasets for left/right lung segmentation. Experimental results show that the segmentation performance of our unsupervised approach is highly competitive with the upper bound of supervised transfer learning.
Tasks	Domain Adaptation, Transfer Learning, Unsupervised Domain Adaptation
Published	2018-06-02
URL	http://arxiv.org/abs/1806.00600v2
PDF	http://arxiv.org/pdf/1806.00600v2.pdf
PWC	https://paperswithcode.com/paper/semantic-aware-generative-adversarial-nets
Repo
Framework

Explaining Deep Learning Models - A Bayesian Non-parametric Approach


Title	Explaining Deep Learning Models - A Bayesian Non-parametric Approach
Authors	Wenbo Guo, Sui Huang, Yunzhe Tao, Xinyu Xing, Lin Lin
Abstract	Understanding and interpreting how machine learning (ML) models make decisions have been a big challenge. While recent research has proposed various technical approaches to provide some clues as to how an ML model makes individual predictions, they cannot provide users with an ability to inspect a model as a complete entity. In this work, we propose a novel technical approach that augments a Bayesian non-parametric regression mixture model with multiple elastic nets. Using the enhanced mixture model, we can extract generalizable insights for a target model through a global approximation. To demonstrate the utility of our approach, we evaluate it on different ML models in the context of image recognition. The empirical results indicate that our proposed approach not only outperforms the state-of-the-art techniques in explaining individual decisions but also provides users with an ability to discover the vulnerabilities of the target ML models.
Tasks
Published	2018-11-07
URL	http://arxiv.org/abs/1811.03422v1
PDF	http://arxiv.org/pdf/1811.03422v1.pdf
PWC	https://paperswithcode.com/paper/explaining-deep-learning-models-a-bayesian
Repo
Framework

Distributed Variational Representation Learning


Title	Distributed Variational Representation Learning
Authors	Inaki Estella Aguerri, Abdellatif Zaidi
Abstract	The problem of distributed representation learning is one in which multiple sources of information $X_1,\ldots,X_K$ are processed separately so as to learn as much information as possible about some ground truth $Y$. We investigate this problem from information-theoretic grounds, through a generalization of Tishby’s centralized Information Bottleneck (IB) method to the distributed setting. Specifically, $K$ encoders, $K \geq 2$, compress their observations $X_1,\ldots,X_K$ separately in a manner such that, collectively, the produced representations preserve as much information as possible about $Y$. We study both discrete memoryless (DM) and memoryless vector Gaussian data models. For the discrete model, we establish a single-letter characterization of the optimal tradeoff between complexity (or rate) and relevance (or information) for a class of memoryless sources (the observations $X_1,\ldots,X_K$ being conditionally independent given $Y$). For the vector Gaussian model, we provide an explicit characterization of the optimal complexity-relevance tradeoff. Furthermore, we develop a variational bound on the complexity-relevance tradeoff which generalizes the evidence lower bound (ELBO) to the distributed setting. We also provide two algorithms that allow to compute this bound: i) a Blahut-Arimoto type iterative algorithm which enables to compute optimal complexity-relevance encoding mappings by iterating over a set of self-consistent equations, and ii) a variational inference type algorithm in which the encoding mappings are parametrized by neural networks and the bound approximated by Markov sampling and optimized with stochastic gradient descent. Numerical results on synthetic and real datasets are provided to support the efficiency of the approaches and algorithms developed in this paper.
Tasks	Representation Learning
Published	2018-07-11
URL	http://arxiv.org/abs/1807.04193v3
PDF	http://arxiv.org/pdf/1807.04193v3.pdf
PWC	https://paperswithcode.com/paper/distributed-variational-representation
Repo
Framework

Neonatal EEG Interpretation and Decision Support Framework for Mobile Platforms


Title	Neonatal EEG Interpretation and Decision Support Framework for Mobile Platforms
Authors	Mark O’Sullivan, Sergi Gomez, Alison O’Shea, Eduard Salgado, Kevin Huillca, Sean Mathieson, Geraldine Boylan, Emanuel Popovici, Andriy Temko
Abstract	This paper proposes and implements an intuitive and pervasive solution for neonatal EEG monitoring assisted by sonification and deep learning AI that provides information about neonatal brain health to all neonatal healthcare professionals, particularly those without EEG interpretation expertise. The system aims to increase the demographic of clinicians capable of diagnosing abnormalities in neonatal EEG. The proposed system uses a low-cost and low-power EEG acquisition system. An Android app provides single-channel EEG visualization, traffic-light indication of the presence of neonatal seizures provided by a trained, deep convolutional neural network and an algorithm for EEG sonification, designed to facilitate the perception of changes in EEG morphology specific to neonatal seizures. The multifaceted EEG interpretation framework is presented and the implemented mobile platform architecture is analyzed with respect to its power consumption and accuracy.
Tasks	EEG
Published	2018-06-08
URL	http://arxiv.org/abs/1806.04037v1
PDF	http://arxiv.org/pdf/1806.04037v1.pdf
PWC	https://paperswithcode.com/paper/neonatal-eeg-interpretation-and-decision
Repo
Framework


Title	LRMM: Learning to Recommend with Missing Modalities
Authors	Cheng Wang, Mathias Niepert, Hui Li
Abstract	Multimodal learning has shown promising performance in content-based recommendation due to the auxiliary user and item information of multiple modalities such as text and images. However, the problem of incomplete and missing modality is rarely explored and most existing methods fail in learning a recommendation model with missing or corrupted modalities. In this paper, we propose LRMM, a novel framework that mitigates not only the problem of missing modalities but also more generally the cold-start problem of recommender systems. We propose modality dropout (m-drop) and a multimodal sequential autoencoder (m-auto) to learn multimodal representations for complementing and imputing missing modalities. Extensive experiments on real-world Amazon data show that LRMM achieves state-of-the-art performance on rating prediction tasks. More importantly, LRMM is more robust to previous methods in alleviating data-sparsity and the cold-start problem.
Tasks	Recommendation Systems
Published	2018-08-21
URL	http://arxiv.org/abs/1808.06791v2
PDF	http://arxiv.org/pdf/1808.06791v2.pdf
PWC	https://paperswithcode.com/paper/lrmm-learning-to-recommend-with-missing
Repo
Framework

Neural Machine Translation Inspired Binary Code Similarity Comparison beyond Function Pairs


Title	Neural Machine Translation Inspired Binary Code Similarity Comparison beyond Function Pairs
Authors	Fei Zuo, Xiaopeng Li, Patrick Young, Lannan Luo, Qiang Zeng, Zhexin Zhang
Abstract	Binary code analysis allows analyzing binary code without having access to the corresponding source code. A binary, after disassembly, is expressed in an assembly language. This inspires us to approach binary analysis by leveraging ideas and techniques from Natural Language Processing (NLP), a rich area focused on processing text of various natural languages. We notice that binary code analysis and NLP share a lot of analogical topics, such as semantics extraction, summarization, and classification. This work utilizes these ideas to address two important code similarity comparison problems. (I) Given a pair of basic blocks for different instruction set architectures (ISAs), determining whether their semantics is similar or not; and (II) given a piece of code of interest, determining if it is contained in another piece of assembly code for a different ISA. The solutions to these two problems have many applications, such as cross-architecture vulnerability discovery and code plagiarism detection. We implement a prototype system INNEREYE and perform a comprehensive evaluation. A comparison between our approach and existing approaches to Problem I shows that our system outperforms them in terms of accuracy, efficiency and scalability. And the case studies utilizing the system demonstrate that our solution to Problem II is effective. Moreover, this research showcases how to apply ideas and techniques from NLP to large-scale binary code analysis.
Tasks	Machine Translation
Published	2018-08-08
URL	http://arxiv.org/abs/1808.04706v2
PDF	http://arxiv.org/pdf/1808.04706v2.pdf
PWC	https://paperswithcode.com/paper/neural-machine-translation-inspired-binary
Repo
Framework

Reinforcement Learning for Fair Dynamic Pricing


Title	Reinforcement Learning for Fair Dynamic Pricing
Authors	Roberto Maestre, Juan Duque, Alberto Rubio, Juan Arévalo
Abstract	Unfair pricing policies have been shown to be one of the most negative perceptions customers can have concerning pricing, and may result in long-term losses for a company. Despite the fact that dynamic pricing models help companies maximize revenue, fairness and equality should be taken into account in order to avoid unfair price differences between groups of customers. This paper shows how to solve dynamic pricing by using Reinforcement Learning (RL) techniques so that prices are maximized while keeping a balance between revenue and fairness. We demonstrate that RL provides two main features to support fairness in dynamic pricing: on the one hand, RL is able to learn from recent experience, adapting the pricing policy to complex market environments; on the other hand, it provides a trade-off between short and long-term objectives, hence integrating fairness into the model’s core. Considering these two features, we propose the application of RL for revenue optimization, with the additional integration of fairness as part of the learning procedure by using Jain’s index as a metric. Results in a simulated environment show a significant improvement in fairness while at the same time maintaining optimisation of revenue.
Tasks
Published	2018-03-27
URL	http://arxiv.org/abs/1803.09967v1
PDF	http://arxiv.org/pdf/1803.09967v1.pdf
PWC	https://paperswithcode.com/paper/reinforcement-learning-for-fair-dynamic
Repo
Framework

Fooling Network Interpretation in Image Classification


Title	Fooling Network Interpretation in Image Classification
Authors	Akshayvarun Subramanya, Vipin Pillai, Hamed Pirsiavash
Abstract	Deep neural networks have been shown to be fooled rather easily using adversarial attack algorithms. Practical methods such as adversarial patches have been shown to be extremely effective in causing misclassification. However, these patches are highlighted using standard network interpretation algorithms, thus revealing the identity of the adversary. We show that it is possible to create adversarial patches which not only fool the prediction, but also change what we interpret regarding the cause of the prediction. Moreover, we introduce our attack as a controlled setting to measure the accuracy of interpretation algorithms. We show this using extensive experiments for Grad-CAM interpretation that transfers to occluding patch interpretation as well. We believe our algorithms can facilitate developing more robust network interpretation tools that truly explain the network’s underlying decision making process.
Tasks	Adversarial Attack, Decision Making, Image Classification
Published	2018-12-06
URL	https://arxiv.org/abs/1812.02843v2
PDF	https://arxiv.org/pdf/1812.02843v2.pdf
PWC	https://paperswithcode.com/paper/towards-hiding-adversarial-examples-from
Repo
Framework

Fast Context-Annotated Classification of Different Types of Web Service Descriptions


Title	Fast Context-Annotated Classification of Different Types of Web Service Descriptions
Authors	Serguei A. Mokhov, Joey Paquet, Arash Khodadadi
Abstract	In the recent rapid growth of web services, IoT, and cloud computing, many web services and APIs appeared on the web. With the failure of global UDDI registries, different service repositories started to appear, trying to list and categorize various types of web services for client applications’ discover and use. In order to increase the effectiveness and speed up the task of finding compatible Web Services in the brokerage when performing service composition or suggesting Web Services to the requests, high-level functionality of the service needs to be determined. Due to the lack of structured support for specifying such functionality, classification of services into a set of abstract categories is necessary. We employ a wide range of Machine Learning and Signal Processing algorithms and techniques in order to find the highest precision achievable in the scope of this article for the fast classification of three type of service descriptions: WSDL, REST, and WADL. In addition, we complement our approach by showing the importance and effect of contextual information on the classification of the service descriptions and show that it improves the accuracy in 5 different categories of services.
Tasks
Published	2018-05-31
URL	http://arxiv.org/abs/1806.02374v1
PDF	http://arxiv.org/pdf/1806.02374v1.pdf
PWC	https://paperswithcode.com/paper/fast-context-annotated-classification-of
Repo
Framework