October 21, 2019

2837 words 14 mins read

Paper Group AWR 94

Learning the Joint Representation of Heterogeneous Temporal Events for Clinical Endpoint Prediction. Capsule-Forensics: Using Capsule Networks to Detect Forged Images and Videos. Integral Privacy for Sampling. A Reductions Approach to Fair Classification. MCRM: Mother Compact Recurrent Memory. Seq2Seq-Vis: A Visual Debugging Tool for Sequence-to-Se …

Learning the Joint Representation of Heterogeneous Temporal Events for Clinical Endpoint Prediction


Title	Learning the Joint Representation of Heterogeneous Temporal Events for Clinical Endpoint Prediction
Authors	Luchen Liu, Jianhao Shen, Ming Zhang, Zichang Wang, Jian Tang
Abstract	The availability of a large amount of electronic health records (EHR) provides huge opportunities to improve health care service by mining these data. One important application is clinical endpoint prediction, which aims to predict whether a disease, a symptom or an abnormal lab test will happen in the future according to patients’ history records. This paper develops deep learning techniques for clinical endpoint prediction, which are effective in many practical applications. However, the problem is very challenging since patients’ history records contain multiple heterogeneous temporal events such as lab tests, diagnosis, and drug administrations. The visiting patterns of different types of events vary significantly, and there exist complex nonlinear relationships between different events. In this paper, we propose a novel model for learning the joint representation of heterogeneous temporal events. The model adds a new gate to control the visiting rates of different events which effectively models the irregular patterns of different events and their nonlinear correlations. Experiment results with real-world clinical data on the tasks of predicting death and abnormal lab tests prove the effectiveness of our proposed approach over competitive baselines.
Tasks
Published	2018-03-13
URL	http://arxiv.org/abs/1803.04837v4
PDF	http://arxiv.org/pdf/1803.04837v4.pdf
PWC	https://paperswithcode.com/paper/learning-the-joint-representation-of
Repo	https://github.com/pkusjh/HELSTM
Framework	none

Capsule-Forensics: Using Capsule Networks to Detect Forged Images and Videos


Title	Capsule-Forensics: Using Capsule Networks to Detect Forged Images and Videos
Authors	Huy H. Nguyen, Junichi Yamagishi, Isao Echizen
Abstract	Recent advances in media generation techniques have made it easier for attackers to create forged images and videos. State-of-the-art methods enable the real-time creation of a forged version of a single video obtained from a social network. Although numerous methods have been developed for detecting forged images and videos, they are generally targeted at certain domains and quickly become obsolete as new kinds of attacks appear. The method introduced in this paper uses a capsule network to detect various kinds of spoofs, from replay attacks using printed images or recorded videos to computer-generated videos using deep convolutional neural networks. It extends the application of capsule networks beyond their original intention to the solving of inverse graphics problems.
Tasks	Detect Forged Images And Videos
Published	2018-10-26
URL	http://arxiv.org/abs/1810.11215v1
PDF	http://arxiv.org/pdf/1810.11215v1.pdf
PWC	https://paperswithcode.com/paper/capsule-forensics-using-capsule-networks-to
Repo	https://github.com/nii-yamagishilab/Capsule-Forensics
Framework	pytorch

Integral Privacy for Sampling


Title	Integral Privacy for Sampling
Authors	Hisham Husain, Zac Cranko, Richard Nock
Abstract	Differential privacy is a leading protection setting, focused by design on individual privacy. Many applications, in medical / pharmaceutical domains or social networks, rather posit privacy at a group level, a setting we call integral privacy. We aim for the strongest form of privacy: the group size is in particular not known in advance. We study a problem with related applications in domains cited above that have recently met with substantial recent press: sampling. Keeping correct utility levels in such a strong model of statistical indistinguishability looks difficult to be achieved with the usual differential privacy toolbox because it would typically scale in the worst case the sensitivity by the sample size and so the noise variance by up to its square. We introduce a trick specific to sampling that bypasses the sensitivity analysis. Privacy enforces an information theoretic barrier on approximation, and we show how to reach this barrier with guarantees on the approximation of the target non private density. We do so using a recent approach to non private density estimation relying on the original boosting theory, learning the sufficient statistics of an exponential family with classifiers. Approximation guarantees cover the mode capture problem. In the context of learning, the sampling problem is particularly important: because integral privacy enjoys the same closure under post-processing as differential privacy does, any algorithm using integrally privacy sampled data would result in an output equally integrally private. We also show that this brings fairness guarantees on post-processing that would eventually elude classical differential privacy: any decision process has bounded data-dependent bias when the data is integrally privately sampled. Experimental results against private kernel density estimation and private GANs displays the quality of our results.
Tasks	Density Estimation
Published	2018-06-13
URL	https://arxiv.org/abs/1806.04819v5
PDF	https://arxiv.org/pdf/1806.04819v5.pdf
PWC	https://paperswithcode.com/paper/integral-privacy-for-sampling-from-mollifier
Repo	https://github.com/karokaram/PrivatedBoostedDensities
Framework	none

A Reductions Approach to Fair Classification


Title	A Reductions Approach to Fair Classification
Authors	Alekh Agarwal, Alina Beygelzimer, Miroslav Dudík, John Langford, Hanna Wallach
Abstract	We present a systematic approach for achieving fairness in a binary classification setting. While we focus on two well-known quantitative definitions of fairness, our approach encompasses many other previously studied definitions as special cases. The key idea is to reduce fair classification to a sequence of cost-sensitive classification problems, whose solutions yield a randomized classifier with the lowest (empirical) error subject to the desired constraints. We introduce two reductions that work for any representation of the cost-sensitive classifier and compare favorably to prior baselines on a variety of data sets, while overcoming several of their disadvantages.
Tasks
Published	2018-03-06
URL	http://arxiv.org/abs/1803.02453v3
PDF	http://arxiv.org/pdf/1803.02453v3.pdf
PWC	https://paperswithcode.com/paper/a-reductions-approach-to-fair-classification
Repo	https://github.com/Microsoft/fairlearn
Framework	none

MCRM: Mother Compact Recurrent Memory


Title	MCRM: Mother Compact Recurrent Memory
Authors	Abduallah A. Mohamed, Christian Claudel
Abstract	LSTMs and GRUs are the most common recurrent neural network architectures used to solve temporal sequence problems. The two architectures have differing data flows dealing with a common component called the cell state (also referred to as the memory). We attempt to enhance the memory by presenting a modification that we call the Mother Compact Recurrent Memory (MCRM). MCRMs are a type of a nested LSTM-GRU architecture where the cell state is the GRU hidden state. The concatenation of the forget gate and input gate interactions from the LSTM are considered an input to the GRU cell. Because MCRMs has this type of nesting, MCRMs have a compact memory pattern consisting of neurons that acts explicitly in both long-term and short-term fashions. For some specific tasks, empirical results show that MCRMs outperform previously used architectures.
Tasks
Published	2018-08-04
URL	https://arxiv.org/abs/1808.02016v3
PDF	https://arxiv.org/pdf/1808.02016v3.pdf
PWC	https://paperswithcode.com/paper/mcrm-mother-compact-recurrent-memory
Repo	https://github.com/abduallahmohamed/MCRM
Framework	pytorch

Seq2Seq-Vis: A Visual Debugging Tool for Sequence-to-Sequence Models


Title	Seq2Seq-Vis: A Visual Debugging Tool for Sequence-to-Sequence Models
Authors	Hendrik Strobelt, Sebastian Gehrmann, Michael Behrisch, Adam Perer, Hanspeter Pfister, Alexander M. Rush
Abstract	Neural Sequence-to-Sequence models have proven to be accurate and robust for many sequence prediction tasks, and have become the standard approach for automatic translation of text. The models work in a five stage blackbox process that involves encoding a source sequence to a vector space and then decoding out to a new target sequence. This process is now standard, but like many deep learning methods remains quite difficult to understand or debug. In this work, we present a visual analysis tool that allows interaction with a trained sequence-to-sequence model through each stage of the translation process. The aim is to identify which patterns have been learned and to detect model errors. We demonstrate the utility of our tool through several real-world large-scale sequence-to-sequence use cases.
Tasks
Published	2018-04-25
URL	http://arxiv.org/abs/1804.09299v2
PDF	http://arxiv.org/pdf/1804.09299v2.pdf
PWC	https://paperswithcode.com/paper/seq2seq-vis-a-visual-debugging-tool-for
Repo	https://github.com/HendrikStrobelt/Seq2Seq-Vis
Framework	none

Graph Convolutional Reinforcement Learning


Title	Graph Convolutional Reinforcement Learning
Authors	Jiechuan Jiang, Chen Dun, Tiejun Huang, Zongqing Lu
Abstract	Learning to cooperate is crucially important in multi-agent environments. The key is to understand the mutual interplay between agents. However, multi-agent environments are highly dynamic, where agents keep moving and their neighbors change quickly. This makes it hard to learn abstract representations of mutual interplay between agents. To tackle these difficulties, we propose graph convolutional reinforcement learning, where graph convolution adapts to the dynamics of the underlying graph of the multi-agent environment, and relation kernels capture the interplay between agents by their relation representations. Latent features produced by convolutional layers from gradually increased receptive fields are exploited to learn cooperation, and cooperation is further improved by temporal relation regularization for consistency. Empirically, we show that our method substantially outperforms existing methods in a variety of cooperative scenarios.
Tasks	Decision Making
Published	2018-10-22
URL	https://arxiv.org/abs/1810.09202v5
PDF	https://arxiv.org/pdf/1810.09202v5.pdf
PWC	https://paperswithcode.com/paper/graph-convolutional-reinforcement-learning
Repo	https://github.com/PKU-AI-Edge/DGN
Framework	tf

Evolutionary Data Measures: Understanding the Difficulty of Text Classification Tasks


Title	Evolutionary Data Measures: Understanding the Difficulty of Text Classification Tasks
Authors	Edward Collins, Nikolai Rozanov, Bingbing Zhang
Abstract	Classification tasks are usually analysed and improved through new model architectures or hyperparameter optimisation but the underlying properties of datasets are discovered on an ad-hoc basis as errors occur. However, understanding the properties of the data is crucial in perfecting models. In this paper we analyse exactly which characteristics of a dataset best determine how difficult that dataset is for the task of text classification. We then propose an intuitive measure of difficulty for text classification datasets which is simple and fast to calculate. We show that this measure generalises to unseen data by comparing it to state-of-the-art datasets and results. This measure can be used to analyse the precise source of errors in a dataset and allows fast estimation of how difficult a dataset is to learn. We searched for this measure by training 12 classical and neural network based models on 78 real-world datasets, then use a genetic algorithm to discover the best measure of difficulty. Our difficulty-calculating code ( https://github.com/Wluper/edm ) and datasets ( http://data.wluper.com ) are publicly available.
Tasks	Text Classification
Published	2018-11-05
URL	http://arxiv.org/abs/1811.01910v2
PDF	http://arxiv.org/pdf/1811.01910v2.pdf
PWC	https://paperswithcode.com/paper/evolutionary-data-measures-understanding-the
Repo	https://github.com/Wluper/edm
Framework	none

Deep Learning on Graphs: A Survey


Title	Deep Learning on Graphs: A Survey
Authors	Ziwei Zhang, Peng Cui, Wenwu Zhu
Abstract	Deep learning has been shown to be successful in a number of domains, ranging from acoustics, images, to natural language processing. However, applying deep learning to the ubiquitous graph data is non-trivial because of the unique characteristics of graphs. Recently, substantial research efforts have been devoted to applying deep learning methods to graphs, resulting in beneficial advances in graph analysis techniques. In this survey, we comprehensively review the different types of deep learning methods on graphs. We divide the existing methods into five categories based on their model architectures and training strategies: graph recurrent neural networks, graph convolutional networks, graph autoencoders, graph reinforcement learning, and graph adversarial methods. We then provide a comprehensive overview of these methods in a systematic manner mainly by following their development history. We also analyze the differences and compositions of different methods. Finally, we briefly outline the applications in which they have been used and discuss potential future research directions.
Tasks
Published	2018-12-11
URL	https://arxiv.org/abs/1812.04202v3
PDF	https://arxiv.org/pdf/1812.04202v3.pdf
PWC	https://paperswithcode.com/paper/deep-learning-on-graphs-a-survey
Repo	https://github.com/NorthPolesky/GNNpaper
Framework	none

Statistically Motivated Second Order Pooling


Title	Statistically Motivated Second Order Pooling
Authors	Kaicheng Yu, Mathieu Salzmann
Abstract	Second-order pooling, a.k.a.~bilinear pooling, has proven effective for deep learning based visual recognition. However, the resulting second-order networks yield a final representation that is orders of magnitude larger than that of standard, first-order ones, making them memory-intensive and cumbersome to deploy. Here, we introduce a general, parametric compression strategy that can produce more compact representations than existing compression techniques, yet outperform both compressed and uncompressed second-order models. Our approach is motivated by a statistical analysis of the network’s activations, relying on operations that lead to a Gaussian-distributed final representation, as inherently used by first-order deep networks. As evidenced by our experiments, this lets us outperform the state-of-the-art first-order and second-order models on several benchmark recognition datasets.
Tasks
Published	2018-01-23
URL	http://arxiv.org/abs/1801.07492v3
PDF	http://arxiv.org/pdf/1801.07492v3.pdf
PWC	https://paperswithcode.com/paper/statistically-motivated-second-order-pooling-1
Repo	https://github.com/kcyu2014/smsop
Framework	tf

Auto-Keras: An Efficient Neural Architecture Search System


Title	Auto-Keras: An Efficient Neural Architecture Search System
Authors	Haifeng Jin, Qingquan Song, Xia Hu
Abstract	Neural architecture search (NAS) has been proposed to automatically tune deep neural networks, but existing search algorithms, e.g., NASNet, PNAS, usually suffer from expensive computational cost. Network morphism, which keeps the functionality of a neural network while changing its neural architecture, could be helpful for NAS by enabling more efficient training during the search. In this paper, we propose a novel framework enabling Bayesian optimization to guide the network morphism for efficient neural architecture search. The framework develops a neural network kernel and a tree-structured acquisition function optimization algorithm to efficiently explores the search space. Intensive experiments on real-world benchmark datasets have been done to demonstrate the superior performance of the developed framework over the state-of-the-art methods. Moreover, we build an open-source AutoML system based on our method, namely Auto-Keras. The system runs in parallel on CPU and GPU, with an adaptive search strategy for different GPU memory limits.
Tasks	AutoML, Neural Architecture Search
Published	2018-06-27
URL	http://arxiv.org/abs/1806.10282v3
PDF	http://arxiv.org/pdf/1806.10282v3.pdf
PWC	https://paperswithcode.com/paper/auto-keras-efficient-neural-architecture
Repo	https://github.com/wpsliu123/AUTOKERAS
Framework	none

Information Aggregation via Dynamic Routing for Sequence Encoding


Title	Information Aggregation via Dynamic Routing for Sequence Encoding
Authors	Jingjing Gong, Xipeng Qiu, Shaojing Wang, Xuanjing Huang
Abstract	While much progress has been made in how to encode a text sequence into a sequence of vectors, less attention has been paid to how to aggregate these preceding vectors (outputs of RNN/CNN) into fixed-size encoding vector. Usually, a simple max or average pooling is used, which is a bottom-up and passive way of aggregation and lack of guidance by task information. In this paper, we propose an aggregation mechanism to obtain a fixed-size encoding with a dynamic routing policy. The dynamic routing policy is dynamically deciding that what and how much information need be transferred from each word to the final encoding of the text sequence. Following the work of Capsule Network, we design two dynamic routing policies to aggregate the outputs of RNN/CNN encoding layer into a final encoding vector. Compared to the other aggregation methods, dynamic routing can refine the messages according to the state of final encoding vector. Experimental results on five text classification tasks show that our method outperforms other aggregating models by a significant margin. Related source code is released on our github page.
Tasks	Sentiment Analysis, Text Classification
Published	2018-06-05
URL	http://arxiv.org/abs/1806.01501v1
PDF	http://arxiv.org/pdf/1806.01501v1.pdf
PWC	https://paperswithcode.com/paper/information-aggregation-via-dynamic-routing
Repo	https://github.com/FudanNLP/Capsule4TextClassification
Framework	tf

Don’t Use Large Mini-Batches, Use Local SGD


Title	Don’t Use Large Mini-Batches, Use Local SGD
Authors	Tao Lin, Sebastian U. Stich, Kumar Kshitij Patel, Martin Jaggi
Abstract	Mini-batch stochastic gradient methods (SGD) are state of the art for distributed training of deep neural networks. Drastic increases in the mini-batch sizes have lead to key efficiency and scalability gains in recent years. However, progress faces a major roadblock, as models trained with large batches often do not generalize well, i.e. they do not show good accuracy on new data. As a remedy, we propose a \emph{post-local} SGD and show that it significantly improves the generalization performance compared to large-batch training on standard benchmarks while enjoying the same efficiency (time-to-accuracy) and scalability. We further provide an extensive study of the communication efficiency vs. performance trade-offs associated with a host of \emph{local SGD} variants.
Tasks
Published	2018-08-22
URL	https://arxiv.org/abs/1808.07217v6
PDF	https://arxiv.org/pdf/1808.07217v6.pdf
PWC	https://paperswithcode.com/paper/dont-use-large-mini-batches-use-local-sgd
Repo	https://github.com/BPrasad123/Data_Science_Learning
Framework	tf

General-to-Detailed GAN for Infrequent Class Medical Images


Title	General-to-Detailed GAN for Infrequent Class Medical Images
Authors	Tatsuki Koga, Naoki Nonaka, Jun Sakuma, Jun Seita
Abstract	Deep learning has significant potential for medical imaging. However, since the incident rate of each disease varies widely, the frequency of classes in a medical image dataset is imbalanced, leading to poor accuracy for such infrequent classes. One possible solution is data augmentation of infrequent classes using synthesized images created by Generative Adversarial Networks (GANs), but conventional GANs also require certain amount of images to learn. To overcome this limitation, here we propose General-to-detailed GAN (GDGAN), serially connected two GANs, one for general labels and the other for detailed labels. GDGAN produced diverse medical images, and the network trained with an augmented dataset outperformed other networks using existing methods with respect to Area-Under-Curve (AUC) of Receiver Operating Characteristic (ROC) curve.
Tasks	Data Augmentation
Published	2018-11-28
URL	http://arxiv.org/abs/1812.01690v1
PDF	http://arxiv.org/pdf/1812.01690v1.pdf
PWC	https://paperswithcode.com/paper/general-to-detailed-gan-for-infrequent-class
Repo	https://github.com/seitalab/GDGAN.pytorch
Framework	pytorch

Symbolic inductive bias for visually grounded learning of spoken language


Title	Symbolic inductive bias for visually grounded learning of spoken language
Authors	Grzegorz Chrupała
Abstract	A widespread approach to processing spoken language is to first automatically transcribe it into text. An alternative is to use an end-to-end approach: recent works have proposed to learn semantic embeddings of spoken language from images with spoken captions, without an intermediate transcription step. We propose to use multitask learning to exploit existing transcribed speech within the end-to-end setting. We describe a three-task architecture which combines the objectives of matching spoken captions with corresponding images, speech with text, and text with images. We show that the addition of the speech/text task leads to substantial performance improvements on image retrieval when compared to training the speech/image task in isolation. We conjecture that this is due to a strong inductive bias transcribed speech provides to the model, and offer supporting evidence for this.
Tasks	Image Retrieval
Published	2018-12-21
URL	https://arxiv.org/abs/1812.09244v3
PDF	https://arxiv.org/pdf/1812.09244v3.pdf
PWC	https://paperswithcode.com/paper/symbolic-inductive-bias-for-visually-grounded
Repo	https://github.com/gchrupala/symbolic-bias
Framework	none