October 19, 2019

2810 words 14 mins read

Paper Group ANR 188

DynMat, a network that can learn after learning. Multi-Kernel Diffusion CNNs for Graph-Based Learning on Point Clouds. Video Frame Interpolation by Plug-and-Play Deep Locally Linear Embedding. ToxicBlend: Virtual Screening of Toxic Compounds with Ensemble Predictors. Computational Decomposition of Style for Controllable and Enhanced Style Transfer. …

DynMat, a network that can learn after learning


Title	DynMat, a network that can learn after learning
Authors	Jung H. Lee
Abstract	To survive in the dynamically-evolving world, we accumulate knowledge and improve our skills based on experience. In the process, gaining new knowledge does not disrupt our vigilance to external stimuli. In other words, our learning process is ‘accumulative’ and ‘online’ without interruption. However, despite the recent success, artificial neural networks (ANNs) must be trained offline, and they suffer catastrophic interference between old and new learning, indicating that ANNs’ conventional learning algorithms may not be suitable for building intelligent agents comparable to our brain. In this study, we propose a novel neural network architecture (DynMat) consisting of dual learning systems, inspired by the complementary learning system (CLS) theory suggesting that the brain relies on short- and long-term learning systems to learn continuously. Our experiments show that 1) DynMat can learn a new class without catastrophic interference and 2) it does not strictly require offline training.
Tasks
Published	2018-06-16
URL	http://arxiv.org/abs/1806.06253v2
PDF	http://arxiv.org/pdf/1806.06253v2.pdf
PWC	https://paperswithcode.com/paper/dynmat-a-network-that-can-learn-after
Repo
Framework

Multi-Kernel Diffusion CNNs for Graph-Based Learning on Point Clouds


Title	Multi-Kernel Diffusion CNNs for Graph-Based Learning on Point Clouds
Authors	Lasse Hansen, Jasper Diesel, Mattias P. Heinrich
Abstract	Graph convolutional networks are a new promising learning approach to deal with data on irregular domains. They are predestined to overcome certain limitations of conventional grid-based architectures and will enable efficient handling of point clouds or related graphical data representations, e.g. superpixel graphs. Learning feature extractors and classifiers on 3D point clouds is still an underdeveloped area and has potential restrictions to equal graph topologies. In this work, we derive a new architectural design that combines rotationally and topologically invariant graph diffusion operators and node-wise feature learning through 1x1 convolutions. By combining multiple isotropic diffusion operations based on the Laplace-Beltrami operator, we can learn an optimal linear combination of diffusion kernels for effective feature propagation across nodes on an irregular graph. We validated our approach for learning point descriptors as well as semantic classification on real 3D point clouds of human poses and demonstrate an improvement from 85% to 95% in Dice overlap with our multi-kernel approach.
Tasks
Published	2018-09-14
URL	http://arxiv.org/abs/1809.05370v1
PDF	http://arxiv.org/pdf/1809.05370v1.pdf
PWC	https://paperswithcode.com/paper/multi-kernel-diffusion-cnns-for-graph-based
Repo
Framework

Video Frame Interpolation by Plug-and-Play Deep Locally Linear Embedding


Title	Video Frame Interpolation by Plug-and-Play Deep Locally Linear Embedding
Authors	Anh-Duc Nguyen, Woojae Kim, Jongyoo Kim, Sanghoon Lee
Abstract	We propose a generative framework which takes on the video frame interpolation problem. Our framework, which we call Deep Locally Linear Embedding (DeepLLE), is powered by a deep convolutional neural network (CNN) while it can be used instantly like conventional models. DeepLLE fits an auto-encoding CNN to a set of several consecutive frames and embeds a linearity constraint on the latent codes so that new frames can be generated by interpolating new latent codes. Different from the current deep learning paradigm which requires training on large datasets, DeepLLE works in a plug-and-play and unsupervised manner, and is able to generate an arbitrary number of frames. Thorough experiments demonstrate that without bells and whistles, our method is highly competitive among current state-of-the-art models.
Tasks	Video Frame Interpolation
Published	2018-07-04
URL	http://arxiv.org/abs/1807.01462v1
PDF	http://arxiv.org/pdf/1807.01462v1.pdf
PWC	https://paperswithcode.com/paper/video-frame-interpolation-by-plug-and-play
Repo
Framework

ToxicBlend: Virtual Screening of Toxic Compounds with Ensemble Predictors


Title	ToxicBlend: Virtual Screening of Toxic Compounds with Ensemble Predictors
Authors	Mikhail Zaslavskiy, Simon Jégou, Eric W. Tramel, Gilles Wainrib
Abstract	Timely assessment of compound toxicity is one of the biggest challenges facing the pharmaceutical industry today. A significant proportion of compounds identified as potential leads are ultimately discarded due to the toxicity they induce. In this paper, we propose a novel machine learning approach for the prediction of molecular activity on ToxCast targets. We combine extreme gradient boosting with fully-connected and graph-convolutional neural network architectures trained on QSAR physical molecular property descriptors, PubChem molecular fingerprints, and SMILES sequences. Our ensemble predictor leverages the strengths of each individual technique, significantly outperforming existing state-of-the art models on the ToxCast and Tox21 toxicity-prediction datasets. We provide free access to molecule toxicity prediction using our model at http://www.owkin.com/toxicblend.
Tasks	Drug Discovery
Published	2018-06-12
URL	http://arxiv.org/abs/1806.04449v1
PDF	http://arxiv.org/pdf/1806.04449v1.pdf
PWC	https://paperswithcode.com/paper/toxicblend-virtual-screening-of-toxic
Repo
Framework

Computational Decomposition of Style for Controllable and Enhanced Style Transfer


Title	Computational Decomposition of Style for Controllable and Enhanced Style Transfer
Authors	Minchao Li, Shikui Tu, Lei Xu
Abstract	Neural style transfer has been demonstrated to be powerful in creating artistic image with help of Convolutional Neural Networks (CNN). However, there is still lack of computational analysis of perceptual components of the artistic style. Different from some early attempts which studied the style by some pre-processing or post-processing techniques, we investigate the characteristics of the style systematically based on feature map produced by CNN. First, we computationally decompose the style into basic elements using not only spectrum based methods including Fast Fourier Transform (FFT), Discrete Cosine Transform (DCT) but also latent variable models such Principal Component Analysis (PCA), Independent Component Analysis (ICA). Then, the decomposition of style induces various ways of controlling the style elements which could be embedded as modules in state-of-the-art style transfer algorithms. Such decomposition of style brings several advantages. It enables the computational coding of different artistic styles by our style basis with similar styles clustering together, and thus it facilitates the mixing or intervention of styles based on the style basis from more than one styles so that compound style or new style could be generated to produce styled images. Experiments demonstrate the effectiveness of our method on not only painting style transfer but also sketch style transfer which indicates possible applications on picture-to-sketch problems.
Tasks	Latent Variable Models, Style Transfer
Published	2018-11-21
URL	http://arxiv.org/abs/1811.08668v2
PDF	http://arxiv.org/pdf/1811.08668v2.pdf
PWC	https://paperswithcode.com/paper/computational-decomposition-of-style-for
Repo
Framework

A hybrid deep learning approach for medical relation extraction


Title	A hybrid deep learning approach for medical relation extraction
Authors	Veera Raghavendra Chikka, Kamalakar Karlapalem
Abstract	Mining relationships between treatment(s) and medical problem(s) is vital in the biomedical domain. This helps in various applications, such as decision support system, safety surveillance, and new treatment discovery. We propose a deep learning approach that utilizes both word level and sentence-level representations to extract the relationships between treatment and problem. While deep learning techniques demand a large amount of data for training, we make use of a rule-based system particularly for relationship classes with fewer samples. Our final relations are derived by jointly combining the results from deep learning and rule-based models. Our system achieved a promising performance on the relationship classes of I2b2 2010 relation extraction task.
Tasks	Medical Relation Extraction, Relation Extraction
Published	2018-06-26
URL	http://arxiv.org/abs/1806.11189v1
PDF	http://arxiv.org/pdf/1806.11189v1.pdf
PWC	https://paperswithcode.com/paper/a-hybrid-deep-learning-approach-for-medical
Repo
Framework

Block Stability for MAP Inference


Title	Block Stability for MAP Inference
Authors	Hunter Lang, David Sontag, Aravindan Vijayaraghavan
Abstract	To understand the empirical success of approximate MAP inference, recent work (Lang et al., 2018) has shown that some popular approximation algorithms perform very well when the input instance is stable. The simplest stability condition assumes that the MAP solution does not change at all when some of the pairwise potentials are (adversarially) perturbed. Unfortunately, this strong condition does not seem to be satisfied in practice. In this paper, we introduce a significantly more relaxed condition that only requires blocks (portions) of an input instance to be stable. Under this block stability condition, we prove that the pairwise LP relaxation is persistent on the stable blocks. We complement our theoretical results with an empirical evaluation of real-world MAP inference instances from computer vision. We design an algorithm to find stable blocks, and find that these real instances have large stable regions. Our work gives a theoretical explanation for the widespread empirical phenomenon of persistency for this LP relaxation.
Tasks
Published	2018-10-12
URL	http://arxiv.org/abs/1810.05305v1
PDF	http://arxiv.org/pdf/1810.05305v1.pdf
PWC	https://paperswithcode.com/paper/block-stability-for-map-inference
Repo
Framework

Disease phenotyping using deep learning: A diabetes case study


Title	Disease phenotyping using deep learning: A diabetes case study
Authors	Sina Rashidian, Janos Hajagos, Richard Moffitt, Fusheng Wang, Xinyu Dong, Kayley Abell-Hart, Kimberly Noel, Rajarsi Gupta, Mathew Tharakan, Veena Lingam, Joel Saltz, Mary Saltz
Abstract	Characterization of a patient clinical phenotype is central to biomedical informatics. ICD codes, assigned to inpatient encounters by coders, is important for population health and cohort discovery when clinical information is limited. While ICD codes are assigned to patients by professionals trained and certified in coding there is substantial variability in coding. We present a methodology that uses deep learning methods to model coder decision making and that predicts ICD codes. Our approach predicts codes based on demographics, lab results, and medications, as well as codes from previous encounters. We are able to predict existing codes with high accuracy for all three of the test cases we investigated: diabetes, acute renal failure, and chronic kidney disease. We employed a panel of clinicians, in a blinded manner, to assess ground truth and compared the predictions of coders, model and clinicians. When disparities between the model prediction and coder assigned codes were reviewed, our model outperformed coder assigned ICD codes.
Tasks	Decision Making
Published	2018-11-28
URL	http://arxiv.org/abs/1811.11818v1
PDF	http://arxiv.org/pdf/1811.11818v1.pdf
PWC	https://paperswithcode.com/paper/disease-phenotyping-using-deep-learning-a
Repo
Framework

Particle-based pedestrian path prediction using LSTM-MDL models


Title	Particle-based pedestrian path prediction using LSTM-MDL models
Authors	Ronny Hug, Stefan Becker, Wolfgang Hübner, Michael Arens
Abstract	Recurrent neural networks are able to learn complex long-term relationships from sequential data and output a pdf over the state space. Therefore, recurrent models are a natural choice to address path prediction tasks, where a trained model is used to generate future expectations from past observations. When applied to security applications, like predicting the path of pedestrians for risk assessment, a point-wise greedy (ML) evaluation of the output pdf is not feasible, since the environment often allows multiple choices. Therefore, a robust risk assessment has to take all options into account, even if they are overall not very likely. Towards this end, a combination of particle filter sampling strategies and a LSTM-MDL model is proposed to address a multi-modal path prediction task. The capabilities and viability of the proposed approach are evaluated on several synthetic test conditions, yielding the counter-intuitive result that the simplest approach performs best. Further, the feasibility of the proposed approach is illustrated on several real world scenes.
Tasks
Published	2018-04-16
URL	http://arxiv.org/abs/1804.05546v3
PDF	http://arxiv.org/pdf/1804.05546v3.pdf
PWC	https://paperswithcode.com/paper/particle-based-pedestrian-path-prediction
Repo
Framework

ESPnet: End-to-End Speech Processing Toolkit


Title	ESPnet: End-to-End Speech Processing Toolkit
Authors	Shinji Watanabe, Takaaki Hori, Shigeki Karita, Tomoki Hayashi, Jiro Nishitoba, Yuya Unno, Nelson Enrique Yalta Soplin, Jahn Heymann, Matthew Wiesner, Nanxin Chen, Adithya Renduchintala, Tsubasa Ochiai
Abstract	This paper introduces a new open source platform for end-to-end speech processing named ESPnet. ESPnet mainly focuses on end-to-end automatic speech recognition (ASR), and adopts widely-used dynamic neural network toolkits, Chainer and PyTorch, as a main deep learning engine. ESPnet also follows the Kaldi ASR toolkit style for data processing, feature extraction/format, and recipes to provide a complete setup for speech recognition and other speech processing experiments. This paper explains a major architecture of this software platform, several important functionalities, which differentiate ESPnet from other open source ASR toolkits, and experimental results with major ASR benchmarks.
Tasks	Speech Recognition
Published	2018-03-30
URL	http://arxiv.org/abs/1804.00015v1
PDF	http://arxiv.org/pdf/1804.00015v1.pdf
PWC	https://paperswithcode.com/paper/espnet-end-to-end-speech-processing-toolkit
Repo
Framework

Robustly Disentangled Causal Mechanisms: Validating Deep Representations for Interventional Robustness


Title	Robustly Disentangled Causal Mechanisms: Validating Deep Representations for Interventional Robustness
Authors	Raphael Suter, Đorđe Miladinović, Bernhard Schölkopf, Stefan Bauer
Abstract	The ability to learn disentangled representations that split underlying sources of variation in high dimensional, unstructured data is important for data efficient and robust use of neural networks. While various approaches aiming towards this goal have been proposed in recent times, a commonly accepted definition and validation procedure is missing. We provide a causal perspective on representation learning which covers disentanglement and domain shift robustness as special cases. Our causal framework allows us to introduce a new metric for the quantitative evaluation of deep latent variable models. We show how this metric can be estimated from labeled observational data and further provide an efficient estimation algorithm that scales linearly in the dataset size.
Tasks	Latent Variable Models, Representation Learning
Published	2018-10-31
URL	https://arxiv.org/abs/1811.00007v2
PDF	https://arxiv.org/pdf/1811.00007v2.pdf
PWC	https://paperswithcode.com/paper/interventional-robustness-of-deep-latent
Repo
Framework

Extractive Summary as Discrete Latent Variables


Title	Extractive Summary as Discrete Latent Variables
Authors	Aran Komatsuzaki
Abstract	In this paper, we compare various methods to compress a text using a neural model. We find that extracting tokens as latent variables significantly outperforms the state-of-the-art discrete latent variable models such as VQ-VAE. Furthermore, we compare various extractive compression schemes. There are two best-performing methods that perform equally. One method is to simply choose the tokens with the highest tf-idf scores. Another is to train a bidirectional language model similar to ELMo and choose the tokens with the highest loss. If we consider any subsequence of a text to be a text in a broader sense, we conclude that language is a strong compression code of itself. Our finding justifies the high quality of generation achieved with hierarchical method, as their latent variables are nothing but natural language summary. We also conclude that there is a hierarchy in language such that an entire text can be predicted much more easily based on a sequence of a small number of keywords, which can be easily found by classical methods as tf-idf. We speculate that this extraction process may be useful for unsupervised hierarchical text generation.
Tasks	Language Modelling, Latent Variable Models, Text Generation
Published	2018-11-14
URL	http://arxiv.org/abs/1811.05542v2
PDF	http://arxiv.org/pdf/1811.05542v2.pdf
PWC	https://paperswithcode.com/paper/extractive-summary-as-discrete-latent
Repo
Framework

Variational Noise-Contrastive Estimation


Title	Variational Noise-Contrastive Estimation
Authors	Benjamin Rhodes, Michael Gutmann
Abstract	Unnormalised latent variable models are a broad and flexible class of statistical models. However, learning their parameters from data is intractable, and few estimation techniques are currently available for such models. To increase the number of techniques in our arsenal, we propose variational noise-contrastive estimation (VNCE), building on NCE which is a method that only applies to unnormalised models. The core idea is to use a variational lower bound to the NCE objective function, which can be optimised in the same fashion as the evidence lower bound (ELBO) in standard variational inference (VI). We prove that VNCE can be used for both parameter estimation of unnormalised models and posterior inference of latent variables. The developed theory shows that VNCE has the same level of generality as standard VI, meaning that advances made there can be directly imported to the unnormalised setting. We validate VNCE on toy models and apply it to a realistic problem of estimating an undirected graphical model from incomplete data.
Tasks	Latent Variable Models
Published	2018-10-18
URL	http://arxiv.org/abs/1810.08010v3
PDF	http://arxiv.org/pdf/1810.08010v3.pdf
PWC	https://paperswithcode.com/paper/variational-noise-contrastive-estimation
Repo
Framework

Decomposing feature-level variation with Covariate Gaussian Process Latent Variable Models


Title	Decomposing feature-level variation with Covariate Gaussian Process Latent Variable Models
Authors	Kaspar Märtens, Kieran R. Campbell, Christopher Yau
Abstract	The interpretation of complex high-dimensional data typically requires the use of dimensionality reduction techniques to extract explanatory low-dimensional representations. However, in many real-world problems these representations may not be sufficient to aid interpretation on their own, and it would be desirable to interpret the model in terms of the original features themselves. Our goal is to characterise how feature-level variation depends on latent low-dimensional representations, external covariates, and non-linear interactions between the two. In this paper, we propose to achieve this through a structured kernel decomposition in a hybrid Gaussian Process model which we call the Covariate Gaussian Process Latent Variable Model (c-GPLVM). We demonstrate the utility of our model on simulated examples and applications in disease progression modelling from high-dimensional gene expression data in the presence of additional phenotypes. In each setting we show how the c-GPLVM can extract low-dimensional structures from high-dimensional data sets whilst allowing a breakdown of feature-level variability that is not present in other commonly used dimensionality reduction approaches.
Tasks	Dimensionality Reduction, Latent Variable Models
Published	2018-10-16
URL	https://arxiv.org/abs/1810.06983v2
PDF	https://arxiv.org/pdf/1810.06983v2.pdf
PWC	https://paperswithcode.com/paper/covariate-gaussian-process-latent-variable
Repo
Framework

WaveletNet: Logarithmic Scale Efficient Convolutional Neural Networks for Edge Devices


Title	WaveletNet: Logarithmic Scale Efficient Convolutional Neural Networks for Edge Devices
Authors	Li Jing, Rumen Dangovski, Marin Soljacic
Abstract	We present a logarithmic-scale efficient convolutional neural network architecture for edge devices, named WaveletNet. Our model is based on the well-known depthwise convolution, and on two new layers, which we introduce in this work: a wavelet convolution and a depthwise fast wavelet transform. By breaking the symmetry in channel dimensions and applying a fast algorithm, WaveletNet shrinks the complexity of convolutional blocks by an O(logD/D) factor, where D is the number of channels. Experiments on CIFAR-10 and ImageNet classification show superior and comparable performances of WaveletNet compared to state-of-the-art models such as MobileNetV2.
Tasks
Published	2018-11-28
URL	http://arxiv.org/abs/1811.11644v1
PDF	http://arxiv.org/pdf/1811.11644v1.pdf
PWC	https://paperswithcode.com/paper/waveletnet-logarithmic-scale-efficient
Repo
Framework