October 18, 2019

3015 words 15 mins read

Paper Group ANR 454

Identifiability of Nonparametric Mixture Models and Bayes Optimal Clustering. Deep nested level sets: Fully automated segmentation of cardiac MR images in patients with pulmonary hypertension. Building a Neural Semantic Parser from a Domain Ontology. Mixed-Stationary Gaussian Process for Flexible Non-Stationary Modeling of Spatial Outcomes. Self-Su …

Identifiability of Nonparametric Mixture Models and Bayes Optimal Clustering


Title	Identifiability of Nonparametric Mixture Models and Bayes Optimal Clustering
Authors	Bryon Aragam, Chen Dan, Eric P. Xing, Pradeep Ravikumar
Abstract	Motivated by problems in data clustering, we establish general conditions under which families of nonparametric mixture models are identifiable, by introducing a novel framework involving clustering overfitted \emph{parametric} (i.e. misspecified) mixture models. These identifiability conditions generalize existing conditions in the literature, and are flexible enough to include for example mixtures of Gaussian mixtures. In contrast to the recent literature on estimating nonparametric mixtures, we allow for general nonparametric mixture components, and instead impose regularity assumptions on the underlying mixing measure. As our primary application, we apply these results to partition-based clustering, generalizing the notion of a Bayes optimal partition from classical parametric model-based clustering to nonparametric settings. Furthermore, this framework is constructive so that it yields a practical algorithm for learning identified mixtures, which is illustrated through several examples on real data. The key conceptual device in the analysis is the convex, metric geometry of probability measures on metric spaces and its connection to the Wasserstein convergence of mixing measures. The result is a flexible framework for nonparametric clustering with formal consistency guarantees.
Tasks
Published	2018-02-12
URL	https://arxiv.org/abs/1802.04397v4
PDF	https://arxiv.org/pdf/1802.04397v4.pdf
PWC	https://paperswithcode.com/paper/identifiability-of-nonparametric-mixture
Repo
Framework

Deep nested level sets: Fully automated segmentation of cardiac MR images in patients with pulmonary hypertension


Title	Deep nested level sets: Fully automated segmentation of cardiac MR images in patients with pulmonary hypertension
Authors	Jinming Duan, Jo Schlemper, Wenjia Bai, Timothy J W Dawes, Ghalib Bello, Georgia Doumou, Antonio De Marvao, Declan P O’Regan, Daniel Rueckert
Abstract	In this paper we introduce a novel and accurate optimisation method for segmentation of cardiac MR (CMR) images in patients with pulmonary hypertension (PH). The proposed method explicitly takes into account the image features learned from a deep neural network. To this end, we estimate simultaneous probability maps over region and edge locations in CMR images using a fully convolutional network. Due to the distinct morphology of the heart in patients with PH, these probability maps can then be incorporated in a single nested level set optimisation framework to achieve multi-region segmentation with high efficiency. The proposed method uses an automatic way for level set initialisation and thus the whole optimisation is fully automated. We demonstrate that the proposed deep nested level set (DNLS) method outperforms existing state-of-the-art methods for CMR segmentation in PH patients.
Tasks
Published	2018-07-27
URL	http://arxiv.org/abs/1807.10760v1
PDF	http://arxiv.org/pdf/1807.10760v1.pdf
PWC	https://paperswithcode.com/paper/deep-nested-level-sets-fully-automated
Repo
Framework

Building a Neural Semantic Parser from a Domain Ontology


Title	Building a Neural Semantic Parser from a Domain Ontology
Authors	Jianpeng Cheng, Siva Reddy, Mirella Lapata
Abstract	Semantic parsing is the task of converting natural language utterances into machine interpretable meaning representations which can be executed against a real-world environment such as a database. Scaling semantic parsing to arbitrary domains faces two interrelated challenges: obtaining broad coverage training data effectively and cheaply; and developing a model that generalizes to compositional utterances and complex intentions. We address these challenges with a framework which allows to elicit training data from a domain ontology and bootstrap a neural parser which recursively builds derivations of logical forms. In our framework meaning representations are described by sequences of natural language templates, where each template corresponds to a decomposed fragment of the underlying meaning representation. Although artificial, templates can be understood and paraphrased by humans to create natural utterances, resulting in parallel triples of utterances, meaning representations, and their decompositions. These allow us to train a neural semantic parser which learns to compose rules in deriving meaning representations. We crowdsource training data on six domains, covering both single-turn utterances which exhibit rich compositionality, and sequential utterances where a complex task is procedurally performed in steps. We then develop neural semantic parsers which perform such compositional tasks. In general, our approach allows to deploy neural semantic parsers quickly and cheaply from a given domain ontology.
Tasks	Semantic Parsing
Published	2018-12-25
URL	http://arxiv.org/abs/1812.10037v1
PDF	http://arxiv.org/pdf/1812.10037v1.pdf
PWC	https://paperswithcode.com/paper/building-a-neural-semantic-parser-from-a
Repo
Framework

Mixed-Stationary Gaussian Process for Flexible Non-Stationary Modeling of Spatial Outcomes


Title	Mixed-Stationary Gaussian Process for Flexible Non-Stationary Modeling of Spatial Outcomes
Authors	Leo L. Duan, Xia Wang, Rhonda D. Szczesniak
Abstract	Gaussian processes (GPs) are commonplace in spatial statistics. Although many non-stationary models have been developed, there is arguably a lack of flexibility compared to equipping each location with its own parameters. However, the latter suffers from intractable computation and can lead to overfitting. Taking the instantaneous stationarity idea, we construct a non-stationary GP with the stationarity parameter individually set at each location. Then we utilize the non-parametric mixture model to reduce the effective number of unique parameters. Different from a simple mixture of independent GPs, the mixture in stationarity allows the components to be spatial correlated, leading to improved prediction efficiency. Theoretical properties are examined and a linearly scalable algorithm is provided. The application is shown through several simulated scenarios as well as the massive spatiotemporally correlated temperature data.
Tasks	Gaussian Processes
Published	2018-07-17
URL	http://arxiv.org/abs/1807.06656v1
PDF	http://arxiv.org/pdf/1807.06656v1.pdf
PWC	https://paperswithcode.com/paper/mixed-stationary-gaussian-process-for
Repo
Framework

Self-Supervised Feature Learning by Learning to Spot Artifacts


Title	Self-Supervised Feature Learning by Learning to Spot Artifacts
Authors	Simon Jenni, Paolo Favaro
Abstract	We introduce a novel self-supervised learning method based on adversarial training. Our objective is to train a discriminator network to distinguish real images from images with synthetic artifacts, and then to extract features from its intermediate layers that can be transferred to other data domains and tasks. To generate images with artifacts, we pre-train a high-capacity autoencoder and then we use a damage and repair strategy: First, we freeze the autoencoder and damage the output of the encoder by randomly dropping its entries. Second, we augment the decoder with a repair network, and train it in an adversarial manner against the discriminator. The repair network helps generate more realistic images by inpainting the dropped feature entries. To make the discriminator focus on the artifacts, we also make it predict what entries in the feature were dropped. We demonstrate experimentally that features learned by creating and spotting artifacts achieve state of the art performance in several benchmarks.
Tasks
Published	2018-06-13
URL	http://arxiv.org/abs/1806.05024v1
PDF	http://arxiv.org/pdf/1806.05024v1.pdf
PWC	https://paperswithcode.com/paper/self-supervised-feature-learning-by-learning
Repo
Framework

Principled Deep Neural Network Training through Linear Programming


Title	Principled Deep Neural Network Training through Linear Programming
Authors	Daniel Bienstock, Gonzalo Muñoz, Sebastian Pokutta
Abstract	Deep Learning has received significant attention due to its impressive performance in many state-of-the-art learning tasks. Unfortunately, while very powerful, Deep Learning is not well understood theoretically and in particular only recently results for the complexity of training deep neural networks have been obtained. In this work we show that large classes of deep neural networks with various architectures (e.g., DNNs, CNNs, Binary Neural Networks, and ResNets), activation functions (e.g., ReLUs and leaky ReLUs), and loss functions (e.g., Hinge loss, Euclidean loss, etc) can be trained to near optimality with desired target accuracy using linear programming in time that is exponential in the input data and parameter space dimension and polynomial in the size of the data set; improvements of the dependence in the input dimension are known to be unlikely assuming $P\neq NP$, and improving the dependence on the parameter space dimension remains open. In particular, we obtain polynomial time algorithms for training for a given fixed network architecture. Our work applies more broadly to empirical risk minimization problems which allows us to generalize various previous results and obtain new complexity results for previously unstudied architectures in the proper learning setting.
Tasks
Published	2018-10-07
URL	http://arxiv.org/abs/1810.03218v2
PDF	http://arxiv.org/pdf/1810.03218v2.pdf
PWC	https://paperswithcode.com/paper/principled-deep-neural-network-training
Repo
Framework

Sensor Adaptation for Improved Semantic Segmentation of Overhead Imagery


Title	Sensor Adaptation for Improved Semantic Segmentation of Overhead Imagery
Authors	Marc Bosch, Gordon A. Christie, Christopher M. Gifford
Abstract	Semantic segmentation is a powerful method to facilitate visual scene understanding. Each pixel is assigned a label according to a pre-defined list of object classes and semantic entities. This becomes very useful as a means to summarize large scale overhead imagery. In this paper we present our work on semantic segmentation with applications to overhead imagery. We propose an algorithm that builds and extends upon the DeepLab framework to be able to refine and resolve small objects (relative to the image size) such as vehicles. We have also investigated sensor adaptation as a means to augment available training data to be able to reduce some of the shortcomings of neural networks when deployed in new environments and to new sensors. We report results on several datasets and compare performance with other state-of-the-art architectures.
Tasks	Scene Understanding, Semantic Segmentation
Published	2018-11-20
URL	http://arxiv.org/abs/1811.08328v1
PDF	http://arxiv.org/pdf/1811.08328v1.pdf
PWC	https://paperswithcode.com/paper/sensor-adaptation-for-improved-semantic
Repo
Framework

Hi-Fi: Hierarchical Feature Integration for Skeleton Detection


Title	Hi-Fi: Hierarchical Feature Integration for Skeleton Detection
Authors	Kai Zhao, Wei Shen, Shanghua Gao, Dandan Li, Ming-Ming Cheng
Abstract	In natural images, the scales (thickness) of object skeletons may dramatically vary among objects and object parts, making object skeleton detection a challenging problem. We present a new convolutional neural network (CNN) architecture by introducing a novel hierarchical feature integration mechanism, named Hi-Fi, to address the skeleton detection problem. The proposed CNN-based approach has a powerful multi-scale feature integration ability that intrinsically captures high-level semantics from deeper layers as well as low-level details from shallower layers. % By hierarchically integrating different CNN feature levels with bidirectional guidance, our approach (1) enables mutual refinement across features of different levels, and (2) possesses the strong ability to capture both rich object context and high-resolution details. Experimental results show that our method significantly outperforms the state-of-the-art methods in terms of effectively fusing features from very different scales, as evidenced by a considerable performance improvement on several benchmarks.
Tasks	Object Skeleton Detection
Published	2018-01-05
URL	http://arxiv.org/abs/1801.01849v4
PDF	http://arxiv.org/pdf/1801.01849v4.pdf
PWC	https://paperswithcode.com/paper/hi-fi-hierarchical-feature-integration-for
Repo
Framework

Respond-CAM: Analyzing Deep Models for 3D Imaging Data by Visualizations


Title	Respond-CAM: Analyzing Deep Models for 3D Imaging Data by Visualizations
Authors	Guannan Zhao, Bo Zhou, Kaiwen Wang, Rui Jiang, Min Xu
Abstract	The convolutional neural network (CNN) has become a powerful tool for various biomedical image analysis tasks, but there is a lack of visual explanation for the machinery of CNNs. In this paper, we present a novel algorithm, Respond-weighted Class Activation Mapping (Respond-CAM), for making CNN-based models interpretable by visualizing input regions that are important for predictions, especially for biomedical 3D imaging data inputs. Our method uses the gradients of any target concept (e.g. the score of target class) that flows into a convolutional layer. The weighted feature maps are combined to produce a heatmap that highlights the important regions in the image for predicting the target concept. We prove a preferable sum-to-score property of the Respond-CAM and verify its significant improvement on 3D images from the current state-of-the-art approach. Our tests on Cellular Electron Cryo-Tomography 3D images show that Respond-CAM achieves superior performance on visualizing the CNNs with 3D biomedical images inputs, and is able to get reasonably good results on visualizing the CNNs with natural image inputs. The Respond-CAM is an efficient and reliable approach for visualizing the CNN machinery, and is applicable to a wide variety of CNN model families and image analysis tasks.
Tasks
Published	2018-05-31
URL	http://arxiv.org/abs/1806.00102v2
PDF	http://arxiv.org/pdf/1806.00102v2.pdf
PWC	https://paperswithcode.com/paper/respond-cam-analyzing-deep-models-for-3d
Repo
Framework

Comparison of Feature Extraction Methods and Predictors for Income Inference


Title	Comparison of Feature Extraction Methods and Predictors for Income Inference
Authors	Martin Fixman, Martin Minnoni, Carlos Sarraute
Abstract	Patterns of mobile phone communications, coupled with the information of the social network graph and financial behavior, allow us to make inferences of users’ socio-economic attributes such as their income level. We present here several methods to extract features from mobile phone usage (calls and messages), and compare different combinations of supervised machine learning techniques and sets of features used as input for the inference of users’ income. Our experimental results show that the Bayesian method based on the communication graph outperforms standard machine learning algorithms using node-based features.
Tasks
Published	2018-11-13
URL	http://arxiv.org/abs/1811.05375v1
PDF	http://arxiv.org/pdf/1811.05375v1.pdf
PWC	https://paperswithcode.com/paper/comparison-of-feature-extraction-methods-and
Repo
Framework

RAFP-Pred: Robust Prediction of Antifreeze Proteins using Localized Analysis of n-Peptide Compositions


Title	RAFP-Pred: Robust Prediction of Antifreeze Proteins using Localized Analysis of n-Peptide Compositions
Authors	Shujaat Khan, Imran Naseem, Roberto Togneri, Mohammed Bennamoun
Abstract	In extreme cold weather, living organisms produce Antifreeze Proteins (AFPs) to counter the otherwise lethal intracellular formation of ice. Structures and sequences of various AFPs exhibit a high degree of heterogeneity, consequently the prediction of the AFPs is considered to be a challenging task. In this research, we propose to handle this arduous manifold learning task using the notion of localized processing. In particular an AFP sequence is segmented into two sub-segments each of which is analyzed for amino acid and di-peptide compositions. We propose to use only the most significant features using the concept of information gain (IG) followed by a random forest classification approach. The proposed RAFP-Pred achieved an excellent performance on a number of standard datasets. We report a high Youden’s index (sensitivity+specificity-1) value of 0.75 on the standard independent test data set outperforming the AFP-PseAAC, AFP_PSSM, AFP-Pred and iAFP by a margin of 0.05, 0.06, 0.14 and 0.68 respectively. The verification rate on the UniProKB dataset is found to be 83.19% which is substantially superior to the 57.18% reported for the iAFP method.
Tasks
Published	2018-09-25
URL	http://arxiv.org/abs/1809.09620v1
PDF	http://arxiv.org/pdf/1809.09620v1.pdf
PWC	https://paperswithcode.com/paper/rafp-pred-robust-prediction-of-antifreeze
Repo
Framework

A Deep Reinforcement Learning Chatbot (Short Version)


Title	A Deep Reinforcement Learning Chatbot (Short Version)
Authors	Iulian V. Serban, Chinnadhurai Sankar, Mathieu Germain, Saizheng Zhang, Zhouhan Lin, Sandeep Subramanian, Taesup Kim, Michael Pieper, Sarath Chandar, Nan Rosemary Ke, Sai Rajeswar, Alexandre de Brebisson, Jose M. R. Sotelo, Dendi Suhubdy, Vincent Michalski, Alexandre Nguyen, Joelle Pineau, Yoshua Bengio
Abstract	We present MILABOT: a deep reinforcement learning chatbot developed by the Montreal Institute for Learning Algorithms (MILA) for the Amazon Alexa Prize competition. MILABOT is capable of conversing with humans on popular small talk topics through both speech and text. The system consists of an ensemble of natural language generation and retrieval models, including neural network and template-based models. By applying reinforcement learning to crowdsourced data and real-world user interactions, the system has been trained to select an appropriate response from the models in its ensemble. The system has been evaluated through A/B testing with real-world users, where it performed significantly better than other systems. The results highlight the potential of coupling ensemble systems with deep reinforcement learning as a fruitful path for developing real-world, open-domain conversational agents.
Tasks	Chatbot, Text Generation
Published	2018-01-20
URL	http://arxiv.org/abs/1801.06700v1
PDF	http://arxiv.org/pdf/1801.06700v1.pdf
PWC	https://paperswithcode.com/paper/a-deep-reinforcement-learning-chatbot-short
Repo
Framework

Emergence of Structured Behaviors from Curiosity-Based Intrinsic Motivation


Title	Emergence of Structured Behaviors from Curiosity-Based Intrinsic Motivation
Authors	Nick Haber, Damian Mrowca, Li Fei-Fei, Daniel L. K. Yamins
Abstract	Infants are experts at playing, with an amazing ability to generate novel structured behaviors in unstructured environments that lack clear extrinsic reward signals. We seek to replicate some of these abilities with a neural network that implements curiosity-driven intrinsic motivation. Using a simple but ecologically naturalistic simulated environment in which the agent can move and interact with objects it sees, the agent learns a world model predicting the dynamic consequences of its actions. Simultaneously, the agent learns to take actions that adversarially challenge the developing world model, pushing the agent to explore novel and informative interactions with its environment. We demonstrate that this policy leads to the self-supervised emergence of a spectrum of complex behaviors, including ego motion prediction, object attention, and object gathering. Moreover, the world model that the agent learns supports improved performance on object dynamics prediction and localization tasks. Our results are a proof-of-principle that computational models of intrinsic motivation might account for key features of developmental visuomotor learning in infants.
Tasks	motion prediction
Published	2018-02-21
URL	http://arxiv.org/abs/1802.07461v1
PDF	http://arxiv.org/pdf/1802.07461v1.pdf
PWC	https://paperswithcode.com/paper/emergence-of-structured-behaviors-from
Repo
Framework

Learning to Play with Intrinsically-Motivated Self-Aware Agents


Title	Learning to Play with Intrinsically-Motivated Self-Aware Agents
Authors	Nick Haber, Damian Mrowca, Li Fei-Fei, Daniel L. K. Yamins
Abstract	Infants are experts at playing, with an amazing ability to generate novel structured behaviors in unstructured environments that lack clear extrinsic reward signals. We seek to mathematically formalize these abilities using a neural network that implements curiosity-driven intrinsic motivation. Using a simple but ecologically naturalistic simulated environment in which an agent can move and interact with objects it sees, we propose a “world-model” network that learns to predict the dynamic consequences of the agent’s actions. Simultaneously, we train a separate explicit “self-model” that allows the agent to track the error map of its own world-model, and then uses the self-model to adversarially challenge the developing world-model. We demonstrate that this policy causes the agent to explore novel and informative interactions with its environment, leading to the generation of a spectrum of complex behaviors, including ego-motion prediction, object attention, and object gathering. Moreover, the world-model that the agent learns supports improved performance on object dynamics prediction, detection, localization and recognition tasks. Taken together, our results are initial steps toward creating flexible autonomous agents that self-supervise in complex novel physical environments.
Tasks	motion prediction
Published	2018-02-21
URL	http://arxiv.org/abs/1802.07442v2
PDF	http://arxiv.org/pdf/1802.07442v2.pdf
PWC	https://paperswithcode.com/paper/learning-to-play-with-intrinsically-motivated
Repo
Framework

Electricity consumption forecasting method based on MPSO-BP neural network model


Title	Electricity consumption forecasting method based on MPSO-BP neural network model
Authors	Youshan Zhang, Liangdong Guo, Qi Li, Junhui Li
Abstract	This paper deals with the problem of the electricity consumption forecasting method. An MPSO-BP (modified particle swarm optimization-back propagation) neural network model is constructed based on the history data of a mineral company of Anshan in China. The simulation showed that the convergence of the algorithm and forecasting accuracy using the obtained model are better than those of other traditional ones, such as BP, PSO, fuzzy neural network and so on. Then we predict the electricity consumption of each month in 2017 based on the MPSO-BP neural network model.
Tasks
Published	2018-10-21
URL	http://arxiv.org/abs/1810.08886v1
PDF	http://arxiv.org/pdf/1810.08886v1.pdf
PWC	https://paperswithcode.com/paper/electricity-consumption-forecasting-method
Repo
Framework