July 29, 2019

2996 words 15 mins read

Paper Group AWR 147

Learning to Generate Reviews and Discovering Sentiment. Resting state fMRI functional connectivity-based classification using a convolutional neural network architecture. Discriminative Region Proposal Adversarial Networks for High-Quality Image-to-Image Translation. Classical Planning in Deep Latent Space: Bridging the Subsymbolic-Symbolic Boundar …

Learning to Generate Reviews and Discovering Sentiment


Title	Learning to Generate Reviews and Discovering Sentiment
Authors	Alec Radford, Rafal Jozefowicz, Ilya Sutskever
Abstract	We explore the properties of byte-level recurrent language models. When given sufficient amounts of capacity, training data, and compute time, the representations learned by these models include disentangled features corresponding to high-level concepts. Specifically, we find a single unit which performs sentiment analysis. These representations, learned in an unsupervised manner, achieve state of the art on the binary subset of the Stanford Sentiment Treebank. They are also very data efficient. When using only a handful of labeled examples, our approach matches the performance of strong baselines trained on full datasets. We also demonstrate the sentiment unit has a direct influence on the generative process of the model. Simply fixing its value to be positive or negative generates samples with the corresponding positive or negative sentiment.
Tasks	Sentiment Analysis, Subjectivity Analysis
Published	2017-04-05
URL	http://arxiv.org/abs/1704.01444v2
PDF	http://arxiv.org/pdf/1704.01444v2.pdf
PWC	https://paperswithcode.com/paper/learning-to-generate-reviews-and-discovering
Repo	https://github.com/faramarzmunshi/d2l-nlp
Framework	tf

Resting state fMRI functional connectivity-based classification using a convolutional neural network architecture


Title	Resting state fMRI functional connectivity-based classification using a convolutional neural network architecture
Authors	Regina Meszlényi, Krisztian Buza, Zoltán Vidnyánszky
Abstract	Machine learning techniques have become increasingly popular in the field of resting state fMRI (functional magnetic resonance imaging) network based classification. However, the application of convolutional networks has been proposed only very recently and has remained largely unexplored. In this paper we describe a convolutional neural network architecture for functional connectome classification called connectome-convolutional neural network (CCNN). Our results on simulated datasets and a publicly available dataset for amnestic mild cognitive impairment classification demonstrate that our CCNN model can efficiently distinguish between subject groups. We also show that the connectome-convolutional network is capable to combine information from diverse functional connectivity metrics and that models using a combination of different connectivity descriptors are able to outperform classifiers using only one metric. From this flexibility follows that our proposed CCNN model can be easily adapted to a wide range of connectome based classification or regression tasks, by varying which connectivity descriptor combinations are used to train the network.
Tasks
Published	2017-07-20
URL	http://arxiv.org/abs/1707.06682v1
PDF	http://arxiv.org/pdf/1707.06682v1.pdf
PWC	https://paperswithcode.com/paper/resting-state-fmri-functional-connectivity
Repo	https://github.com/MRegina/connectome_conv_net
Framework	tf

Discriminative Region Proposal Adversarial Networks for High-Quality Image-to-Image Translation


Title	Discriminative Region Proposal Adversarial Networks for High-Quality Image-to-Image Translation
Authors	Chao Wang, Haiyong Zheng, Zhibin Yu, Ziqiang Zheng, Zhaorui Gu, Bing Zheng
Abstract	Image-to-image translation has been made much progress with embracing Generative Adversarial Networks (GANs). However, it’s still very challenging for translation tasks that require high quality, especially at high-resolution and photorealism. In this paper, we present Discriminative Region Proposal Adversarial Networks (DRPAN) for high-quality image-to-image translation. We decompose the procedure of image-to-image translation task into three iterated steps, first is to generate an image with global structure but some local artifacts (via GAN), second is using our DRPnet to propose the most fake region from the generated image, and third is to implement “image inpainting” on the most fake region for more realistic result through a reviser, so that the system (DRPAN) can be gradually optimized to synthesize images with more attention on the most artifact local part. Experiments on a variety of image-to-image translation tasks and datasets validate that our method outperforms state-of-the-arts for producing high-quality translation results in terms of both human perceptual studies and automatic quantitative measures.
Tasks	Image Inpainting, Image-to-Image Translation
Published	2017-11-27
URL	http://arxiv.org/abs/1711.09554v3
PDF	http://arxiv.org/pdf/1711.09554v3.pdf
PWC	https://paperswithcode.com/paper/discriminative-region-proposal-adversarial
Repo	https://github.com/godisboy/DRPAN
Framework	pytorch

Classical Planning in Deep Latent Space: Bridging the Subsymbolic-Symbolic Boundary


Title	Classical Planning in Deep Latent Space: Bridging the Subsymbolic-Symbolic Boundary
Authors	Masataro Asai, Alex Fukunaga
Abstract	Current domain-independent, classical planners require symbolic models of the problem domain and instance as input, resulting in a knowledge acquisition bottleneck. Meanwhile, although deep learning has achieved significant success in many fields, the knowledge is encoded in a subsymbolic representation which is incompatible with symbolic systems such as planners. We propose LatPlan, an unsupervised architecture combining deep learning and classical planning. Given only an unlabeled set of image pairs showing a subset of transitions allowed in the environment (training inputs), and a pair of images representing the initial and the goal states (planning inputs), LatPlan finds a plan to the goal state in a symbolic latent space and returns a visualized plan execution. The contribution of this paper is twofold: (1) State Autoencoder, which finds a propositional state representation of the environment using a Variational Autoencoder. It generates a discrete latent vector from the images, based on which a PDDL model can be constructed and then solved by an off-the-shelf planner. (2) Action Autoencoder / Discriminator, a neural architecture which jointly finds the action symbols and the implicit action models (preconditions/effects), and provides a successor function for the implicit graph search. We evaluate LatPlan using image-based versions of 3 planning domains: 8-puzzle, Towers of Hanoi and LightsOut.
Tasks
Published	2017-04-29
URL	http://arxiv.org/abs/1705.00154v3
PDF	http://arxiv.org/pdf/1705.00154v3.pdf
PWC	https://paperswithcode.com/paper/classical-planning-in-deep-latent-space
Repo	https://github.com/guicho271828/latplan
Framework	tf

Safe Semi-Supervised Learning of Sum-Product Networks


Title	Safe Semi-Supervised Learning of Sum-Product Networks
Authors	Martin Trapp, Tamas Madl, Robert Peharz, Franz Pernkopf, Robert Trappl
Abstract	In several domains obtaining class annotations is expensive while at the same time unlabelled data are abundant. While most semi-supervised approaches enforce restrictive assumptions on the data distribution, recent work has managed to learn semi-supervised models in a non-restrictive regime. However, so far such approaches have only been proposed for linear models. In this work, we introduce semi-supervised parameter learning for Sum-Product Networks (SPNs). SPNs are deep probabilistic models admitting inference in linear time in number of network edges. Our approach has several advantages, as it (1) allows generative and discriminative semi-supervised learning, (2) guarantees that adding unlabelled data can increase, but not degrade, the performance (safe), and (3) is computationally efficient and does not enforce restrictive assumptions on the data distribution. We show on a variety of data sets that safe semi-supervised learning with SPNs is competitive compared to state-of-the-art and can lead to a better generative and discriminative objective value than a purely supervised approach.
Tasks
Published	2017-10-10
URL	http://arxiv.org/abs/1710.03444v1
PDF	http://arxiv.org/pdf/1710.03444v1.pdf
PWC	https://paperswithcode.com/paper/safe-semi-supervised-learning-of-sum-product
Repo	https://github.com/trappmartin/SSLSPN_UAI2017
Framework	none

Bayesian Verification under Model Uncertainty


Title	Bayesian Verification under Model Uncertainty
Authors	Lenz Belzner, Thomas Gabor
Abstract	Machine learning enables systems to build and update domain models based on runtime observations. In this paper, we study statistical model checking and runtime verification for systems with this ability. Two challenges arise: (1) Models built from limited runtime data yield uncertainty to be dealt with. (2) There is no definition of satisfaction w.r.t. uncertain hypotheses. We propose such a definition of subjective satisfaction based on recently introduced satisfaction functions. We also propose the BV algorithm as a Bayesian solution to runtime verification of subjective satisfaction under model uncertainty. BV provides user-definable stochastic bounds for type I and II errors. We discuss empirical results from an example application to illustrate our ideas.
Tasks
Published	2017-02-28
URL	http://arxiv.org/abs/1702.08725v1
PDF	http://arxiv.org/pdf/1702.08725v1.pdf
PWC	https://paperswithcode.com/paper/bayesian-verification-under-model-uncertainty
Repo	https://github.com/jazzbob/bv
Framework	none

Learning from Complementary Labels


Title	Learning from Complementary Labels
Authors	Takashi Ishida, Gang Niu, Weihua Hu, Masashi Sugiyama
Abstract	Collecting labeled data is costly and thus a critical bottleneck in real-world classification tasks. To mitigate this problem, we propose a novel setting, namely learning from complementary labels for multi-class classification. A complementary label specifies a class that a pattern does not belong to. Collecting complementary labels would be less laborious than collecting ordinary labels, since users do not have to carefully choose the correct class from a long list of candidate classes. However, complementary labels are less informative than ordinary labels and thus a suitable approach is needed to better learn from them. In this paper, we show that an unbiased estimator to the classification risk can be obtained only from complementarily labeled data, if a loss function satisfies a particular symmetric condition. We derive estimation error bounds for the proposed method and prove that the optimal parametric convergence rate is achieved. We further show that learning from complementary labels can be easily combined with learning from ordinary labels (i.e., ordinary supervised learning), providing a highly practical implementation of the proposed method. Finally, we experimentally demonstrate the usefulness of the proposed methods.
Tasks
Published	2017-05-22
URL	http://arxiv.org/abs/1705.07541v2
PDF	http://arxiv.org/pdf/1705.07541v2.pdf
PWC	https://paperswithcode.com/paper/learning-from-complementary-labels
Repo	https://github.com/takashiishida/comp
Framework	pytorch

Aspect-Based Relational Sentiment Analysis Using a Stacked Neural Network Architecture


Title	Aspect-Based Relational Sentiment Analysis Using a Stacked Neural Network Architecture
Authors	Soufian Jebbara, Philipp Cimiano
Abstract	Sentiment analysis can be regarded as a relation extraction problem in which the sentiment of some opinion holder towards a certain aspect of a product, theme or event needs to be extracted. We present a novel neural architecture for sentiment analysis as a relation extraction problem that addresses this problem by dividing it into three subtasks: i) identification of aspect and opinion terms, ii) labeling of opinion terms with a sentiment, and iii) extraction of relations between opinion terms and aspect terms. For each subtask, we propose a neural network based component and combine all of them into a complete system for relational sentiment analysis. The component for aspect and opinion term extraction is a hybrid architecture consisting of a recurrent neural network stacked on top of a convolutional neural network. This approach outperforms a standard convolutional deep neural architecture as well as a recurrent network architecture and performs competitively compared to other methods on two datasets of annotated customer reviews. To extract sentiments for individual opinion terms, we propose a recurrent architecture in combination with word distance features and achieve promising results, outperforming a majority baseline by 18% accuracy and providing the first results for the USAGE dataset. Our relation extraction component outperforms the current state-of-the-art in aspect-opinion relation extraction by 15% F-Measure.
Tasks	Relation Extraction, Sentiment Analysis
Published	2017-09-19
URL	http://arxiv.org/abs/1709.06309v1
PDF	http://arxiv.org/pdf/1709.06309v1.pdf
PWC	https://paperswithcode.com/paper/aspect-based-relational-sentiment-analysis
Repo	https://github.com/santhoshmani888/Aspect-Based-sentiment-analysis
Framework	none

Mirror Descent Search and its Acceleration


Title	Mirror Descent Search and its Acceleration
Authors	Megumi Miyashita, Shiro Yano, Toshiyuki Kondo
Abstract	In recent years, attention has been focused on the relationship between black-box optimiza- tion problem and reinforcement learning problem. In this research, we propose the Mirror Descent Search (MDS) algorithm which is applicable both for black box optimization prob- lems and reinforcement learning problems. Our method is based on the mirror descent method, which is a general optimization algorithm. The contribution of this research is roughly twofold. We propose two essential algorithms, called MDS and Accelerated Mirror Descent Search (AMDS), and two more approximate algorithms: Gaussian Mirror Descent Search (G-MDS) and Gaussian Accelerated Mirror Descent Search (G-AMDS). This re- search shows that the advanced methods developed in the context of the mirror descent research can be applied to reinforcement learning problem. We also clarify the relationship between an existing reinforcement learning algorithm and our method. With two evaluation experiments, we show our proposed algorithms converge faster than some state-of-the-art methods.
Tasks
Published	2017-09-08
URL	http://arxiv.org/abs/1709.02535v2
PDF	http://arxiv.org/pdf/1709.02535v2.pdf
PWC	https://paperswithcode.com/paper/mirror-descent-search-and-its-acceleration
Repo	https://github.com/mmilk1231/MirrorDescentSearch
Framework	none

Learned Primal-dual Reconstruction


Title	Learned Primal-dual Reconstruction
Authors	Jonas Adler, Ozan Öktem
Abstract	We propose the Learned Primal-Dual algorithm for tomographic reconstruction. The algorithm accounts for a (possibly non-linear) forward operator in a deep neural network by unrolling a proximal primal-dual optimization method, but where the proximal operators have been replaced with convolutional neural networks. The algorithm is trained end-to-end, working directly from raw measured data and it does not depend on any initial reconstruction such as FBP. We compare performance of the proposed method on low dose CT reconstruction against FBP, TV, and deep learning based post-processing of FBP. For the Shepp-Logan phantom we obtain >6dB PSNR improvement against all compared methods. For human phantoms the corresponding improvement is 6.6dB over TV and 2.2dB over learned post-processing along with a substantial improvement in the SSIM. Finally, our algorithm involves only ten forward-back-projection computations, making the method feasible for time critical clinical applications.
Tasks
Published	2017-07-20
URL	http://arxiv.org/abs/1707.06474v3
PDF	http://arxiv.org/pdf/1707.06474v3.pdf
PWC	https://paperswithcode.com/paper/learned-primal-dual-reconstruction
Repo	https://github.com/adler-j/learned_primal_dual
Framework	tf

pix2code: Generating Code from a Graphical User Interface Screenshot


Title	pix2code: Generating Code from a Graphical User Interface Screenshot
Authors	Tony Beltramelli
Abstract	Transforming a graphical user interface screenshot created by a designer into computer code is a typical task conducted by a developer in order to build customized software, websites, and mobile applications. In this paper, we show that deep learning methods can be leveraged to train a model end-to-end to automatically generate code from a single input image with over 77% of accuracy for three different platforms (i.e. iOS, Android and web-based technologies).
Tasks	Code Generation
Published	2017-05-22
URL	http://arxiv.org/abs/1705.07962v2
PDF	http://arxiv.org/pdf/1705.07962v2.pdf
PWC	https://paperswithcode.com/paper/pix2code-generating-code-from-a-graphical
Repo	https://github.com/RAHUL-KESHERVANI/my_pix2code
Framework	none

Gradual Learning of Recurrent Neural Networks


Title	Gradual Learning of Recurrent Neural Networks
Authors	Ziv Aharoni, Gal Rattner, Haim Permuter
Abstract	Recurrent Neural Networks (RNNs) achieve state-of-the-art results in many sequence-to-sequence modeling tasks. However, RNNs are difficult to train and tend to suffer from overfitting. Motivated by the Data Processing Inequality (DPI), we formulate the multi-layered network as a Markov chain, introducing a training method that comprises training the network gradually and using layer-wise gradient clipping. We found that applying our methods, combined with previously introduced regularization and optimization methods, resulted in improvements in state-of-the-art architectures operating in language modeling tasks.
Tasks	Language Modelling
Published	2017-08-29
URL	http://arxiv.org/abs/1708.08863v2
PDF	http://arxiv.org/pdf/1708.08863v2.pdf
PWC	https://paperswithcode.com/paper/gradual-learning-of-recurrent-neural-networks
Repo	https://github.com/zivaharoni/gradual-learning-rnn
Framework	pytorch

Model-Based Planning with Discrete and Continuous Actions


Title	Model-Based Planning with Discrete and Continuous Actions
Authors	Mikael Henaff, William F. Whitney, Yann LeCun
Abstract	Action planning using learned and differentiable forward models of the world is a general approach which has a number of desirable properties, including improved sample complexity over model-free RL methods, reuse of learned models across different tasks, and the ability to perform efficient gradient-based optimization in continuous action spaces. However, this approach does not apply straightforwardly when the action space is discrete. In this work, we show that it is in fact possible to effectively perform planning via backprop in discrete action spaces, using a simple paramaterization of the actions vectors on the simplex combined with input noise when training the forward model. Our experiments show that this approach can match or outperform model-free RL and discrete planning methods on gridworld navigation tasks in terms of performance and/or planning time while using limited environment interactions, and can additionally be used to perform model-based control in a challenging new task where the action space combines discrete and continuous actions. We furthermore propose a policy distillation approach which yields a fast policy network which can be used at inference time, removing the need for an iterative planning procedure.
Tasks
Published	2017-05-19
URL	http://arxiv.org/abs/1705.07177v2
PDF	http://arxiv.org/pdf/1705.07177v2.pdf
PWC	https://paperswithcode.com/paper/model-based-planning-with-discrete-and
Repo	https://github.com/jackdawe/joliRL
Framework	torch

Deep Class Aware Denoising


Title	Deep Class Aware Denoising
Authors	Tal Remez, Or Litany, Raja Giryes, Alex M. Bronstein
Abstract	The increasing demand for high image quality in mobile devices brings forth the need for better computational enhancement techniques, and image denoising in particular. At the same time, the images captured by these devices can be categorized into a small set of semantic classes. However simple, this observation has not been exploited in image denoising until now. In this paper, we demonstrate how the reconstruction quality improves when a denoiser is aware of the type of content in the image. To this end, we first propose a new fully convolutional deep neural network architecture which is simple yet powerful as it achieves state-of-the-art performance even without being class-aware. We further show that a significant boost in performance of up to $0.4$ dB PSNR can be achieved by making our network class-aware, namely, by fine-tuning it for images belonging to a specific semantic class. Relying on the hugely successful existing image classifiers, this research advocates for using a class-aware approach in all image enhancement tasks.
Tasks	Denoising, Image Denoising, Image Enhancement
Published	2017-01-06
URL	http://arxiv.org/abs/1701.01698v2
PDF	http://arxiv.org/pdf/1701.01698v2.pdf
PWC	https://paperswithcode.com/paper/deep-class-aware-denoising
Repo	https://github.com/TalRemez/deep_class_aware_denoising
Framework	tf

NiftyNet: a deep-learning platform for medical imaging


Title	NiftyNet: a deep-learning platform for medical imaging
Authors	Eli Gibson, Wenqi Li, Carole Sudre, Lucas Fidon, Dzhoshkun I. Shakir, Guotai Wang, Zach Eaton-Rosen, Robert Gray, Tom Doel, Yipeng Hu, Tom Whyntie, Parashkev Nachev, Marc Modat, Dean C. Barratt, Sébastien Ourselin, M. Jorge Cardoso, Tom Vercauteren
Abstract	Medical image analysis and computer-assisted intervention problems are increasingly being addressed with deep-learning-based solutions. Established deep-learning platforms are flexible but do not provide specific functionality for medical image analysis and adapting them for this application requires substantial implementation effort. Thus, there has been substantial duplication of effort and incompatible infrastructure developed across many research groups. This work presents the open-source NiftyNet platform for deep learning in medical imaging. The ambition of NiftyNet is to accelerate and simplify the development of these solutions, and to provide a common mechanism for disseminating research outputs for the community to use, adapt and build upon. NiftyNet provides a modular deep-learning pipeline for a range of medical imaging applications including segmentation, regression, image generation and representation learning applications. Components of the NiftyNet pipeline including data loading, data augmentation, network architectures, loss functions and evaluation metrics are tailored to, and take advantage of, the idiosyncracies of medical image analysis and computer-assisted intervention. NiftyNet is built on TensorFlow and supports TensorBoard visualization of 2D and 3D images and computational graphs by default. We present 3 illustrative medical image analysis applications built using NiftyNet: (1) segmentation of multiple abdominal organs from computed tomography; (2) image regression to predict computed tomography attenuation maps from brain magnetic resonance images; and (3) generation of simulated ultrasound images for specified anatomical poses. NiftyNet enables researchers to rapidly develop and distribute deep learning solutions for segmentation, regression, image generation and representation learning applications, or extend the platform to new applications.
Tasks	Data Augmentation, Image Generation, Medical Image Generation, Representation Learning
Published	2017-09-11
URL	http://arxiv.org/abs/1709.03485v2
PDF	http://arxiv.org/pdf/1709.03485v2.pdf
PWC	https://paperswithcode.com/paper/niftynet-a-deep-learning-platform-for-medical
Repo	https://github.com/charan223/Brain-Tumor-Segmentation-using-Topological-Loss
Framework	tf