July 27, 2019

2747 words 13 mins read

Paper Group ANR 576

DeepProbe: Information Directed Sequence Understanding and Chatbot Design via Recurrent Neural Networks. Going Wider: Recurrent Neural Network With Parallel Cells. Generative Models of Visually Grounded Imagination. Distributed SAGA: Maintaining linear convergence rate with limited communication. Scalable and Compact 3D Action Recognition with Appr …

DeepProbe: Information Directed Sequence Understanding and Chatbot Design via Recurrent Neural Networks


Title	DeepProbe: Information Directed Sequence Understanding and Chatbot Design via Recurrent Neural Networks
Authors	Zi Yin, Keng-hao Chang, Ruofei Zhang
Abstract	Information extraction and user intention identification are central topics in modern query understanding and recommendation systems. In this paper, we propose DeepProbe, a generic information-directed interaction framework which is built around an attention-based sequence to sequence (seq2seq) recurrent neural network. DeepProbe can rephrase, evaluate, and even actively ask questions, leveraging the generative ability and likelihood estimation made possible by seq2seq models. DeepProbe makes decisions based on a derived uncertainty (entropy) measure conditioned on user inputs, possibly with multiple rounds of interactions. Three applications, namely a rewritter, a relevance scorer and a chatbot for ad recommendation, were built around DeepProbe, with the first two serving as precursory building blocks for the third. We first use the seq2seq model in DeepProbe to rewrite a user query into one of standard query form, which is submitted to an ordinary recommendation system. Secondly, we evaluate DeepProbe’s seq2seq model-based relevance scoring. Finally, we build a chatbot prototype capable of making active user interactions, which can ask questions that maximize information gain, allowing for a more efficient user intention idenfication process. We evaluate first two applications by 1) comparing with baselines by BLEU and AUC, and 2) human judge evaluation. Both demonstrate significant improvements compared with current state-of-the-art systems, proving their values as useful tools on their own, and at the same time laying a good foundation for the ongoing chatbot application.
Tasks	Chatbot, Recommendation Systems
Published	2017-07-18
URL	http://arxiv.org/abs/1707.05470v2
PDF	http://arxiv.org/pdf/1707.05470v2.pdf
PWC	https://paperswithcode.com/paper/deepprobe-information-directed-sequence
Repo
Framework

Going Wider: Recurrent Neural Network With Parallel Cells


Title	Going Wider: Recurrent Neural Network With Parallel Cells
Authors	Danhao Zhu, Si Shen, Xin-Yu Dai, Jiajun Chen
Abstract	Recurrent Neural Network (RNN) has been widely applied for sequence modeling. In RNN, the hidden states at current step are full connected to those at previous step, thus the influence from less related features at previous step may potentially decrease model’s learning ability. We propose a simple technique called parallel cells (PCs) to enhance the learning ability of Recurrent Neural Network (RNN). In each layer, we run multiple small RNN cells rather than one single large cell. In this paper, we evaluate PCs on 2 tasks. On language modeling task on PTB (Penn Tree Bank), our model outperforms state of art models by decreasing perplexity from 78.6 to 75.3. On Chinese-English translation task, our model increases BLEU score for 0.39 points than baseline model.
Tasks	Language Modelling
Published	2017-05-03
URL	http://arxiv.org/abs/1705.01346v1
PDF	http://arxiv.org/pdf/1705.01346v1.pdf
PWC	https://paperswithcode.com/paper/going-wider-recurrent-neural-network-with
Repo
Framework

Generative Models of Visually Grounded Imagination


Title	Generative Models of Visually Grounded Imagination
Authors	Ramakrishna Vedantam, Ian Fischer, Jonathan Huang, Kevin Murphy
Abstract	It is easy for people to imagine what a man with pink hair looks like, even if they have never seen such a person before. We call the ability to create images of novel semantic concepts visually grounded imagination. In this paper, we show how we can modify variational auto-encoders to perform this task. Our method uses a novel training objective, and a novel product-of-experts inference network, which can handle partially specified (abstract) concepts in a principled and efficient way. We also propose a set of easy-to-compute evaluation metrics that capture our intuitive notions of what it means to have good visual imagination, namely correctness, coverage, and compositionality (the 3 C’s). Finally, we perform a detailed comparison of our method with two existing joint image-attribute VAE methods (the JMVAE method of Suzuki et.al. and the BiVCCA method of Wang et.al.) by applying them to two datasets: the MNIST-with-attributes dataset (which we introduce here), and the CelebA dataset.
Tasks
Published	2017-05-30
URL	http://arxiv.org/abs/1705.10762v8
PDF	http://arxiv.org/pdf/1705.10762v8.pdf
PWC	https://paperswithcode.com/paper/generative-models-of-visually-grounded
Repo
Framework

Distributed SAGA: Maintaining linear convergence rate with limited communication


Title	Distributed SAGA: Maintaining linear convergence rate with limited communication
Authors	Clément Calauzènes, Nicolas Le Roux
Abstract	In recent years, variance-reducing stochastic methods have shown great practical performance, exhibiting linear convergence rate when other stochastic methods offered a sub-linear rate. However, as datasets grow ever bigger and clusters become widespread, the need for fast distribution methods is pressing. We propose here a distribution scheme for SAGA which maintains a linear convergence rate, even when communication between nodes is limited.
Tasks
Published	2017-05-29
URL	http://arxiv.org/abs/1705.10405v1
PDF	http://arxiv.org/pdf/1705.10405v1.pdf
PWC	https://paperswithcode.com/paper/distributed-saga-maintaining-linear
Repo
Framework

Scalable and Compact 3D Action Recognition with Approximated RBF Kernel Machines


Title	Scalable and Compact 3D Action Recognition with Approximated RBF Kernel Machines
Authors	Jacopo Cavazza, Pietro Morerio, Vittorio Murino
Abstract	Despite the recent deep learning (DL) revolution, kernel machines still remain powerful methods for action recognition. DL has brought the use of large datasets and this is typically a problem for kernel approaches, which are not scaling up efficiently due to kernel Gram matrices. Nevertheless, kernel methods are still attractive and more generally applicable since they can equally manage different sizes of the datasets, also in cases where DL techniques show some limitations. This work investigates these issues by proposing an explicit approximated representation that, together with a linear model, is an equivalent, yet scalable, implementation of a kernel machine. Our approximation is directly inspired by the exact feature map that is induced by an RBF Gaussian kernel but, unlike the latter, it is finite dimensional and very compact. We justify the soundness of our idea with a theoretical analysis which proves the unbiasedness of the approximation, and provides a vanishing bound for its variance, which is shown to decrease much rapidly than in alternative methods in the literature. In a broad experimental validation, we assess the superiority of our approximation in terms of 1) ease and speed of training, 2) compactness of the model, and 3) improvements with respect to the state-of-the-art performance.
Tasks	3D Human Action Recognition, Temporal Action Localization
Published	2017-11-28
URL	http://arxiv.org/abs/1711.10290v1
PDF	http://arxiv.org/pdf/1711.10290v1.pdf
PWC	https://paperswithcode.com/paper/scalable-and-compact-3d-action-recognition
Repo
Framework

Predicting Cardiovascular Risk Factors from Retinal Fundus Photographs using Deep Learning


Title	Predicting Cardiovascular Risk Factors from Retinal Fundus Photographs using Deep Learning
Authors	Ryan Poplin, Avinash V. Varadarajan, Katy Blumer, Yun Liu, Michael V. McConnell, Greg S. Corrado, Lily Peng, Dale R. Webster
Abstract	Traditionally, medical discoveries are made by observing associations and then designing experiments to test these hypotheses. However, observing and quantifying associations in images can be difficult because of the wide variety of features, patterns, colors, values, shapes in real data. In this paper, we use deep learning, a machine learning technique that learns its own features, to discover new knowledge from retinal fundus images. Using models trained on data from 284,335 patients, and validated on two independent datasets of 12,026 and 999 patients, we predict cardiovascular risk factors not previously thought to be present or quantifiable in retinal images, such as such as age (within 3.26 years), gender (0.97 AUC), smoking status (0.71 AUC), HbA1c (within 1.39%), systolic blood pressure (within 11.23mmHg) as well as major adverse cardiac events (0.70 AUC). We further show that our models used distinct aspects of the anatomy to generate each prediction, such as the optic disc or blood vessels, opening avenues of further research.
Tasks
Published	2017-08-31
URL	http://arxiv.org/abs/1708.09843v2
PDF	http://arxiv.org/pdf/1708.09843v2.pdf
PWC	https://paperswithcode.com/paper/predicting-cardiovascular-risk-factors-from
Repo
Framework

Learning Task Specifications from Demonstrations


Title	Learning Task Specifications from Demonstrations
Authors	Marcell Vazquez-Chanlatte, Susmit Jha, Ashish Tiwari, Mark K. Ho, Sanjit A. Seshia
Abstract	Real world applications often naturally decompose into several sub-tasks. In many settings (e.g., robotics) demonstrations provide a natural way to specify the sub-tasks. However, most methods for learning from demonstrations either do not provide guarantees that the artifacts learned for the sub-tasks can be safely recombined or limit the types of composition available. Motivated by this deficit, we consider the problem of inferring Boolean non-Markovian rewards (also known as logical trace properties or specifications) from demonstrations provided by an agent operating in an uncertain, stochastic environment. Crucially, specifications admit well-defined composition rules that are typically easy to interpret. In this paper, we formulate the specification inference task as a maximum a posteriori (MAP) probability inference problem, apply the principle of maximum entropy to derive an analytic demonstration likelihood model and give an efficient approach to search for the most likely specification in a large candidate pool of specifications. In our experiments, we demonstrate how learning specifications can help avoid common problems that often arise due to ad-hoc reward composition.
Tasks
Published	2017-10-11
URL	http://arxiv.org/abs/1710.03875v5
PDF	http://arxiv.org/pdf/1710.03875v5.pdf
PWC	https://paperswithcode.com/paper/learning-task-specifications-from
Repo
Framework

Train, Diagnose and Fix: Interpretable Approach for Fine-grained Action Recognition


Title	Train, Diagnose and Fix: Interpretable Approach for Fine-grained Action Recognition
Authors	Jingxuan Hou, Tae Soo Kim, Austin Reiter
Abstract	Despite the growing discriminative capabilities of modern deep learning methods for recognition tasks, the inner workings of the state-of-art models still remain mostly black-boxes. In this paper, we propose a systematic interpretation of model parameters and hidden representations of Residual Temporal Convolutional Networks (Res-TCN) for action recognition in time-series data. We also propose a Feature Map Decoder as part of the interpretation analysis, which outputs a representation of model’s hidden variables in the same domain as the input. Such analysis empowers us to expose model’s characteristic learning patterns in an interpretable way. For example, through the diagnosis analysis, we discovered that our model has learned to achieve view-point invariance by implicitly learning to perform rotational normalization of the input to a more discriminative view. Based on the findings from the model interpretation analysis, we propose a targeted refinement technique, which can generalize to various other recognition models. The proposed work introduces a three-stage paradigm for model learning: training, interpretable diagnosis and targeted refinement. We validate our approach on skeleton based 3D human action recognition benchmark of NTU RGB+D. We show that the proposed workflow is an effective model learning strategy and the resulting Multi-stream Residual Temporal Convolutional Network (MS-Res-TCN) achieves the state-of-the-art performance on NTU RGB+D.
Tasks	3D Human Action Recognition, Temporal Action Localization, Time Series
Published	2017-11-22
URL	http://arxiv.org/abs/1711.08502v1
PDF	http://arxiv.org/pdf/1711.08502v1.pdf
PWC	https://paperswithcode.com/paper/train-diagnose-and-fix-interpretable-approach
Repo
Framework

Paradigm Completion for Derivational Morphology


Title	Paradigm Completion for Derivational Morphology
Authors	Ryan Cotterell, Ekaterina Vylomova, Huda Khayrallah, Christo Kirov, David Yarowsky
Abstract	The generation of complex derived word forms has been an overlooked problem in NLP; we fill this gap by applying neural sequence-to-sequence models to the task. We overview the theoretical motivation for a paradigmatic treatment of derivational morphology, and introduce the task of derivational paradigm completion as a parallel to inflectional paradigm completion. State-of-the-art neural models, adapted from the inflection task, are able to learn a range of derivation patterns, and outperform a non-neural baseline by 16.4%. However, due to semantic, historical, and lexical considerations involved in derivational morphology, future work will be needed to achieve performance parity with inflection-generating systems.
Tasks
Published	2017-08-30
URL	http://arxiv.org/abs/1708.09151v1
PDF	http://arxiv.org/pdf/1708.09151v1.pdf
PWC	https://paperswithcode.com/paper/paradigm-completion-for-derivational
Repo
Framework

A Compact Kernel Approximation for 3D Action Recognition


Title	A Compact Kernel Approximation for 3D Action Recognition
Authors	Jacopo Cavazza, Pietro Morerio, Vittorio Murino
Abstract	3D action recognition was shown to benefit from a covariance representation of the input data (joint 3D positions). A kernel machine feed with such feature is an effective paradigm for 3D action recognition, yielding state-of-the-art results. Yet, the whole framework is affected by the well-known scalability issue. In fact, in general, the kernel function has to be evaluated for all pairs of instances inducing a Gram matrix whose complexity is quadratic in the number of samples. In this work we reduce such complexity to be linear by proposing a novel and explicit feature map to approximate the kernel function. This allows to train a linear classifier with an explicit feature encoding, which implicitly implements a Log-Euclidean machine in a scalable fashion. Not only we prove that the proposed approximation is unbiased, but also we work out an explicit strong bound for its variance, attesting a theoretical superiority of our approach with respect to existing ones. Experimentally, we verify that our representation provides a compact encoding and outperforms other approximation schemes on a number of publicly available benchmark datasets for 3D action recognition.
Tasks	3D Human Action Recognition, Temporal Action Localization
Published	2017-09-06
URL	http://arxiv.org/abs/1709.01695v2
PDF	http://arxiv.org/pdf/1709.01695v2.pdf
PWC	https://paperswithcode.com/paper/a-compact-kernel-approximation-for-3d-action
Repo
Framework

Learning the Kernel for Classification and Regression


Title	Learning the Kernel for Classification and Regression
Authors	Chen Li, Luca Venturi, Ruitu Xu
Abstract	We investigate a series of learning kernel problems with polynomial combinations of base kernels, which will help us solve regression and classification problems. We also perform some numerical experiments of polynomial kernels with regression and classification tasks on different datasets.
Tasks
Published	2017-12-22
URL	http://arxiv.org/abs/1712.08597v2
PDF	http://arxiv.org/pdf/1712.08597v2.pdf
PWC	https://paperswithcode.com/paper/learning-the-kernel-for-classification-and
Repo
Framework

Guarantees for Greedy Maximization of Non-submodular Functions with Applications


Title	Guarantees for Greedy Maximization of Non-submodular Functions with Applications
Authors	Andrew An Bian, Joachim M. Buhmann, Andreas Krause, Sebastian Tschiatschek
Abstract	We investigate the performance of the standard Greedy algorithm for cardinality constrained maximization of non-submodular nondecreasing set functions. While there are strong theoretical guarantees on the performance of Greedy for maximizing submodular functions, there are few guarantees for non-submodular ones. However, Greedy enjoys strong empirical performance for many important non-submodular functions, e.g., the Bayesian A-optimality objective in experimental design. We prove theoretical guarantees supporting the empirical performance. Our guarantees are characterized by a combination of the (generalized) curvature $\alpha$ and the submodularity ratio $\gamma$. In particular, we prove that Greedy enjoys a tight approximation guarantee of $\frac{1}{\alpha}(1- e^{-\gamma\alpha})$ for cardinality constrained maximization. In addition, we bound the submodularity ratio and curvature for several important real-world objectives, including the Bayesian A-optimality objective, the determinantal function of a square submatrix and certain linear programs with combinatorial constraints. We experimentally validate our theoretical findings for both synthetic and real-world applications.
Tasks
Published	2017-03-06
URL	https://arxiv.org/abs/1703.02100v4
PDF	https://arxiv.org/pdf/1703.02100v4.pdf
PWC	https://paperswithcode.com/paper/guarantees-for-greedy-maximization-of-non
Repo
Framework

TheoSea: Marching Theory to Light


Title	TheoSea: Marching Theory to Light
Authors	Mark A. Stalzer, Chao Ju
Abstract	There is sufficient information in the far-field of a radiating dipole antenna to rediscover the Maxwell Equations and the wave equations of light, including the speed of light $c.$ TheoSea is a Julia program that does this in about a second, and the key insight is that the compactness of theories drives the search. The program is a computational embodiment of the scientific method: observation, consideration of candidate theories, and validation.
Tasks
Published	2017-08-14
URL	http://arxiv.org/abs/1708.04927v1
PDF	http://arxiv.org/pdf/1708.04927v1.pdf
PWC	https://paperswithcode.com/paper/theosea-marching-theory-to-light
Repo
Framework

Robust Deep Reinforcement Learning with Adversarial Attacks


Title	Robust Deep Reinforcement Learning with Adversarial Attacks
Authors	Anay Pattanaik, Zhenyi Tang, Shuijing Liu, Gautham Bommannan, Girish Chowdhary
Abstract	This paper proposes adversarial attacks for Reinforcement Learning (RL) and then improves the robustness of Deep Reinforcement Learning algorithms (DRL) to parameter uncertainties with the help of these attacks. We show that even a naively engineered attack successfully degrades the performance of DRL algorithm. We further improve the attack using gradient information of an engineered loss function which leads to further degradation in performance. These attacks are then leveraged during training to improve the robustness of RL within robust control framework. We show that this adversarial training of DRL algorithms like Deep Double Q learning and Deep Deterministic Policy Gradients leads to significant increase in robustness to parameter variations for RL benchmarks such as Cart-pole, Mountain Car, Hopper and Half Cheetah environment.
Tasks	Q-Learning
Published	2017-12-11
URL	http://arxiv.org/abs/1712.03632v1
PDF	http://arxiv.org/pdf/1712.03632v1.pdf
PWC	https://paperswithcode.com/paper/robust-deep-reinforcement-learning-with
Repo
Framework

Deep Sampling Networks


Title	Deep Sampling Networks
Authors	Bolun Cai, Xiangmin Xu, Kailing Guo, Kui Jia, Dacheng Tao
Abstract	Deep convolutional neural networks achieve excellent image up-sampling performance. However, CNN-based methods tend to restore high-resolution results highly depending on traditional interpolations (e.g. bicubic). In this paper, we present a deep sampling network (DSN) for down-sampling and up-sampling without any cheap interpolation. First, the down-sampling subnetwork is trained without supervision, thereby preserving more information and producing better visual effects in the low-resolution image. Second, the up-sampling subnetwork learns a sub-pixel residual with dense connections to accelerate convergence and improve performance. DSN’s down-sampling subnetwork can be used to generate photo-realistic low-resolution images and replace traditional down-sampling method in image processing. With the powerful down-sampling process, the co-training DSN set a new state-of-the-art performance for image super-resolution. Moreover, DSN is compatible with existing image codecs to improve image compression.
Tasks	Image Compression, Image Super-Resolution, Super-Resolution
Published	2017-12-04
URL	http://arxiv.org/abs/1712.00926v2
PDF	http://arxiv.org/pdf/1712.00926v2.pdf
PWC	https://paperswithcode.com/paper/deep-sampling-networks
Repo
Framework