July 26, 2019

3014 words 15 mins read

Paper Group ANR 763

Budgeted Experiment Design for Causal Structure Learning. Neural Network Based Reinforcement Learning for Audio-Visual Gaze Control in Human-Robot Interaction. Deep Learning-Based Communication Over the Air. A GRU-Gated Attention Model for Neural Machine Translation. BLADE: Filter Learning for General Purpose Computational Photography. Adversarial …

Budgeted Experiment Design for Causal Structure Learning


Title	Budgeted Experiment Design for Causal Structure Learning
Authors	AmirEmad Ghassami, Saber Salehkaleybar, Negar Kiyavash, Elias Bareinboim
Abstract	We study the problem of causal structure learning when the experimenter is limited to perform at most $k$ non-adaptive experiments of size $1$. We formulate the problem of finding the best intervention target set as an optimization problem, which aims to maximize the average number of edges whose directions are resolved. We prove that the corresponding objective function is submodular and a greedy algorithm suffices to achieve $(1-\frac{1}{e})$-approximation of the optimal value. We further present an accelerated variant of the greedy algorithm, which can lead to orders of magnitude performance speedup. We validate our proposed approach on synthetic and real graphs. The results show that compared to the purely observational setting, our algorithm orients the majority of the edges through a considerably small number of interventions.
Tasks
Published	2017-09-11
URL	http://arxiv.org/abs/1709.03625v2
PDF	http://arxiv.org/pdf/1709.03625v2.pdf
PWC	https://paperswithcode.com/paper/budgeted-experiment-design-for-causal
Repo
Framework

Neural Network Based Reinforcement Learning for Audio-Visual Gaze Control in Human-Robot Interaction


Title	Neural Network Based Reinforcement Learning for Audio-Visual Gaze Control in Human-Robot Interaction
Authors	Stéphane Lathuilière, Benoit Massé, Pablo Mesejo, Radu Horaud
Abstract	This paper introduces a novel neural network-based reinforcement learning approach for robot gaze control. Our approach enables a robot to learn and to adapt its gaze control strategy for human-robot interaction neither with the use of external sensors nor with human supervision. The robot learns to focus its attention onto groups of people from its own audio-visual experiences, independently of the number of people, of their positions and of their physical appearances. In particular, we use a recurrent neural network architecture in combination with Q-learning to find an optimal action-selection policy; we pre-train the network using a simulated environment that mimics realistic scenarios that involve speaking/silent participants, thus avoiding the need of tedious sessions of a robot interacting with people. Our experimental evaluation suggests that the proposed method is robust against parameter estimation, i.e. the parameter values yielded by the method do not have a decisive impact on the performance. The best results are obtained when both audio and visual information is jointly used. Experiments with the Nao robot indicate that our framework is a step forward towards the autonomous learning of socially acceptable gaze behavior.
Tasks	Q-Learning
Published	2017-11-18
URL	http://arxiv.org/abs/1711.06834v2
PDF	http://arxiv.org/pdf/1711.06834v2.pdf
PWC	https://paperswithcode.com/paper/neural-network-based-reinforcement-learning
Repo
Framework

Deep Learning-Based Communication Over the Air


Title	Deep Learning-Based Communication Over the Air
Authors	Sebastian Dörner, Sebastian Cammerer, Jakob Hoydis, Stephan ten Brink
Abstract	End-to-end learning of communications systems is a fascinating novel concept that has so far only been validated by simulations for block-based transmissions. It allows learning of transmitter and receiver implementations as deep neural networks (NNs) that are optimized for an arbitrary differentiable end-to-end performance metric, e.g., block error rate (BLER). In this paper, we demonstrate that over-the-air transmissions are possible: We build, train, and run a complete communications system solely composed of NNs using unsynchronized off-the-shelf software-defined radios (SDRs) and open-source deep learning (DL) software libraries. We extend the existing ideas towards continuous data transmission which eases their current restriction to short block lengths but also entails the issue of receiver synchronization. We overcome this problem by introducing a frame synchronization module based on another NN. A comparison of the BLER performance of the “learned” system with that of a practical baseline shows competitive performance close to 1 dB, even without extensive hyperparameter tuning. We identify several practical challenges of training such a system over actual channels, in particular the missing channel gradient, and propose a two-step learning procedure based on the idea of transfer learning that circumvents this issue.
Tasks	Transfer Learning
Published	2017-07-11
URL	http://arxiv.org/abs/1707.03384v1
PDF	http://arxiv.org/pdf/1707.03384v1.pdf
PWC	https://paperswithcode.com/paper/deep-learning-based-communication-over-the
Repo
Framework

A GRU-Gated Attention Model for Neural Machine Translation


Title	A GRU-Gated Attention Model for Neural Machine Translation
Authors	Biao Zhang, Deyi Xiong, Jinsong Su
Abstract	Neural machine translation (NMT) heavily relies on an attention network to produce a context vector for each target word prediction. In practice, we find that context vectors for different target words are quite similar to one another and therefore are insufficient in discriminatively predicting target words. The reason for this might be that context vectors produced by the vanilla attention network are just a weighted sum of source representations that are invariant to decoder states. In this paper, we propose a novel GRU-gated attention model (GAtt) for NMT which enhances the degree of discrimination of context vectors by enabling source representations to be sensitive to the partial translation generated by the decoder. GAtt uses a gated recurrent unit (GRU) to combine two types of information: treating a source annotation vector originally produced by the bidirectional encoder as the history state while the corresponding previous decoder state as the input to the GRU. The GRU-combined information forms a new source annotation vector. In this way, we can obtain translation-sensitive source representations which are then feed into the attention network to generate discriminative context vectors. We further propose a variant that regards a source annotation vector as the current input while the previous decoder state as the history. Experiments on NIST Chinese-English translation tasks show that both GAtt-based models achieve significant improvements over the vanilla attentionbased NMT. Further analyses on attention weights and context vectors demonstrate the effectiveness of GAtt in improving the discrimination power of representations and handling the challenging issue of over-translation.
Tasks	Machine Translation
Published	2017-04-27
URL	https://arxiv.org/abs/1704.08430v2
PDF	https://arxiv.org/pdf/1704.08430v2.pdf
PWC	https://paperswithcode.com/paper/a-gru-gated-attention-model-for-neural
Repo
Framework

BLADE: Filter Learning for General Purpose Computational Photography


Title	BLADE: Filter Learning for General Purpose Computational Photography
Authors	Pascal Getreuer, Ignacio Garcia-Dorado, John Isidoro, Sungjoon Choi, Frank Ong, Peyman Milanfar
Abstract	The Rapid and Accurate Image Super Resolution (RAISR) method of Romano, Isidoro, and Milanfar is a computationally efficient image upscaling method using a trained set of filters. We describe a generalization of RAISR, which we name Best Linear Adaptive Enhancement (BLADE). This approach is a trainable edge-adaptive filtering framework that is general, simple, computationally efficient, and useful for a wide range of problems in computational photography. We show applications to operations which may appear in a camera pipeline including denoising, demosaicing, and stylization.
Tasks	Demosaicking, Denoising, Image Super-Resolution, Super-Resolution
Published	2017-11-29
URL	http://arxiv.org/abs/1711.10700v2
PDF	http://arxiv.org/pdf/1711.10700v2.pdf
PWC	https://paperswithcode.com/paper/blade-filter-learning-for-general-purpose
Repo
Framework

Adversarial Advantage Actor-Critic Model for Task-Completion Dialogue Policy Learning


Title	Adversarial Advantage Actor-Critic Model for Task-Completion Dialogue Policy Learning
Authors	Baolin Peng, Xiujun Li, Jianfeng Gao, Jingjing Liu, Yun-Nung Chen, Kam-Fai Wong
Abstract	This paper presents a new method — adversarial advantage actor-critic (Adversarial A2C), which significantly improves the efficiency of dialogue policy learning in task-completion dialogue systems. Inspired by generative adversarial networks (GAN), we train a discriminator to differentiate responses/actions generated by dialogue agents from responses/actions by experts. Then, we incorporate the discriminator as another critic into the advantage actor-critic (A2C) framework, to encourage the dialogue agent to explore state-action within the regions where the agent takes actions similar to those of the experts. Experimental results in a movie-ticket booking domain show that the proposed Adversarial A2C can accelerate policy exploration efficiently.
Tasks	Task-Completion Dialogue Policy Learning
Published	2017-10-31
URL	http://arxiv.org/abs/1710.11277v2
PDF	http://arxiv.org/pdf/1710.11277v2.pdf
PWC	https://paperswithcode.com/paper/adversarial-advantage-actor-critic-model-for
Repo
Framework

Compressed Sensing, ASBSR-method of image sampling and reconstruction and the problem of digital image acquisition with the lowest possible sampling rate


Title	Compressed Sensing, ASBSR-method of image sampling and reconstruction and the problem of digital image acquisition with the lowest possible sampling rate
Authors	Leonid P. Yaroslavsky
Abstract	The problem of minimization of the number of measurements needed for digital image acquisition and reconstruction with a given accuracy is addressed. Basics of the sampling theory are outlined to show that the lower bound of signal sampling rate sufficient for signal reconstruction with a given accuracy is equal to the spectrum sparsity of the signal sparse approximation that has this accuracy. It is revealed that the compressed sensing approach, which was advanced as a solution to the sampling rate minimization problem, is far from reaching the sampling rate theoretical minimum. Potentials and limitations of compressed sensing are demystified using a simple and intutive model, A method of image Arbitrary Sampling and Bounded Spectrum Reconstruction (ASBSR-method) is described that allows to draw near the image sampling rate theoretical minimum. Presented and discussed are also results of experimental verification of the ASBSR-method and its possible applicability extensions to solving various underdetermined inverse problems such as color image demosaicing, image in-painting, image reconstruction from their sparsely sampled or decimated projections, image reconstruction from the modulus of its Fourier spectrum, and image reconstruction from its sparse samples in Fourier domain
Tasks	Demosaicking, Image Reconstruction
Published	2017-10-10
URL	http://arxiv.org/abs/1710.05985v2
PDF	http://arxiv.org/pdf/1710.05985v2.pdf
PWC	https://paperswithcode.com/paper/compressed-sensing-asbsr-method-of-image
Repo
Framework

Extreme value statistics for censored data with heavy tails under competing risks


Title	Extreme value statistics for censored data with heavy tails under competing risks
Authors	Julien Worms, Rym Worms
Abstract	This paper addresses the problem of estimating, in the presence of random censoring as well as competing risks, the extreme value index of the (sub)-distribution function associated to one particular cause, in the heavy-tail case. Asymptotic normality of the proposed estimator (which has the form of an Aalen-Johansen integral, and is the first estimator proposed in this context) is established. A small simulation study exhibits its performances for finite samples. Estimation of extreme quantiles of the cumulative incidence function is also addressed.
Tasks
Published	2017-01-19
URL	http://arxiv.org/abs/1701.05458v1
PDF	http://arxiv.org/pdf/1701.05458v1.pdf
PWC	https://paperswithcode.com/paper/extreme-value-statistics-for-censored-data
Repo
Framework

Learning Disjunctions of Predicates


Title	Learning Disjunctions of Predicates
Authors	Nader H. Bshouty, Dana Drachsler-Cohen, Martin Vechev, Eran Yahav
Abstract	Let $F$ be a set of boolean functions. We present an algorithm for learning $F_\vee := {\vee_{f\in S} f \mid S \subseteq F}$ from membership queries. Our algorithm asks at most $F \cdot OPT(F_\vee)$ membership queries where $OPT(F_\vee)$ is the minimum worst case number of membership queries for learning $F_\vee$. When $F$ is a set of halfspaces over a constant dimension space or a set of variable inequalities, our algorithm runs in polynomial time. The problem we address has practical importance in the field of program synthesis, where the goal is to synthesize a program that meets some requirements. Program synthesis has become popular especially in settings aiming to help end users. In such settings, the requirements are not provided upfront and the synthesizer can only learn them by posing membership queries to the end user. Our work enables such synthesizers to learn the exact requirements while bounding the number of membership queries.
Tasks	Program Synthesis
Published	2017-06-15
URL	http://arxiv.org/abs/1706.05070v1
PDF	http://arxiv.org/pdf/1706.05070v1.pdf
PWC	https://paperswithcode.com/paper/learning-disjunctions-of-predicates
Repo
Framework

Langevin Dynamics with Continuous Tempering for Training Deep Neural Networks


Title	Langevin Dynamics with Continuous Tempering for Training Deep Neural Networks
Authors	Nanyang Ye, Zhanxing Zhu, Rafal K. Mantiuk
Abstract	Minimizing non-convex and high-dimensional objective functions is challenging, especially when training modern deep neural networks. In this paper, a novel approach is proposed which divides the training process into two consecutive phases to obtain better generalization performance: Bayesian sampling and stochastic optimization. The first phase is to explore the energy landscape and to capture the “fat” modes; and the second one is to fine-tune the parameter learned from the first phase. In the Bayesian learning phase, we apply continuous tempering and stochastic approximation into the Langevin dynamics to create an efficient and effective sampler, in which the temperature is adjusted automatically according to the designed “temperature dynamics”. These strategies can overcome the challenge of early trapping into bad local minima and have achieved remarkable improvements in various types of neural networks as shown in our theoretical analysis and empirical experiments.
Tasks	Stochastic Optimization
Published	2017-03-13
URL	http://arxiv.org/abs/1703.04379v4
PDF	http://arxiv.org/pdf/1703.04379v4.pdf
PWC	https://paperswithcode.com/paper/langevin-dynamics-with-continuous-tempering
Repo
Framework

Learning Dense Facial Correspondences in Unconstrained Images


Title	Learning Dense Facial Correspondences in Unconstrained Images
Authors	Ronald Yu, Shunsuke Saito, Haoxiang Li, Duygu Ceylan, Hao Li
Abstract	We present a minimalistic but effective neural network that computes dense facial correspondences in highly unconstrained RGB images. Our network learns a per-pixel flow and a matchability mask between 2D input photographs of a person and the projection of a textured 3D face model. To train such a network, we generate a massive dataset of synthetic faces with dense labels using renderings of a morphable face model with variations in pose, expressions, lighting, and occlusions. We found that a training refinement using real photographs is required to drastically improve the ability to handle real images. When combined with a facial detection and 3D face fitting step, we show that our approach outperforms the state-of-the-art face alignment methods in terms of accuracy and speed. By directly estimating dense correspondences, we do not rely on the full visibility of sparse facial landmarks and are not limited to the model space of regression-based approaches. We also assess our method on video frames and demonstrate successful per-frame processing under extreme pose variations, occlusions, and lighting conditions. Compared to existing 3D facial tracking techniques, our fitting does not rely on previous frames or frontal facial initialization and is robust to imperfect face detections.
Tasks	Face Alignment
Published	2017-09-02
URL	http://arxiv.org/abs/1709.00536v1
PDF	http://arxiv.org/pdf/1709.00536v1.pdf
PWC	https://paperswithcode.com/paper/learning-dense-facial-correspondences-in
Repo
Framework

Single Image Deraining using Scale-Aware Multi-Stage Recurrent Network


Title	Single Image Deraining using Scale-Aware Multi-Stage Recurrent Network
Authors	Ruoteng Li, Loong-Fah Cheong, Robby T. Tan
Abstract	Given a single input rainy image, our goal is to visually remove rain streaks and the veiling effect caused by scattering and transmission of rain streaks and rain droplets. We are particularly concerned with heavy rain, where rain streaks of various sizes and directions can overlap each other and the veiling effect reduces contrast severely. To achieve our goal, we introduce a scale-aware multi-stage convolutional neural network. Our main idea here is that different sizes of rain-streaks visually degrade the scene in different ways. Large nearby streaks obstruct larger regions and are likely to reflect specular highlights more prominently than smaller distant streaks. These different effects of different streaks have their own characteristics in their image features, and thus need to be treated differently. To realize this, we create parallel sub-networks that are trained and made aware of these different scales of rain streaks. To our knowledge, this idea of parallel sub-networks that treats the same class of objects according to their unique sub-classes is novel, particularly in the context of rain removal. To verify our idea, we conducted experiments on both synthetic and real images, and found that our method is effective and outperforms the state-of-the-art methods.
Tasks	Rain Removal, Single Image Deraining
Published	2017-12-19
URL	http://arxiv.org/abs/1712.06830v1
PDF	http://arxiv.org/pdf/1712.06830v1.pdf
PWC	https://paperswithcode.com/paper/single-image-deraining-using-scale-aware
Repo
Framework

Automated Algorithm Selection on Continuous Black-Box Problems By Combining Exploratory Landscape Analysis and Machine Learning


Title	Automated Algorithm Selection on Continuous Black-Box Problems By Combining Exploratory Landscape Analysis and Machine Learning
Authors	Pascal Kerschke, Heike Trautmann
Abstract	In this paper, we build upon previous work on designing informative and efficient Exploratory Landscape Analysis features for characterizing problems’ landscapes and show their effectiveness in automatically constructing algorithm selection models in continuous black-box optimization problems. Focussing on algorithm performance results of the COCO platform of several years, we construct a representative set of high-performing complementary solvers and present an algorithm selection model that - compared to the portfolio’s single best solver - on average requires less than half of the resources for solving a given problem. Therefore, there is a huge gain in efficiency compared to classical ensemble methods combined with an increased insight into problem characteristics and algorithm properties by using informative features. Acting on the assumption that the function set of the Black-Box Optimization Benchmark is representative enough for practical applications the model allows for selecting the best suited optimization algorithm within the considered set for unseen problems prior to the optimization itself based on a small sample of function evaluations. Note that such a sample can even be reused for the initial population of an evolutionary (optimization) algorithm so that even the feature costs become negligible.
Tasks
Published	2017-11-24
URL	http://arxiv.org/abs/1711.08921v3
PDF	http://arxiv.org/pdf/1711.08921v3.pdf
PWC	https://paperswithcode.com/paper/automated-algorithm-selection-on-continuous
Repo
Framework

Joint Dictionary Learning for Example-based Image Super-resolution


Title	Joint Dictionary Learning for Example-based Image Super-resolution
Authors	Mojtaba Sahraee-Ardakan, Mohsen Joneidi
Abstract	In this paper, we propose a new joint dictionary learning method for example-based image super-resolution (SR), using sparse representation. The low-resolution (LR) dictionary is trained from a set of LR sample image patches. Using the sparse representation coefficients of these LR patches over the LR dictionary, the high-resolution (HR) dictionary is trained by minimizing the reconstruction error of HR sample patches. The error criterion used here is the mean square error. In this way we guarantee that the HR patches have the same sparse representation over HR dictionary as the LR patches over the LR dictionary, and at the same time, these sparse representations can well reconstruct the HR patches. Simulation results show the effectiveness of our method compared to the state-of-art SR algorithms.
Tasks	Dictionary Learning, Image Super-Resolution, Super-Resolution
Published	2017-01-12
URL	http://arxiv.org/abs/1701.03420v1
PDF	http://arxiv.org/pdf/1701.03420v1.pdf
PWC	https://paperswithcode.com/paper/joint-dictionary-learning-for-example-based
Repo
Framework

Leveraging Term Banks for Answering Complex Questions: A Case for Sparse Vectors


Title	Leveraging Term Banks for Answering Complex Questions: A Case for Sparse Vectors
Authors	Peter D. Turney
Abstract	While open-domain question answering (QA) systems have proven effective for answering simple questions, they struggle with more complex questions. Our goal is to answer more complex questions reliably, without incurring a significant cost in knowledge resource construction to support the QA. One readily available knowledge resource is a term bank, enumerating the key concepts in a domain. We have developed an unsupervised learning approach that leverages a term bank to guide a QA system, by representing the terminological knowledge with thousands of specialized vector spaces. In experiments with complex science questions, we show that this approach significantly outperforms several state-of-the-art QA systems, demonstrating that significant leverage can be gained from continuous vector representations of domain terminology.
Tasks	Open-Domain Question Answering, Question Answering
Published	2017-04-11
URL	http://arxiv.org/abs/1704.03543v1
PDF	http://arxiv.org/pdf/1704.03543v1.pdf
PWC	https://paperswithcode.com/paper/leveraging-term-banks-for-answering-complex
Repo
Framework