February 1, 2020

3434 words 17 mins read

Paper Group AWR 194

Adaptive Posterior Learning: few-shot learning with a surprise-based memory module. Learning to Balance: Bayesian Meta-Learning for Imbalanced and Out-of-distribution Tasks. Reinforcement Learning from Hierarchical Critics. ACNN: a Full Resolution DCNN for Medical Image Segmentation. Quality-Gated Convolutional LSTM for Enhancing Compressed Video. …

Adaptive Posterior Learning: few-shot learning with a surprise-based memory module


Title	Adaptive Posterior Learning: few-shot learning with a surprise-based memory module
Authors	Tiago Ramalho, Marta Garnelo
Abstract	The ability to generalize quickly from few observations is crucial for intelligent systems. In this paper we introduce APL, an algorithm that approximates probability distributions by remembering the most surprising observations it has encountered. These past observations are recalled from an external memory module and processed by a decoder network that can combine information from different memory slots to generalize beyond direct recall. We show this algorithm can perform as well as state of the art baselines on few-shot classification benchmarks with a smaller memory footprint. In addition, its memory compression allows it to scale to thousands of unknown labels. Finally, we introduce a meta-learning reasoning task which is more challenging than direct classification. In this setting, APL is able to generalize with fewer than one example per class via deductive reasoning.
Tasks	Few-Shot Learning, Meta-Learning
Published	2019-02-07
URL	http://arxiv.org/abs/1902.02527v1
PDF	http://arxiv.org/pdf/1902.02527v1.pdf
PWC	https://paperswithcode.com/paper/adaptive-posterior-learning-few-shot-learning
Repo	https://github.com/cogentlabs/apl
Framework	pytorch

Learning to Balance: Bayesian Meta-Learning for Imbalanced and Out-of-distribution Tasks


Title	Learning to Balance: Bayesian Meta-Learning for Imbalanced and Out-of-distribution Tasks
Authors	Hae Beom Lee, Hayeon Lee, Donghyun Na, Saehoon Kim, Minseop Park, Eunho Yang, Sung Ju Hwang
Abstract	While tasks could come with varying the number of instances and classes in realistic settings, the existing meta-learning approaches for few-shot classification assume that the number of instances per task and class is fixed. Due to such restriction, they learn to equally utilize the meta-knowledge across all the tasks, even when the number of instances per task and class largely varies. Moreover, they do not consider distributional difference in unseen tasks, on which the meta-knowledge may have less usefulness depending on the task relatedness. To overcome these limitations, we propose a novel meta-learning model that adaptively balances the effect of the meta-learning and task-specific learning within each task. Through the learning of the balancing variables, we can decide whether to obtain a solution by relying on the meta-knowledge or task-specific learning. We formulate this objective into a Bayesian inference framework and tackle it using variational inference. We validate our Bayesian Task-Adaptive Meta-Learning (Bayesian TAML) on multiple realistic task- and class-imbalanced datasets, on which it significantly outperforms existing meta-learning approaches. Further ablation study confirms the effectiveness of each balancing component and the Bayesian learning framework.
Tasks	Bayesian Inference, Meta-Learning
Published	2019-05-30
URL	https://arxiv.org/abs/1905.12917v2
PDF	https://arxiv.org/pdf/1905.12917v2.pdf
PWC	https://paperswithcode.com/paper/learning-to-balance-bayesian-meta-learning
Repo	https://github.com/haebeom-lee/l2b
Framework	tf

Reinforcement Learning from Hierarchical Critics


Title	Reinforcement Learning from Hierarchical Critics
Authors	Zehong Cao, Chin-Teng Lin
Abstract	In this study, we investigate the use of global information to speed up the learning process and increase the cumulative rewards of reinforcement learning (RL) in competition tasks. Within the actor-critic RL, we introduce multiple cooperative critics from two levels of the hierarchy and propose a reinforcement learning from hierarchical critics (RLHC) algorithm. In our approach, each agent receives value information from local and global critics regarding a competition task and accesses multiple cooperative critics in a top-down hierarchy. Thus, each agent not only receives low-level details but also considers coordination from higher levels, thereby obtaining global information to improve the training performance. Then, we test the proposed RLHC algorithm against the benchmark algorithm, proximal policy optimisation (PPO), for two experimental scenarios performed in a Unity environment consisting of tennis and soccer agents’ competitions. The results showed that RLHC outperforms the benchmark on both competition tasks.
Tasks	Multi-agent Reinforcement Learning
Published	2019-02-08
URL	https://arxiv.org/abs/1902.03079v4
PDF	https://arxiv.org/pdf/1902.03079v4.pdf
PWC	https://paperswithcode.com/paper/hierarchical-critics-assignment-for-multi
Repo	https://github.com/czh513/Hierarchical-Critics-Assignment-for-Multi-agent-Reinforcement-Learning
Framework	tf

ACNN: a Full Resolution DCNN for Medical Image Segmentation


Title	ACNN: a Full Resolution DCNN for Medical Image Segmentation
Authors	Xiao-Yun Zhou, Jian-Qing Zheng, Peichao Li, Guang-Zhong Yang
Abstract	Deep Convolutional Neural Networks (DCNNs) are used extensively in medical image segmentation and hence 3D navigation for robot-assisted Minimally Invasive Surgeries (MISs). However, current DCNNs usually use down sampling layers for increasing the receptive field and gaining abstract semantic information. These down sampling layers decrease the spatial dimension of feature maps, which can be detrimental to image segmentation. Atrous convolution is an alternative for the down sampling layer. It increases the receptive field whilst maintains the spatial dimension of feature maps. In this paper, a method for effective atrous rate setting is proposed to achieve the largest and fully-covered receptive field with a minimum number of atrous convolutional layers. Furthermore, a new and full resolution DCNN - Atrous Convolutional Neural Network (ACNN), which incorporates cascaded atrous II-blocks, residual learning and Instance Normalization (IN). Application results of the proposed ACNN to Magnetic Resonance Imaging (MRI) and Computed Tomography (CT) image segmentation demonstrate that the proposed ACNN can achieve higher segmentation Intersection over Unions (IoUs) to U-Net and Deeplabv3+, but with significantly reduced trainable parameters.
Tasks	Computed Tomography (CT), Medical Image Segmentation, Semantic Segmentation
Published	2019-01-26
URL	https://arxiv.org/abs/1901.09203v3
PDF	https://arxiv.org/pdf/1901.09203v3.pdf
PWC	https://paperswithcode.com/paper/atrous-convolutional-neural-network-acnn-for
Repo	https://github.com/IsaacPatole/Traffic-Net-Classification
Framework	none

Quality-Gated Convolutional LSTM for Enhancing Compressed Video


Title	Quality-Gated Convolutional LSTM for Enhancing Compressed Video
Authors	Ren Yang, Xiaoyan Sun, Mai Xu, Wenjun Zeng
Abstract	The past decade has witnessed great success in applying deep learning to enhance the quality of compressed video. However, the existing approaches aim at quality enhancement on a single frame, or only using fixed neighboring frames. Thus they fail to take full advantage of the inter-frame correlation in the video. This paper proposes the Quality-Gated Convolutional Long Short-Term Memory (QG-ConvLSTM) network with bi-directional recurrent structure to fully exploit the advantageous information in a large range of frames. More importantly, due to the obvious quality fluctuation among compressed frames, higher quality frames can provide more useful information for other frames to enhance quality. Therefore, we propose learning the “forget” and “input” gates in the ConvLSTM cell from quality-related features. As such, the frames with various quality contribute to the memory in ConvLSTM with different importance, making the information of each frame reasonably and adequately used. Finally, the experiments validate the effectiveness of our QG-ConvLSTM approach in advancing the state-of-the-art quality enhancement of compressed video, and the ablation study shows that our QG-ConvLSTM approach is learnt to make a trade-off between quality and correlation when leveraging multi-frame information. The project page: https://github.com/ryangchn/QG-ConvLSTM.git.
Tasks
Published	2019-03-11
URL	http://arxiv.org/abs/1903.04596v3
PDF	http://arxiv.org/pdf/1903.04596v3.pdf
PWC	https://paperswithcode.com/paper/quality-gated-convolutional-lstm-for
Repo	https://github.com/ryangchn/QG-ConvLSTM
Framework	tf

Improving Question Answering with External Knowledge


Title	Improving Question Answering with External Knowledge
Authors	Xiaoman Pan, Kai Sun, Dian Yu, Jianshu Chen, Heng Ji, Claire Cardie, Dong Yu
Abstract	We focus on multiple-choice question answering (QA) tasks in subject areas such as science, where we require both broad background knowledge and the facts from the given subject-area reference corpus. In this work, we explore simple yet effective methods for exploiting two sources of external knowledge for subject-area QA. The first enriches the original subject-area reference corpus with relevant text snippets extracted from an open-domain resource (i.e., Wikipedia) that cover potentially ambiguous concepts in the question and answer options. As in other QA research, the second method simply increases the amount of training data by appending additional in-domain subject-area instances. Experiments on three challenging multiple-choice science QA tasks (i.e., ARC-Easy, ARC-Challenge, and OpenBookQA) demonstrate the effectiveness of our methods: in comparison to the previous state-of-the-art, we obtain absolute gains in accuracy of up to 8.1%, 13.0%, and 12.8%, respectively. While we observe consistent gains when we introduce knowledge from Wikipedia, we find that employing additional QA training instances is not uniformly helpful: performance degrades when the added instances exhibit a higher level of difficulty than the original training data. As one of the first studies on exploiting unstructured external knowledge for subject-area QA, we hope our methods, observations, and discussion of the exposed limitations may shed light on further developments in the area.
Tasks	Question Answering
Published	2019-02-03
URL	https://arxiv.org/abs/1902.00993v3
PDF	https://arxiv.org/pdf/1902.00993v3.pdf
PWC	https://paperswithcode.com/paper/improving-question-answering-with-external
Repo	https://github.com/nlpdata/external
Framework	none

Unsupervised Domain Adversarial Self-Calibration for Electromyographic-based Gesture Recognition


Title	Unsupervised Domain Adversarial Self-Calibration for Electromyographic-based Gesture Recognition
Authors	Ulysse Côté-Allard, Gabriel Gagnon-Turcotte, Angkoon Phinyomark, Kyrre Glette, Erik Scheme, François Laviolette, Benoit Gosselin
Abstract	Surface electromyography (sEMG) provides an intuitive and non-invasive interface from which to control machines. However, preserving the myoelectric control system’s performance over multiple days is challenging, due to the transient nature of this recording technique. In practice, if the system is to remain usable, a time-consuming and periodic re-calibration is necessary. In the case where the sEMG interface is employed every few days, the user might need to do this re-calibration before every use. Thus, severely limiting the practicality of such a control method. Consequently, this paper proposes tackling the especially challenging task of adapting to sEMG signals when multiple days have elapsed between each recording, by presenting SCADANN, a new, deep learning-based, self-calibrating algorithm. SCADANN is ranked against three state of the art domain adversarial algorithms and a multiple-vote self-calibrating algorithm on both offline and online datasets. Overall, SCADANN is shown to systematically improve classifiers’ performance over no adaptation and ranks first on almost all the cases tested.
Tasks	Calibration, Gesture Recognition
Published	2019-12-21
URL	https://arxiv.org/abs/1912.11037v1
PDF	https://arxiv.org/pdf/1912.11037v1.pdf
PWC	https://paperswithcode.com/paper/unsupervised-domain-adversarial-self
Repo	https://github.com/UlysseCoteAllard/LongTermEMG
Framework	pytorch

Prediction, Consistency, Curvature: Representation Learning for Locally-Linear Control


Title	Prediction, Consistency, Curvature: Representation Learning for Locally-Linear Control
Authors	Nir Levine, Yinlam Chow, Rui Shu, Ang Li, Mohammad Ghavamzadeh, Hung Bui
Abstract	Many real-world sequential decision-making problems can be formulated as optimal control with high-dimensional observations and unknown dynamics. A promising approach is to embed the high-dimensional observations into a lower-dimensional latent representation space, estimate the latent dynamics model, then utilize this model for control in the latent space. An important open question is how to learn a representation that is amenable to existing control algorithms? In this paper, we focus on learning representations for locally-linear control algorithms, such as iterative LQR (iLQR). By formulating and analyzing the representation learning problem from an optimal control perspective, we establish three underlying principles that the learned representation should comprise: 1) accurate prediction in the observation space, 2) consistency between latent and observation space dynamics, and 3) low curvature in the latent space transitions. These principles naturally correspond to a loss function that consists of three terms: prediction, consistency, and curvature (PCC). Crucially, to make PCC tractable, we derive an amortized variational bound for the PCC loss function. Extensive experiments on benchmark domains demonstrate that the new variational-PCC learning algorithm benefits from significantly more stable and reproducible training, and leads to superior control performance. Further ablation studies give support to the importance of all three PCC components for learning a good latent space for control.
Tasks	Decision Making, Representation Learning
Published	2019-09-04
URL	https://arxiv.org/abs/1909.01506v2
PDF	https://arxiv.org/pdf/1909.01506v2.pdf
PWC	https://paperswithcode.com/paper/prediction-consistency-curvature
Repo	https://github.com/VinAIResearch/PCC-pytorch
Framework	pytorch

Image Super-Resolution as a Defense Against Adversarial Attacks


Title	Image Super-Resolution as a Defense Against Adversarial Attacks
Authors	Aamir Mustafa, Salman H. Khan, Munawar Hayat, Jianbing Shen, Ling Shao
Abstract	Convolutional Neural Networks have achieved significant success across multiple computer vision tasks. However, they are vulnerable to carefully crafted, human-imperceptible adversarial noise patterns which constrain their deployment in critical security-sensitive systems. This paper proposes a computationally efficient image enhancement approach that provides a strong defense mechanism to effectively mitigate the effect of such adversarial perturbations. We show that deep image restoration networks learn mapping functions that can bring off-the-manifold adversarial samples onto the natural image manifold, thus restoring classification towards correct classes. A distinguishing feature of our approach is that, in addition to providing robustness against attacks, it simultaneously enhances image quality and retains models performance on clean images. Furthermore, the proposed method does not modify the classifier or requires a separate mechanism to detect adversarial images. The effectiveness of the scheme has been demonstrated through extensive experiments, where it has proven a strong defense in gray-box settings. The proposed scheme is simple and has the following advantages: (1) it does not require any model training or parameter optimization, (2) it complements other existing defense mechanisms, (3) it is agnostic to the attacked model and attack type and (4) it provides superior performance across all popular attack algorithms. Our codes are publicly available at https://github.com/aamir-mustafa/super-resolution-adversarial-defense.
Tasks	Adversarial Defense, Image Enhancement, Image Restoration, Image Super-Resolution, Super-Resolution
Published	2019-01-07
URL	https://arxiv.org/abs/1901.01677v2
PDF	https://arxiv.org/pdf/1901.01677v2.pdf
PWC	https://paperswithcode.com/paper/image-super-resolution-as-a-defense-against
Repo	https://github.com/aamir-mustafa/super-resolution-adversarial-defense
Framework	pytorch

Curriculum Learning Strategies for IR: An Empirical Study on Conversation Response Ranking


Title	Curriculum Learning Strategies for IR: An Empirical Study on Conversation Response Ranking
Authors	Gustavo Penha, Claudia Hauff
Abstract	Neural ranking models are traditionally trained on a series of random batches, sampled uniformly from the entire training set. Curriculum learning has recently been shown to improve neural models’ effectiveness by sampling batches non-uniformly, going from easy to difficult instances during training. In the context of neural Information Retrieval (IR) curriculum learning has not been explored yet, and so it remains unclear (1) how to measure the difficulty of training instances and (2) how to transition from easy to difficult instances during training. To address both challenges and determine whether curriculum learning is beneficial for neural ranking models, we need large-scale datasets and a retrieval task that allows us to conduct a wide range of experiments. For this purpose, we resort to the task of conversation response ranking: ranking responses given the conversation history. In order to deal with challenge (1), we explore scoring functions to measure the difficulty of conversations based on different input spaces. To address challenge (2) we evaluate different pacing functions, which determine the velocity in which we go from easy to difficult instances. We find that, overall, by just intelligently sorting the training data (i.e., by performing curriculum learning) we can improve the retrieval effectiveness by up to 2%.
Tasks	Information Retrieval
Published	2019-12-18
URL	https://arxiv.org/abs/1912.08555v1
PDF	https://arxiv.org/pdf/1912.08555v1.pdf
PWC	https://paperswithcode.com/paper/curriculum-learning-strategies-for-ir-an
Repo	https://github.com/Guzpenha/transformers_cl
Framework	pytorch

Fast Underwater Image Enhancement for Improved Visual Perception


Title	Fast Underwater Image Enhancement for Improved Visual Perception
Authors	Md Jahidul Islam, Youya Xia, Junaed Sattar
Abstract	In this paper, we present a conditional generative adversarial network-based model for real-time underwater image enhancement. To supervise the adversarial training, we formulate an objective function that evaluates the perceptual image quality based on its global content, color, local texture, and style information. We also present EUVP, a large-scale dataset of a paired and unpaired collection of underwater images (of `poor' and` good’ quality) that are captured using seven different cameras over various visibility conditions during oceanic explorations and human-robot collaborative experiments. In addition, we perform several qualitative and quantitative evaluations which suggest that the proposed model can learn to enhance underwater image quality from both paired and unpaired training. More importantly, the enhanced images provide improved performances of standard models for underwater object detection, human pose estimation, and saliency prediction. These results validate that it is suitable for real-time preprocessing in the autonomy pipeline by visually-guided underwater robots. The model and associated training pipelines are available at https://github.com/xahidbuffon/funie-gan.
Tasks	Image Enhancement, Object Detection, Pose Estimation, Saliency Prediction
Published	2019-03-23
URL	https://arxiv.org/abs/1903.09766v3
PDF	https://arxiv.org/pdf/1903.09766v3.pdf
PWC	https://paperswithcode.com/paper/fast-underwater-image-enhancement-for
Repo	https://github.com/xahidbuffon/funie-gan
Framework	tf

FEELVOS: Fast End-to-End Embedding Learning for Video Object Segmentation


Title	FEELVOS: Fast End-to-End Embedding Learning for Video Object Segmentation
Authors	Paul Voigtlaender, Yuning Chai, Florian Schroff, Hartwig Adam, Bastian Leibe, Liang-Chieh Chen
Abstract	Many of the recent successful methods for video object segmentation (VOS) are overly complicated, heavily rely on fine-tuning on the first frame, and/or are slow, and are hence of limited practical use. In this work, we propose FEELVOS as a simple and fast method which does not rely on fine-tuning. In order to segment a video, for each frame FEELVOS uses a semantic pixel-wise embedding together with a global and a local matching mechanism to transfer information from the first frame and from the previous frame of the video to the current frame. In contrast to previous work, our embedding is only used as an internal guidance of a convolutional network. Our novel dynamic segmentation head allows us to train the network, including the embedding, end-to-end for the multiple object segmentation task with a cross entropy loss. We achieve a new state of the art in video object segmentation without fine-tuning with a J&F measure of 71.5% on the DAVIS 2017 validation set. We make our code and models available at https://github.com/tensorflow/models/tree/master/research/feelvos.
Tasks	Semantic Segmentation, Video Object Segmentation, Video Semantic Segmentation
Published	2019-02-25
URL	http://arxiv.org/abs/1902.09513v2
PDF	http://arxiv.org/pdf/1902.09513v2.pdf
PWC	https://paperswithcode.com/paper/feelvos-fast-end-to-end-embedding-learning
Repo	https://github.com/tensorflow/models/tree/master/research/feelvos
Framework	tf

Leveraging Procedural Generation to Benchmark Reinforcement Learning


Title	Leveraging Procedural Generation to Benchmark Reinforcement Learning
Authors	Karl Cobbe, Christopher Hesse, Jacob Hilton, John Schulman
Abstract	In this report, we introduce Procgen Benchmark, a suite of 16 procedurally generated game-like environments designed to benchmark both sample efficiency and generalization in reinforcement learning. We believe that the community will benefit from increased access to high quality training environments, and we provide detailed experimental protocols for using this benchmark. We empirically demonstrate that diverse environment distributions are essential to adequately train and evaluate RL agents, thereby motivating the extensive use of procedural content generation. We then use this benchmark to investigate the effects of scaling model size, finding that larger models significantly improve both sample efficiency and generalization.
Tasks
Published	2019-12-03
URL	https://arxiv.org/abs/1912.01588v1
PDF	https://arxiv.org/pdf/1912.01588v1.pdf
PWC	https://paperswithcode.com/paper/leveraging-procedural-generation-to-benchmark
Repo	https://github.com/openai/train-procgen
Framework	tf

Triple2Vec: Learning Triple Embeddings from Knowledge Graphs


Title	Triple2Vec: Learning Triple Embeddings from Knowledge Graphs
Authors	Valeria Fionda, Giuseppe Pirró
Abstract	Graph embedding techniques allow to learn high-quality feature vectors from graph structures and are useful in a variety of tasks, from node classification to clustering. Existing approaches have only focused on learning feature vectors for the nodes in a (knowledge) graph. To the best of our knowledge, none of them has tackled the problem of embedding of graph edges, that is, knowledge graph triples. The approaches that are closer to this task have focused on homogeneous graphs involving only one type of edge and obtain edge embeddings by applying some operation (e.g., average) on the embeddings of the endpoint nodes. The goal of this paper is to introduce Triple2Vec, a new technique to directly embed edges in (knowledge) graphs. Trple2Vec builds upon three main ingredients. The first is the notion of line graph. The line graph of a graph is another graph representing the adjacency between edges of the original graph. In particular, the nodes of the line graph are the edges of the original graph. We show that directly applying existing embedding techniques on the nodes of the line graph to learn edge embeddings is not enough in the context of knowledge graphs. Thus, we introduce the notion of triple line graph. The second is an edge weighting mechanism both for line graphs derived from knowledge graphs and homogeneous graphs. The third is a strategy based on graph walks on the weighted triple line graph that can preserve proximity between nodes. Embeddings are finally generated by adopting the SkipGram model, where sentences are replaced with graph walks. We evaluate our approach on different real world (knowledge) graphs and compared it with related work.
Tasks	Graph Embedding, Knowledge Graphs, Node Classification
Published	2019-05-28
URL	https://arxiv.org/abs/1905.11691v1
PDF	https://arxiv.org/pdf/1905.11691v1.pdf
PWC	https://paperswithcode.com/paper/triple2vec-learning-triple-embeddings-from
Repo	https://github.com/Chrisackerman1/Triple2Vec-Learning-Triple-Embeddings-from-Knowledge-Graphs
Framework	none

Novelty Search for Deep Reinforcement Learning Policy Network Weights by Action Sequence Edit Metric Distance


Title	Novelty Search for Deep Reinforcement Learning Policy Network Weights by Action Sequence Edit Metric Distance
Authors	Ethan C. Jackson, Mark Daley
Abstract	Reinforcement learning (RL) problems often feature deceptive local optima, and learning methods that optimize purely for reward signal often fail to learn strategies for overcoming them. Deep neuroevolution and novelty search have been proposed as effective alternatives to gradient-based methods for learning RL policies directly from pixels. In this paper, we introduce and evaluate the use of novelty search over agent action sequences by string edit metric distance as a means for promoting innovation. We also introduce a method for stagnation detection and population resampling inspired by recent developments in the RL community that uses the same mechanisms as novelty search to promote and develop innovative policies. Our methods extend a state-of-the-art method for deep neuroevolution using a simple-yet-effective genetic algorithm (GA) designed to efficiently learn deep RL policy network weights. Experiments using four games from the Atari 2600 benchmark were conducted. Results provide further evidence that GAs are competitive with gradient-based algorithms for deep RL. Results also demonstrate that novelty search over action sequences is an effective source of selection pressure that can be integrated into existing evolutionary algorithms for deep RL.
Tasks
Published	2019-02-08
URL	http://arxiv.org/abs/1902.03142v1
PDF	http://arxiv.org/pdf/1902.03142v1.pdf
PWC	https://paperswithcode.com/paper/novelty-search-for-deep-reinforcement
Repo	https://github.com/ethancjackson/NoveltySearchLevenshtein
Framework	none