October 20, 2019

3073 words 15 mins read

Paper Group AWR 230

Automated Curriculum Learning by Rewarding Temporally Rare Events. pair2vec: Compositional Word-Pair Embeddings for Cross-Sentence Inference. Efficient Bias-Span-Constrained Exploration-Exploitation in Reinforcement Learning. Cross-Domain 3D Equivariant Image Embeddings. Decentralized Computation Offloading for Multi-User Mobile Edge Computing: A D …

Automated Curriculum Learning by Rewarding Temporally Rare Events


Title	Automated Curriculum Learning by Rewarding Temporally Rare Events
Authors	Niels Justesen, Sebastian Risi
Abstract	Reward shaping allows reinforcement learning (RL) agents to accelerate learning by receiving additional reward signals. However, these signals can be difficult to design manually, especially for complex RL tasks. We propose a simple and general approach that determines the reward of pre-defined events by their rarity alone. Here events become less rewarding as they are experienced more often, which encourages the agent to continually explore new types of events as it learns. The adaptiveness of this reward function results in a form of automated curriculum learning that does not have to be specified by the experimenter. We demonstrate that this \emph{Rarity of Events} (RoE) approach enables the agent to succeed in challenging VizDoom scenarios without access to the extrinsic reward from the environment. Furthermore, the results demonstrate that RoE learns a more versatile policy that adapts well to critical changes in the environment. Rewarding events based on their rarity could help in many unsolved RL environments that are characterized by sparse extrinsic rewards but a plethora of known event types.
Tasks
Published	2018-03-19
URL	http://arxiv.org/abs/1803.07131v2
PDF	http://arxiv.org/pdf/1803.07131v2.pdf
PWC	https://paperswithcode.com/paper/automated-curriculum-learning-by-rewarding
Repo	https://github.com/lasseuth1/blood_bowl2
Framework	pytorch

pair2vec: Compositional Word-Pair Embeddings for Cross-Sentence Inference


Title	pair2vec: Compositional Word-Pair Embeddings for Cross-Sentence Inference
Authors	Mandar Joshi, Eunsol Choi, Omer Levy, Daniel S. Weld, Luke Zettlemoyer
Abstract	Reasoning about implied relationships (e.g., paraphrastic, common sense, encyclopedic) between pairs of words is crucial for many cross-sentence inference problems. This paper proposes new methods for learning and using embeddings of word pairs that implicitly represent background knowledge about such relationships. Our pairwise embeddings are computed as a compositional function on word representations, which is learned by maximizing the pointwise mutual information (PMI) with the contexts in which the two words co-occur. We add these representations to the cross-sentence attention layer of existing inference models (e.g. BiDAF for QA, ESIM for NLI), instead of extending or replacing existing word embeddings. Experiments show a gain of 2.7% on the recently released SQuAD2.0 and 1.3% on MultiNLI. Our representations also aid in better generalization with gains of around 6-7% on adversarial SQuAD datasets, and 8.8% on the adversarial entailment test set by Glockner et al. (2018).
Tasks	Common Sense Reasoning, Word Embeddings
Published	2018-10-20
URL	http://arxiv.org/abs/1810.08854v2
PDF	http://arxiv.org/pdf/1810.08854v2.pdf
PWC	https://paperswithcode.com/paper/pair2vec-compositional-word-pair-embeddings
Repo	https://github.com/ghlee0304/NLP-with-tensorflow-pytorch
Framework	tf

Efficient Bias-Span-Constrained Exploration-Exploitation in Reinforcement Learning


Title	Efficient Bias-Span-Constrained Exploration-Exploitation in Reinforcement Learning
Authors	Ronan Fruit, Matteo Pirotta, Alessandro Lazaric, Ronald Ortner
Abstract	We introduce SCAL, an algorithm designed to perform efficient exploration-exploitation in any unknown weakly-communicating Markov decision process (MDP) for which an upper bound $c$ on the span of the optimal bias function is known. For an MDP with $S$ states, $A$ actions and $\Gamma \leq S$ possible next states, we prove a regret bound of $\widetilde{O}(c\sqrt{\Gamma SAT})$, which significantly improves over existing algorithms (e.g., UCRL and PSRL), whose regret scales linearly with the MDP diameter $D$. In fact, the optimal bias span is finite and often much smaller than $D$ (e.g., $D=\infty$ in non-communicating MDPs). A similar result was originally derived by Bartlett and Tewari (2009) for REGAL.C, for which no tractable algorithm is available. In this paper, we relax the optimization problem at the core of REGAL.C, we carefully analyze its properties, and we provide the first computationally efficient algorithm to solve it. Finally, we report numerical simulations supporting our theoretical findings and showing how SCAL significantly outperforms UCRL in MDPs with large diameter and small span.
Tasks	Efficient Exploration
Published	2018-02-12
URL	http://arxiv.org/abs/1802.04020v2
PDF	http://arxiv.org/pdf/1802.04020v2.pdf
PWC	https://paperswithcode.com/paper/efficient-bias-span-constrained-exploration
Repo	https://github.com/RonanFR/UCRL
Framework	none

Cross-Domain 3D Equivariant Image Embeddings


Title	Cross-Domain 3D Equivariant Image Embeddings
Authors	Carlos Esteves, Avneesh Sud, Zhengyi Luo, Kostas Daniilidis, Ameesh Makadia
Abstract	Spherical convolutional networks have been introduced recently as tools to learn powerful feature representations of 3D shapes. Spherical CNNs are equivariant to 3D rotations making them ideally suited to applications where 3D data may be observed in arbitrary orientations. In this paper we learn 2D image embeddings with a similar equivariant structure: embedding the image of a 3D object should commute with rotations of the object. We introduce a cross-domain embedding from 2D images into a spherical CNN latent space. This embedding encodes images with 3D shape properties and is equivariant to 3D rotations of the observed object. The model is supervised only by target embeddings obtained from a spherical CNN pretrained for 3D shape classification. We show that learning a rich embedding for images with appropriate geometric structure is sufficient for tackling varied applications, such as relative pose estimation and novel view synthesis, without requiring additional task-specific supervision.
Tasks	Novel View Synthesis, Pose Estimation
Published	2018-12-06
URL	https://arxiv.org/abs/1812.02716v2
PDF	https://arxiv.org/pdf/1812.02716v2.pdf
PWC	https://paperswithcode.com/paper/cross-domain-3d-equivariant-image-embeddings
Repo	https://github.com/machc/spherical_embeddings
Framework	tf

Decentralized Computation Offloading for Multi-User Mobile Edge Computing: A Deep Reinforcement Learning Approach


Title	Decentralized Computation Offloading for Multi-User Mobile Edge Computing: A Deep Reinforcement Learning Approach
Authors	Zhao Chen, Xiaodong Wang
Abstract	Mobile edge computing (MEC) emerges recently as a promising solution to relieve resource-limited mobile devices from computation-intensive tasks, which enables devices to offload workloads to nearby MEC servers and improve the quality of computation experience. Nevertheless, by considering a MEC system consisting of multiple mobile users with stochastic task arrivals and wireless channels in this paper, the design of computation offloading policies is challenging to minimize the long-term average computation cost in terms of power consumption and buffering delay. A deep reinforcement learning (DRL) based decentralized dynamic computation offloading strategy is investigated to build a scalable MEC system with limited feedback. Specifically, a continuous action space-based DRL approach named deep deterministic policy gradient (DDPG) is adopted to learn efficient computation offloading policies independently at each mobile user. Thus, powers of both local execution and task offloading can be adaptively allocated by the learned policies from each user’s local observation of the MEC system. Numerical results are illustrated to demonstrate that efficient policies can be learned at each user, and performance of the proposed DDPG based decentralized strategy outperforms the conventional deep Q-network (DQN) based discrete power control strategy and some other greedy strategies with reduced computation cost. Besides, the power-delay tradeoff is also analyzed for both the DDPG based and DQN based strategies.
Tasks
Published	2018-12-16
URL	http://arxiv.org/abs/1812.07394v1
PDF	http://arxiv.org/pdf/1812.07394v1.pdf
PWC	https://paperswithcode.com/paper/decentralized-computation-offloading-for
Repo	https://github.com/swordest/mec_drl
Framework	tf

Emergence of Communication in an Interactive World with Consistent Speakers


Title	Emergence of Communication in an Interactive World with Consistent Speakers
Authors	Ben Bogin, Mor Geva, Jonathan Berant
Abstract	Training agents to communicate with one another given task-based supervision only has attracted considerable attention recently, due to the growing interest in developing models for human-agent interaction. Prior work on the topic focused on simple environments, where training using policy gradient was feasible despite the non-stationarity of the agents during training. In this paper, we present a more challenging environment for testing the emergence of communication from raw pixels, where training using policy gradient fails. We propose a new model and training algorithm, that utilizes the structure of a learned representation space to produce more consistent speakers at the initial phases of training, which stabilizes learning. We empirically show that our algorithm substantially improves performance compared to policy gradient. We also propose a new alignment-based metric for measuring context-independence in emerged communication and find our method increases context-independence compared to policy gradient and other competitive baselines.
Tasks
Published	2018-09-03
URL	http://arxiv.org/abs/1809.00549v2
PDF	http://arxiv.org/pdf/1809.00549v2.pdf
PWC	https://paperswithcode.com/paper/emergence-of-communication-in-an-interactive
Repo	https://github.com/benbogin/emergence-communication-cco
Framework	none

Ordinal Pooling Networks: For Preserving Information over Shrinking Feature Maps


Title	Ordinal Pooling Networks: For Preserving Information over Shrinking Feature Maps
Authors	Ashwani Kumar
Abstract	In the framework of convolutional neural networks that lie at the heart of deep learning, downsampling is often performed with a max-pooling operation that only retains the element with maximum activation, while completely discarding the information contained in other elements in a pooling region. To address this issue, a novel pooling scheme, Ordinal Pooling Network (OPN), is introduced in this work. OPN rearranges all the elements of a pooling region in a sequence and assigns different weights to these elements based upon their orders in the sequence, where the weights are learned via the gradient-based optimisation. The results of our small-scale experiments on image classification task demonstrate that this scheme leads to a consistent improvement in the accuracy over max-pooling operation. This improvement is expected to increase in deeper networks, where several layers of pooling become necessary.
Tasks	Image Classification
Published	2018-04-08
URL	http://arxiv.org/abs/1804.02702v2
PDF	http://arxiv.org/pdf/1804.02702v2.pdf
PWC	https://paperswithcode.com/paper/ordinal-pooling-networks-for-preserving
Repo	https://github.com/ash80/Ordinal-Pooling-Networks
Framework	tf

Training for Faster Adversarial Robustness Verification via Inducing ReLU Stability


Title	Training for Faster Adversarial Robustness Verification via Inducing ReLU Stability
Authors	Kai Y. Xiao, Vincent Tjeng, Nur Muhammad Shafiullah, Aleksander Madry
Abstract	We explore the concept of co-design in the context of neural network verification. Specifically, we aim to train deep neural networks that not only are robust to adversarial perturbations but also whose robustness can be verified more easily. To this end, we identify two properties of network models - weight sparsity and so-called ReLU stability - that turn out to significantly impact the complexity of the corresponding verification task. We demonstrate that improving weight sparsity alone already enables us to turn computationally intractable verification problems into tractable ones. Then, improving ReLU stability leads to an additional 4-13x speedup in verification times. An important feature of our methodology is its “universality,” in the sense that it can be used with a broad range of training procedures and verification approaches.
Tasks
Published	2018-09-09
URL	http://arxiv.org/abs/1809.03008v3
PDF	http://arxiv.org/pdf/1809.03008v3.pdf
PWC	https://paperswithcode.com/paper/training-for-faster-adversarial-robustness
Repo	https://github.com/MadryLab/relu_stable
Framework	tf

Generalisation in humans and deep neural networks


Title	Generalisation in humans and deep neural networks
Authors	Robert Geirhos, Carlos R. Medina Temme, Jonas Rauber, Heiko H. Schütt, Matthias Bethge, Felix A. Wichmann
Abstract	We compare the robustness of humans and current convolutional deep neural networks (DNNs) on object recognition under twelve different types of image degradations. First, using three well known DNNs (ResNet-152, VGG-19, GoogLeNet) we find the human visual system to be more robust to nearly all of the tested image manipulations, and we observe progressively diverging classification error-patterns between humans and DNNs when the signal gets weaker. Secondly, we show that DNNs trained directly on distorted images consistently surpass human performance on the exact distortion types they were trained on, yet they display extremely poor generalisation abilities when tested on other distortion types. For example, training on salt-and-pepper noise does not imply robustness on uniform white noise and vice versa. Thus, changes in the noise distribution between training and testing constitutes a crucial challenge to deep learning vision systems that can be systematically addressed in a lifelong machine learning approach. Our new dataset consisting of 83K carefully measured human psychophysical trials provide a useful reference for lifelong robustness against image degradations set by the human visual system.
Tasks	Object Recognition
Published	2018-08-27
URL	http://arxiv.org/abs/1808.08750v2
PDF	http://arxiv.org/pdf/1808.08750v2.pdf
PWC	https://paperswithcode.com/paper/generalisation-in-humans-and-deep-neural
Repo	https://github.com/rgeirhos/generalisation-humans-DNNs
Framework	tf

DeClarE: Debunking Fake News and False Claims using Evidence-Aware Deep Learning


Title	DeClarE: Debunking Fake News and False Claims using Evidence-Aware Deep Learning
Authors	Kashyap Popat, Subhabrata Mukherjee, Andrew Yates, Gerhard Weikum
Abstract	Misinformation such as fake news is one of the big challenges of our society. Research on automated fact-checking has proposed methods based on supervised learning, but these approaches do not consider external evidence apart from labeled training instances. Recent approaches counter this deficit by considering external sources related to a claim. However, these methods require substantial feature modeling and rich lexicons. This paper overcomes these limitations of prior work with an end-to-end model for evidence-aware credibility assessment of arbitrary textual claims, without any human intervention. It presents a neural network model that judiciously aggregates signals from external evidence articles, the language of these articles and the trustworthiness of their sources. It also derives informative features for generating user-comprehensible explanations that makes the neural network predictions transparent to the end-user. Experiments with four datasets and ablation studies show the strength of our method.
Tasks
Published	2018-09-17
URL	http://arxiv.org/abs/1809.06416v1
PDF	http://arxiv.org/pdf/1809.06416v1.pdf
PWC	https://paperswithcode.com/paper/declare-debunking-fake-news-and-false-claims
Repo	https://github.com/atulkumarin/DeClare
Framework	pytorch

Layouts from Panoramic Images with Geometry and Deep Learning


Title	Layouts from Panoramic Images with Geometry and Deep Learning
Authors	Clara Fernandez-Labrador, Alejandro Perez-Yus, Gonzalo Lopez-Nicolas, Jose J. Guerrero
Abstract	In this paper, we propose a novel procedure for 3D layout recovery of indoor scenes from single 360 degrees panoramic images. With such images, all scene is seen at once, allowing to recover closed geometries. Our method combines strategically the accuracy provided by geometric reasoning (lines and vanishing points) with the higher level of data abstraction and pattern recognition achieved by deep learning techniques (edge and normal maps). Thus, we extract structural corners from which we generate layout hypotheses of the room assuming Manhattan world. The best layout model is selected, achieving good performance on both simple rooms (box-type) and complex shaped rooms (with more than four walls). Experiments of the proposed approach are conducted within two public datasets, SUN360 and Stanford (2D-3D-S) demonstrating the advantages of estimating layouts by combining geometry and deep learning and the effectiveness of our proposal with respect to the state of the art.
Tasks
Published	2018-06-21
URL	http://arxiv.org/abs/1806.08294v1
PDF	http://arxiv.org/pdf/1806.08294v1.pdf
PWC	https://paperswithcode.com/paper/layouts-from-panoramic-images-with-geometry
Repo	https://github.com/cfernandezlab/Lines-and-Vanishing-Points-directly-on-Panoramas
Framework	none

Deep Feature Aggregation and Image Re-ranking with Heat Diffusion for Image Retrieval


Title	Deep Feature Aggregation and Image Re-ranking with Heat Diffusion for Image Retrieval
Authors	Shanmin Pang, Jin Ma, Jianru Xue, Jihua Zhu, Vicente Ordonez
Abstract	Image retrieval based on deep convolutional features has demonstrated state-of-the-art performance in popular benchmarks. In this paper, we present a unified solution to address deep convolutional feature aggregation and image re-ranking by simulating the dynamics of heat diffusion. A distinctive problem in image retrieval is that repetitive or \emph{bursty} features tend to dominate final image representations, resulting in representations less distinguishable. We show that by considering each deep feature as a heat source, our unsupervised aggregation method is able to avoid over-representation of \emph{bursty} features. We additionally provide a practical solution for the proposed aggregation method and further show the efficiency of our method in experimental evaluation. Inspired by the aforementioned deep feature aggregation method, we also propose a method to re-rank a number of top ranked images for a given query image by considering the query as the heat source. Finally, we extensively evaluate the proposed approach with pre-trained and fine-tuned deep networks on common public benchmarks and show superior performance compared to previous work.
Tasks	Image Retrieval
Published	2018-05-22
URL	http://arxiv.org/abs/1805.08587v5
PDF	http://arxiv.org/pdf/1805.08587v5.pdf
PWC	https://paperswithcode.com/paper/deep-feature-aggregation-and-image-re-ranking
Repo	https://github.com/MaJinWakeUp/HeWR
Framework	none

Deep Neural Networks for Survival Analysis Based on a Multi-Task Framework


Title	Deep Neural Networks for Survival Analysis Based on a Multi-Task Framework
Authors	Stephane Fotso
Abstract	Survival analysis/time-to-event models are extremely useful as they can help companies predict when a customer will buy a product, churn or default on a loan, and therefore help them improve their ROI. In this paper, we introduce a new method to calculate survival functions using the Multi-Task Logistic Regression (MTLR) model as its base and a deep learning architecture as its core. Based on the Concordance index (C-index) and Brier score, this method outperforms the MTLR in all the experiments disclosed in this paper as well as the Cox Proportional Hazard (CoxPH) model when nonlinear dependencies are found.
Tasks	Survival Analysis
Published	2018-01-17
URL	http://arxiv.org/abs/1801.05512v1
PDF	http://arxiv.org/pdf/1801.05512v1.pdf
PWC	https://paperswithcode.com/paper/deep-neural-networks-for-survival-analysis
Repo	https://github.com/havakv/pycox
Framework	pytorch

k-meansNet: When k-means Meets Differentiable Programming


Title	k-meansNet: When k-means Meets Differentiable Programming
Authors	Xi Peng, Ivor W. Tsang, Joey Tianyi Zhou, Hongyuan Zhu
Abstract	In this paper, we study two challenging problems. The first one is how to implement \textit{k}-means in the neural network, which enjoys efficient training based on the stochastic algorithm. The second one is how to enhance the interpretability of network design for clustering. To solve the problems, we propose a neural network which is a novel formulation of the vanilla $k$-means objective. Our contribution is in twofold. From the view of neural networks, the proposed \textit{k}-meansNet is with explicit interpretability in neural processing. We could understand not only why the network structure is presented like itself but also why it could perform data clustering. Such an interpretable neural network remarkably differs from the existing works that usually employ visualization technique to explain the result of the neural network. From the view of \textit{k}-means, three highly desired properties are achieved, i.e. robustness to initialization, the capability of handling new coming data, and provable convergence. Extensive experimental studies show that our method achieves promising performance comparing with 12 clustering methods on some challenging datasets.
Tasks
Published	2018-08-22
URL	http://arxiv.org/abs/1808.07292v2
PDF	http://arxiv.org/pdf/1808.07292v2.pdf
PWC	https://paperswithcode.com/paper/k-meansnet-when-k-means-meets-differentiable
Repo	https://github.com/sunze1/Differential-Programming
Framework	tf

Prediction Error Meta Classification in Semantic Segmentation: Detection via Aggregated Dispersion Measures of Softmax Probabilities


Title	Prediction Error Meta Classification in Semantic Segmentation: Detection via Aggregated Dispersion Measures of Softmax Probabilities
Authors	Matthias Rottmann, Pascal Colling, Thomas-Paul Hack, Robin Chan, Fabian Hüger, Peter Schlicht, Hanno Gottschalk
Abstract	We present a method that “meta” classifies whether seg-ments predicted by a semantic segmentation neural networkintersect with the ground truth. For this purpose, we employ measures of dispersion for predicted pixel-wise class probability distributions, like classification entropy, that yield heat maps of the input scene’s size. We aggregate these dispersion measures segment-wise and derive metrics that are well-correlated with the segment-wise IoU of prediction and ground truth. This procedure yields an almost plug and play post-processing tool to rate the prediction quality of semantic segmentation networks on segment level. This is especially relevant for monitoring neural networks in online applications like automated driving or medical imaging where reliability is of utmost importance. In our tests, we use publicly available state-of-the-art networks trained on the Cityscapes dataset and the BraTS2017 dataset and analyze the predictive power of different metrics as well as different sets of metrics. To this end, we compute logistic LASSO regression fits for the task of classifying IoU=0 vs. IoU>0 per segment and obtain AUROC values of up to 91.55%. We complement these tests with linear regression fits to predict the segment-wise IoU and obtain prediction standard deviations of down to 0.130 as well as $R^2$ values of up to 84.15%. We show that these results clearly outperform standard approaches.
Tasks	Semantic Segmentation
Published	2018-11-01
URL	https://arxiv.org/abs/1811.00648v2
PDF	https://arxiv.org/pdf/1811.00648v2.pdf
PWC	https://paperswithcode.com/paper/prediction-error-meta-classification-in
Repo	https://github.com/mrottmann/MetaSeg
Framework	tf