October 21, 2019

3022 words 15 mins read

Paper Group AWR 98

Reward Estimation for Variance Reduction in Deep Reinforcement Learning. Human-centric Indoor Scene Synthesis Using Stochastic Grammar. DeepGeo: Photo Localization with Deep Neural Network. Learning deep kernels for exponential family densities. Attention Gated Networks: Learning to Leverage Salient Regions in Medical Images. Private Machine Learni …

Reward Estimation for Variance Reduction in Deep Reinforcement Learning


Title	Reward Estimation for Variance Reduction in Deep Reinforcement Learning
Authors	Joshua Romoff, Peter Henderson, Alexandre Piché, Vincent Francois-Lavet, Joelle Pineau
Abstract	Reinforcement Learning (RL) agents require the specification of a reward signal for learning behaviours. However, introduction of corrupt or stochastic rewards can yield high variance in learning. Such corruption may be a direct result of goal misspecification, randomness in the reward signal, or correlation of the reward with external factors that are not known to the agent. Corruption or stochasticity of the reward signal can be especially problematic in robotics, where goal specification can be particularly difficult for complex tasks. While many variance reduction techniques have been studied to improve the robustness of the RL process, handling such stochastic or corrupted reward structures remains difficult. As an alternative for handling this scenario in model-free RL methods, we suggest using an estimator for both rewards and value functions. We demonstrate that this improves performance under corrupted stochastic rewards in both the tabular and non-linear function approximation settings for a variety of noise types and environments. The use of reward estimation is a robust and easy-to-implement improvement for handling corrupted reward signals in model-free RL.
Tasks
Published	2018-05-09
URL	http://arxiv.org/abs/1805.03359v2
PDF	http://arxiv.org/pdf/1805.03359v2.pdf
PWC	https://paperswithcode.com/paper/reward-estimation-for-variance-reduction-in
Repo	https://github.com/facebookresearch/reward-estimator-corl
Framework	pytorch

Human-centric Indoor Scene Synthesis Using Stochastic Grammar


Title	Human-centric Indoor Scene Synthesis Using Stochastic Grammar
Authors	Siyuan Qi, Yixin Zhu, Siyuan Huang, Chenfanfu Jiang, Song-Chun Zhu
Abstract	We present a human-centric method to sample and synthesize 3D room layouts and 2D images thereof, to obtain large-scale 2D/3D image data with perfect per-pixel ground truth. An attributed spatial And-Or graph (S-AOG) is proposed to represent indoor scenes. The S-AOG is a probabilistic grammar model, in which the terminal nodes are object entities. Human contexts as contextual relations are encoded by Markov Random Fields (MRF) on the terminal nodes. We learn the distributions from an indoor scene dataset and sample new layouts using Monte Carlo Markov Chain. Experiments demonstrate that our method can robustly sample a large variety of realistic room layouts based on three criteria: (i) visual realism comparing to a state-of-the-art room arrangement method, (ii) accuracy of the affordance maps with respect to groundtruth, and (ii) the functionality and naturalness of synthesized rooms evaluated by human subjects. The code is available at https://github.com/SiyuanQi/human-centric-scene-synthesis.
Tasks	Indoor Scene Synthesis
Published	2018-08-25
URL	http://arxiv.org/abs/1808.08473v1
PDF	http://arxiv.org/pdf/1808.08473v1.pdf
PWC	https://paperswithcode.com/paper/human-centric-indoor-scene-synthesis-using
Repo	https://github.com/SiyuanQi/human-centric-scene-synthesis
Framework	none

DeepGeo: Photo Localization with Deep Neural Network


Title	DeepGeo: Photo Localization with Deep Neural Network
Authors	Sudharshan Suresh, Nathaniel Chodosh, Montiel Abello
Abstract	In this paper we address the task of determining the geographical location of an image, a pertinent problem in learning and computer vision. This research was inspired from playing GeoGuessr, a game that tests a humans’ ability to localize themselves using just images of their surroundings. In particular, we wish to investigate how geographical, ecological and man-made features generalize for random location prediction. This is framed as a classification problem: given images sampled from the USA, the most-probable state among 50 is predicted. Previous work uses models extensively trained on large, unfiltered online datasets that are primed towards specific locations. To this end, we create (and open-source) the 50States10K dataset - with 0.5 million Google Street View images of the country. A deep neural network based on the ResNet architecture is trained, and four different strategies of incorporating low-level cardinality information are presented. This model achieves an accuracy 20 times better than chance on a test dataset, which rises to 71.87% when taking the best of top-5 guesses. The network also beats human subjects in 4 out of 5 rounds of GeoGuessr.
Tasks
Published	2018-10-07
URL	http://arxiv.org/abs/1810.03077v1
PDF	http://arxiv.org/pdf/1810.03077v1.pdf
PWC	https://paperswithcode.com/paper/deepgeo-photo-localization-with-deep-neural
Repo	https://github.com/suddhu/DeepGeo
Framework	none

Learning deep kernels for exponential family densities


Title	Learning deep kernels for exponential family densities
Authors	Li Wenliang, Dougal Sutherland, Heiko Strathmann, Arthur Gretton
Abstract	The kernel exponential family is a rich class of distributions,which can be fit efficiently and with statistical guarantees by score matching. Being required to choose a priori a simple kernel such as the Gaussian, however, limits its practical applicability. We provide a scheme for learning a kernel parameterized by a deep network, which can find complex location-dependent local features of the data geometry. This gives a very rich class of density models, capable of fitting complex structures on moderate-dimensional problems. Compared to deep density models fit via maximum likelihood, our approach provides a complementary set of strengths and tradeoffs: in empirical studies, the former can yield higher likelihoods, whereas the latter gives better estimates of the gradient of the log density, the score, which describes the distribution’s shape.
Tasks
Published	2018-11-20
URL	https://arxiv.org/abs/1811.08357v3
PDF	https://arxiv.org/pdf/1811.08357v3.pdf
PWC	https://paperswithcode.com/paper/learning-deep-kernels-for-exponential-family
Repo	https://github.com/kevin-w-li/deep-kexpfam
Framework	tf

Attention Gated Networks: Learning to Leverage Salient Regions in Medical Images


Title	Attention Gated Networks: Learning to Leverage Salient Regions in Medical Images
Authors	Jo Schlemper, Ozan Oktay, Michiel Schaap, Mattias Heinrich, Bernhard Kainz, Ben Glocker, Daniel Rueckert
Abstract	We propose a novel attention gate (AG) model for medical image analysis that automatically learns to focus on target structures of varying shapes and sizes. Models trained with AGs implicitly learn to suppress irrelevant regions in an input image while highlighting salient features useful for a specific task. This enables us to eliminate the necessity of using explicit external tissue/organ localisation modules when using convolutional neural networks (CNNs). AGs can be easily integrated into standard CNN models such as VGG or U-Net architectures with minimal computational overhead while increasing the model sensitivity and prediction accuracy. The proposed AG models are evaluated on a variety of tasks, including medical image classification and segmentation. For classification, we demonstrate the use case of AGs in scan plane detection for fetal ultrasound screening. We show that the proposed attention mechanism can provide efficient object localisation while improving the overall prediction performance by reducing false positives. For segmentation, the proposed architecture is evaluated on two large 3D CT abdominal datasets with manual annotations for multiple organs. Experimental results show that AG models consistently improve the prediction performance of the base architectures across different datasets and training sizes while preserving computational efficiency. Moreover, AGs guide the model activations to be focused around salient regions, which provides better insights into how model predictions are made. The source code for the proposed AG models is publicly available.
Tasks	Image Classification
Published	2018-08-22
URL	http://arxiv.org/abs/1808.08114v2
PDF	http://arxiv.org/pdf/1808.08114v2.pdf
PWC	https://paperswithcode.com/paper/attention-gated-networks-learning-to-leverage
Repo	https://github.com/iversonicter/Learn-to-pay-attention
Framework	pytorch

Private Machine Learning in TensorFlow using Secure Computation


Title	Private Machine Learning in TensorFlow using Secure Computation
Authors	Morten Dahl, Jason Mancuso, Yann Dupis, Ben Decoste, Morgan Giraud, Ian Livingstone, Justin Patriquin, Gavin Uhma
Abstract	We present a framework for experimenting with secure multi-party computation directly in TensorFlow. By doing so we benefit from several properties valuable to both researchers and practitioners, including tight integration with ordinary machine learning processes, existing optimizations for distributed computation in TensorFlow, high-level abstractions for expressing complex algorithms and protocols, and an expanded set of familiar tooling. We give an open source implementation of a state-of-the-art protocol and report on concrete benchmarks using typical models from private machine learning.
Tasks
Published	2018-10-18
URL	http://arxiv.org/abs/1810.08130v2
PDF	http://arxiv.org/pdf/1810.08130v2.pdf
PWC	https://paperswithcode.com/paper/private-machine-learning-in-tensorflow-using
Repo	https://github.com/anoushkrit/Knowledge
Framework	pytorch

Analyzing Human-Human Interactions: A Survey


Title	Analyzing Human-Human Interactions: A Survey
Authors	Alexandros Stergiou, Ronald Poppe
Abstract	Many videos depict people, and it is their interactions that inform us of their activities, relation to one another and the cultural and social setting. With advances in human action recognition, researchers have begun to address the automated recognition of these human-human interactions from video. The main challenges stem from dealing with the considerable variation in recording setting, the appearance of the people depicted and the coordinated performance of their interaction. This survey provides a summary of these challenges and datasets to address these, followed by an in-depth discussion of relevant vision-based recognition and detection methods. We focus on recent, promising work based on deep learning and convolutional neural networks (CNNs). Finally, we outline directions to overcome the limitations of the current state-of-the-art to analyze and, eventually, understand social human actions.
Tasks	Temporal Action Localization
Published	2018-07-31
URL	https://arxiv.org/abs/1808.00022v2
PDF	https://arxiv.org/pdf/1808.00022v2.pdf
PWC	https://paperswithcode.com/paper/understanding-human-human-interactions-a
Repo	https://github.com/alexandrosstergiou/Inception_v3_TV_Human_Interactions
Framework	tf

DeepMimic: Example-Guided Deep Reinforcement Learning of Physics-Based Character Skills


Title	DeepMimic: Example-Guided Deep Reinforcement Learning of Physics-Based Character Skills
Authors	Xue Bin Peng, Pieter Abbeel, Sergey Levine, Michiel van de Panne
Abstract	A longstanding goal in character animation is to combine data-driven specification of behavior with a system that can execute a similar behavior in a physical simulation, thus enabling realistic responses to perturbations and environmental variation. We show that well-known reinforcement learning (RL) methods can be adapted to learn robust control policies capable of imitating a broad range of example motion clips, while also learning complex recoveries, adapting to changes in morphology, and accomplishing user-specified goals. Our method handles keyframed motions, highly-dynamic actions such as motion-captured flips and spins, and retargeted motions. By combining a motion-imitation objective with a task objective, we can train characters that react intelligently in interactive settings, e.g., by walking in a desired direction or throwing a ball at a user-specified target. This approach thus combines the convenience and motion quality of using motion clips to define the desired style and appearance, with the flexibility and generality afforded by RL methods and physics-based animation. We further explore a number of methods for integrating multiple clips into the learning process to develop multi-skilled agents capable of performing a rich repertoire of diverse skills. We demonstrate results using multiple characters (human, Atlas robot, bipedal dinosaur, dragon) and a large variety of skills, including locomotion, acrobatics, and martial arts.
Tasks
Published	2018-04-08
URL	http://arxiv.org/abs/1804.02717v3
PDF	http://arxiv.org/pdf/1804.02717v3.pdf
PWC	https://paperswithcode.com/paper/deepmimic-example-guided-deep-reinforcement
Repo	https://github.com/xbpeng/DeepMimic
Framework	tf

Mixed Link Networks


Title	Mixed Link Networks
Authors	Wenhai Wang, Xiang Li, Jian Yang, Tong Lu
Abstract	Basing on the analysis by revealing the equivalence of modern networks, we find that both ResNet and DenseNet are essentially derived from the same “dense topology”, yet they only differ in the form of connection – addition (dubbed “inner link”) vs. concatenation (dubbed “outer link”). However, both two forms of connections have the superiority and insufficiency. To combine their advantages and avoid certain limitations on representation learning, we present a highly efficient and modularized Mixed Link Network (MixNet) which is equipped with flexible inner link and outer link modules. Consequently, ResNet, DenseNet and Dual Path Network (DPN) can be regarded as a special case of MixNet, respectively. Furthermore, we demonstrate that MixNets can achieve superior efficiency in parameter over the state-of-the-art architectures on many competitive datasets like CIFAR-10/100, SVHN and ImageNet.
Tasks	Representation Learning
Published	2018-02-06
URL	http://arxiv.org/abs/1802.01808v1
PDF	http://arxiv.org/pdf/1802.01808v1.pdf
PWC	https://paperswithcode.com/paper/mixed-link-networks
Repo	https://github.com/DeepInsight-PCALab/MixNet
Framework	pytorch

Robust Classification with Convolutional Prototype Learning


Title	Robust Classification with Convolutional Prototype Learning
Authors	Hong-Ming Yang, Xu-Yao Zhang, Fei Yin, Cheng-Lin Liu
Abstract	Convolutional neural networks (CNNs) have been widely used for image classification. Despite its high accuracies, CNN has been shown to be easily fooled by some adversarial examples, indicating that CNN is not robust enough for pattern classification. In this paper, we argue that the lack of robustness for CNN is caused by the softmax layer, which is a totally discriminative model and based on the assumption of closed world (i.e., with a fixed number of categories). To improve the robustness, we propose a novel learning framework called convolutional prototype learning (CPL). The advantage of using prototypes is that it can well handle the open world recognition problem and therefore improve the robustness. Under the framework of CPL, we design multiple classification criteria to train the network. Moreover, a prototype loss (PL) is proposed as a regularization to improve the intra-class compactness of the feature representation, which can be viewed as a generative model based on the Gaussian assumption of different classes. Experiments on several datasets demonstrate that CPL can achieve comparable or even better results than traditional CNN, and from the robustness perspective, CPL shows great advantages for both the rejection and incremental category learning tasks.
Tasks	Image Classification
Published	2018-05-09
URL	http://arxiv.org/abs/1805.03438v1
PDF	http://arxiv.org/pdf/1805.03438v1.pdf
PWC	https://paperswithcode.com/paper/robust-classification-with-convolutional
Repo	https://github.com/shivgahlout/Robust-Classification-with-Convolutional-Prototype-Learning-Pytorch
Framework	pytorch

Improving Knowledge Graph Embedding Using Simple Constraints


Title	Improving Knowledge Graph Embedding Using Simple Constraints
Authors	Boyang Ding, Quan Wang, Bin Wang, Li Guo
Abstract	Embedding knowledge graphs (KGs) into continuous vector spaces is a focus of current research. Early works performed this task via simple models developed over KG triples. Recent attempts focused on either designing more complicated triple scoring models, or incorporating extra information beyond triples. This paper, by contrast, investigates the potential of using very simple constraints to improve KG embedding. We examine non-negativity constraints on entity representations and approximate entailment constraints on relation representations. The former help to learn compact and interpretable representations for entities. The latter further encode regularities of logical entailment between relations into their distributed representations. These constraints impose prior beliefs upon the structure of the embedding space, without negative impacts on efficiency or scalability. Evaluation on WordNet, Freebase, and DBpedia shows that our approach is simple yet surprisingly effective, significantly and consistently outperforming competitive baselines. The constraints imposed indeed improve model interpretability, leading to a substantially increased structuring of the embedding space. Code and data are available at https://github.com/iieir-km/ComplEx-NNE_AER.
Tasks	Graph Embedding, Knowledge Graph Embedding, Knowledge Graphs
Published	2018-05-07
URL	http://arxiv.org/abs/1805.02408v2
PDF	http://arxiv.org/pdf/1805.02408v2.pdf
PWC	https://paperswithcode.com/paper/improving-knowledge-graph-embedding-using
Repo	https://github.com/iieir-km/ComplEx-NNE_AER
Framework	none

Evoplex: A platform for agent-based modeling on networks


Title	Evoplex: A platform for agent-based modeling on networks
Authors	Marcos Cardinot, Colm O’Riordan, Josephine Griffith, Matjaž Perc
Abstract	Agent-based modeling and network science have been used extensively to advance our understanding of emergent collective behavior in systems that are composed of a large number of simple interacting individuals or agents. With the increasing availability of high computational power in affordable personal computers, dedicated efforts to develop multi-threaded, scalable and easy-to-use software for agent-based simulations are needed more than ever. Evoplex meets this need by providing a fast, robust and extensible platform for developing agent-based models and multi-agent systems on networks. Each agent is represented as a node and interacts with its neighbors, as defined by the network structure. Evoplex is ideal for modeling complex systems, for example in evolutionary game theory and computational social science. In Evoplex, the models are not coupled to the execution parameters or the visualization tools, and there is a user-friendly graphical interface which makes it easy for all users, ranging from newcomers to experienced, to create, analyze, replicate and reproduce the experiments.
Tasks
Published	2018-11-25
URL	http://arxiv.org/abs/1811.10116v2
PDF	http://arxiv.org/pdf/1811.10116v2.pdf
PWC	https://paperswithcode.com/paper/evoplex-a-platform-for-agent-based-modeling
Repo	https://github.com/evoplex/model-prisonersDilemma
Framework	none

Optimization, fast and slow: optimally switching between local and Bayesian optimization


Title	Optimization, fast and slow: optimally switching between local and Bayesian optimization
Authors	Mark McLeod, Michael A. Osborne, Stephen J. Roberts
Abstract	We develop the first Bayesian Optimization algorithm, BLOSSOM, which selects between multiple alternative acquisition functions and traditional local optimization at each step. This is combined with a novel stopping condition based on expected regret. This pairing allows us to obtain the best characteristics of both local and Bayesian optimization, making efficient use of function evaluations while yielding superior convergence to the global minimum on a selection of optimization problems, and also halting optimization once a principled and intuitive stopping condition has been fulfilled.
Tasks
Published	2018-05-22
URL	http://arxiv.org/abs/1805.08610v1
PDF	http://arxiv.org/pdf/1805.08610v1.pdf
PWC	https://paperswithcode.com/paper/optimization-fast-and-slow-optimally
Repo	https://github.com/markm541374/gpbo
Framework	none

Discovering physical concepts with neural networks


Title	Discovering physical concepts with neural networks
Authors	Raban Iten, Tony Metger, Henrik Wilming, Lidia del Rio, Renato Renner
Abstract	Despite the success of neural networks at solving concrete physics problems, their use as a general-purpose tool for scientific discovery is still in its infancy. Here, we approach this problem by modelling a neural network architecture after the human physical reasoning process, which has similarities to representation learning. This allows us to make progress towards the long-term goal of machine-assisted scientific discovery from experimental data without making prior assumptions about the system. We apply this method to toy examples and show that the network finds the physically relevant parameters, exploits conservation laws to make predictions, and can help to gain conceptual insights, e.g. Copernicus’ conclusion that the solar system is heliocentric.
Tasks	Representation Learning, Time Series
Published	2018-07-26
URL	https://arxiv.org/abs/1807.10300v3
PDF	https://arxiv.org/pdf/1807.10300v3.pdf
PWC	https://paperswithcode.com/paper/discovering-physical-concepts-with-neural
Repo	https://github.com/fd17/SciNet_PyTorch
Framework	pytorch

CheMixNet: Mixed DNN Architectures for Predicting Chemical Properties using Multiple Molecular Representations


Title	CheMixNet: Mixed DNN Architectures for Predicting Chemical Properties using Multiple Molecular Representations
Authors	Arindam Paul, Dipendra Jha, Reda Al-Bahrani, Wei-keng Liao, Alok Choudhary, Ankit Agrawal
Abstract	SMILES is a linear representation of chemical structures which encodes the connection table, and the stereochemistry of a molecule as a line of text with a grammar structure denoting atoms, bonds, rings and chains, and this information can be used to predict chemical properties. Molecular fingerprints are representations of chemical structures, successfully used in similarity search, clustering, classification, drug discovery, and virtual screening and are a standard and computationally efficient abstract representation where structural features are represented as a bit string. Both SMILES and molecular fingerprints are different representations for describing the structure of a molecule. There exist several predictive models for learning chemical properties based on either SMILES or molecular fingerprints. Here, our goal is to build predictive models that can leverage both these molecular representations. In this work, we present CheMixNet – a set of neural networks for predicting chemical properties from a mixture of features learned from the two molecular representations – SMILES as sequences and molecular fingerprints as vector inputs. We demonstrate the efficacy of CheMixNet architectures by evaluating on six different datasets. The proposed CheMixNet models not only outperforms the candidate neural architectures such as contemporary fully connected networks that uses molecular fingerprints and 1-D CNN and RNN models trained SMILES sequences, but also other state-of-the-art architectures such as Chemception and Molecular Graph Convolutions.
Tasks	Drug Discovery
Published	2018-11-14
URL	http://arxiv.org/abs/1811.08283v2
PDF	http://arxiv.org/pdf/1811.08283v2.pdf
PWC	https://paperswithcode.com/paper/chemixnet-mixed-dnn-architectures-for
Repo	https://github.com/paularindam/CheMixNet
Framework	tf