July 28, 2019

2888 words 14 mins read

Paper Group ANR 351

Deeper, Broader and Artier Domain Generalization. Gaussian Process Decentralized Data Fusion Meets Transfer Learning in Large-Scale Distributed Cooperative Perception. Control Variates for Stochastic Gradient MCMC. A Vision For Continuous Automated Game Design. Action and perception for spatiotemporal patterns. Sign Language Recognition Using Tempo …

Deeper, Broader and Artier Domain Generalization


Title	Deeper, Broader and Artier Domain Generalization
Authors	Da Li, Yongxin Yang, Yi-Zhe Song, Timothy M. Hospedales
Abstract	The problem of domain generalization is to learn from multiple training domains, and extract a domain-agnostic model that can then be applied to an unseen domain. Domain generalization (DG) has a clear motivation in contexts where there are target domains with distinct characteristics, yet sparse data for training. For example recognition in sketch images, which are distinctly more abstract and rarer than photos. Nevertheless, DG methods have primarily been evaluated on photo-only benchmarks focusing on alleviating the dataset bias where both problems of domain distinctiveness and data sparsity can be minimal. We argue that these benchmarks are overly straightforward, and show that simple deep learning baselines perform surprisingly well on them. In this paper, we make two main contributions: Firstly, we build upon the favorable domain shift-robust properties of deep learning methods, and develop a low-rank parameterized CNN model for end-to-end DG learning. Secondly, we develop a DG benchmark dataset covering photo, sketch, cartoon and painting domains. This is both more practically relevant, and harder (bigger domain shift) than existing benchmarks. The results show that our method outperforms existing DG alternatives, and our dataset provides a more significant DG challenge to drive future research.
Tasks	Domain Generalization
Published	2017-10-09
URL	http://arxiv.org/abs/1710.03077v1
PDF	http://arxiv.org/pdf/1710.03077v1.pdf
PWC	https://paperswithcode.com/paper/deeper-broader-and-artier-domain
Repo
Framework

Gaussian Process Decentralized Data Fusion Meets Transfer Learning in Large-Scale Distributed Cooperative Perception


Title	Gaussian Process Decentralized Data Fusion Meets Transfer Learning in Large-Scale Distributed Cooperative Perception
Authors	Ruofei Ouyang, Kian Hsiang Low
Abstract	This paper presents novel Gaussian process decentralized data fusion algorithms exploiting the notion of agent-centric support sets for distributed cooperative perception of large-scale environmental phenomena. To overcome the limitations of scale in existing works, our proposed algorithms allow every mobile sensing agent to choose a different support set and dynamically switch to another during execution for encapsulating its own data into a local summary that, perhaps surprisingly, can still be assimilated with the other agents’ local summaries (i.e., based on their current choices of support sets) into a globally consistent summary to be used for predicting the phenomenon. To achieve this, we propose a novel transfer learning mechanism for a team of agents capable of sharing and transferring information encapsulated in a summary based on a support set to that utilizing a different support set with some loss that can be theoretically bounded and analyzed. To alleviate the issue of information loss accumulating over multiple instances of transfer learning, we propose a new information sharing mechanism to be incorporated into our algorithms in order to achieve memory-efficient lazy transfer learning. Empirical evaluation on real-world datasets show that our algorithms outperform the state-of-the-art methods.
Tasks	Transfer Learning
Published	2017-11-16
URL	http://arxiv.org/abs/1711.06064v1
PDF	http://arxiv.org/pdf/1711.06064v1.pdf
PWC	https://paperswithcode.com/paper/gaussian-process-decentralized-data-fusion
Repo
Framework

Control Variates for Stochastic Gradient MCMC


Title	Control Variates for Stochastic Gradient MCMC
Authors	Jack Baker, Paul Fearnhead, Emily B. Fox, Christopher Nemeth
Abstract	It is well known that Markov chain Monte Carlo (MCMC) methods scale poorly with dataset size. A popular class of methods for solving this issue is stochastic gradient MCMC. These methods use a noisy estimate of the gradient of the log posterior, which reduces the per iteration computational cost of the algorithm. Despite this, there are a number of results suggesting that stochastic gradient Langevin dynamics (SGLD), probably the most popular of these methods, still has computational cost proportional to the dataset size. We suggest an alternative log posterior gradient estimate for stochastic gradient MCMC, which uses control variates to reduce the variance. We analyse SGLD using this gradient estimate, and show that, under log-concavity assumptions on the target distribution, the computational cost required for a given level of accuracy is independent of the dataset size. Next we show that a different control variate technique, known as zero variance control variates can be applied to SGMCMC algorithms for free. This post-processing step improves the inference of the algorithm by reducing the variance of the MCMC output. Zero variance control variates rely on the gradient of the log posterior; we explore how the variance reduction is affected by replacing this with the noisy gradient estimate calculated by SGMCMC.
Tasks
Published	2017-06-16
URL	http://arxiv.org/abs/1706.05439v2
PDF	http://arxiv.org/pdf/1706.05439v2.pdf
PWC	https://paperswithcode.com/paper/control-variates-for-stochastic-gradient-mcmc
Repo
Framework

A Vision For Continuous Automated Game Design


Title	A Vision For Continuous Automated Game Design
Authors	Michael Cook
Abstract	ANGELINA is an automated game design system which has previously been built as a single software block which designs games from start to finish. In this paper we outline a roadmap for the development of a new version of ANGELINA, designed to iterate on games in different ways to produce a continuous creative process that will improve the quality of its work, but more importantly improve the perception of the software as being an independently creative piece of software. We provide an initial report of the system’s structure here as well as results from the first working module of the system.
Tasks
Published	2017-07-30
URL	http://arxiv.org/abs/1707.09661v1
PDF	http://arxiv.org/pdf/1707.09661v1.pdf
PWC	https://paperswithcode.com/paper/a-vision-for-continuous-automated-game-design
Repo
Framework

Action and perception for spatiotemporal patterns


Title	Action and perception for spatiotemporal patterns
Authors	Martin Biehl, Daniel Polani
Abstract	This is a contribution to the formalization of the concept of agents in multivariate Markov chains. Agents are commonly defined as entities that act, perceive, and are goal-directed. In a multivariate Markov chain (e.g. a cellular automaton) the transition matrix completely determines the dynamics. This seems to contradict the possibility of acting entities within such a system. Here we present definitions of actions and perceptions within multivariate Markov chains based on entity-sets. Entity-sets represent a largely independent choice of a set of spatiotemporal patterns that are considered as all the entities within the Markov chain. For example, the entity-set can be chosen according to operational closure conditions or complete specific integration. Importantly, the perception-action loop also induces an entity-set and is a multivariate Markov chain. We then show that our definition of actions leads to non-heteronomy and that of perceptions specialize to the usual concept of perception in the perception-action loop.
Tasks
Published	2017-06-12
URL	http://arxiv.org/abs/1706.03576v1
PDF	http://arxiv.org/pdf/1706.03576v1.pdf
PWC	https://paperswithcode.com/paper/action-and-perception-for-spatiotemporal
Repo
Framework

Sign Language Recognition Using Temporal Classification


Title	Sign Language Recognition Using Temporal Classification
Authors	Hardie Cate, Fahim Dalvi, Zeshan Hussain
Abstract	Devices like the Myo armband available in the market today enable us to collect data about the position of a user’s hands and fingers over time. We can use these technologies for sign language translation since each sign is roughly a combination of gestures across time. In this work, we utilize a dataset collected by a group at the University of South Wales, which contains parameters, such as hand position, hand rotation, and finger bend, for 95 unique signs. For each input stream representing a sign, we predict which sign class this stream falls into. We begin by implementing baseline SVM and logistic regression models, which perform reasonably well on high quality data. Lower quality data requires a more sophisticated approach, so we explore different methods in temporal classification, including long short term memory architectures and sequential pattern mining methods.
Tasks	Sequential Pattern Mining, Sign Language Recognition, Sign Language Translation
Published	2017-01-07
URL	http://arxiv.org/abs/1701.01875v1
PDF	http://arxiv.org/pdf/1701.01875v1.pdf
PWC	https://paperswithcode.com/paper/sign-language-recognition-using-temporal
Repo
Framework

Triangle Generative Adversarial Networks


Title	Triangle Generative Adversarial Networks
Authors	Zhe Gan, Liqun Chen, Weiyao Wang, Yunchen Pu, Yizhe Zhang, Hao Liu, Chunyuan Li, Lawrence Carin
Abstract	A Triangle Generative Adversarial Network ($\Delta$-GAN) is developed for semi-supervised cross-domain joint distribution matching, where the training data consists of samples from each domain, and supervision of domain correspondence is provided by only a few paired samples. $\Delta$-GAN consists of four neural networks, two generators and two discriminators. The generators are designed to learn the two-way conditional distributions between the two domains, while the discriminators implicitly define a ternary discriminative function, which is trained to distinguish real data pairs and two kinds of fake data pairs. The generators and discriminators are trained together using adversarial learning. Under mild assumptions, in theory the joint distributions characterized by the two generators concentrate to the data distribution. In experiments, three different kinds of domain pairs are considered, image-label, image-image and image-attribute pairs. Experiments on semi-supervised image classification, image-to-image translation and attribute-based image generation demonstrate the superiority of the proposed approach.
Tasks	Image Classification, Image Generation, Image-to-Image Translation, Semi-Supervised Image Classification
Published	2017-09-19
URL	http://arxiv.org/abs/1709.06548v2
PDF	http://arxiv.org/pdf/1709.06548v2.pdf
PWC	https://paperswithcode.com/paper/triangle-generative-adversarial-networks
Repo
Framework

Empirical Analysis of the Hessian of Over-Parametrized Neural Networks


Title	Empirical Analysis of the Hessian of Over-Parametrized Neural Networks
Authors	Levent Sagun, Utku Evci, V. Ugur Guney, Yann Dauphin, Leon Bottou
Abstract	We study the properties of common loss surfaces through their Hessian matrix. In particular, in the context of deep learning, we empirically show that the spectrum of the Hessian is composed of two parts: (1) the bulk centered near zero, (2) and outliers away from the bulk. We present numerical evidence and mathematical justifications to the following conjectures laid out by Sagun et al. (2016): Fixing data, increasing the number of parameters merely scales the bulk of the spectrum; fixing the dimension and changing the data (for instance adding more clusters or making the data less separable) only affects the outliers. We believe that our observations have striking implications for non-convex optimization in high dimensions. First, the flatness of such landscapes (which can be measured by the singularity of the Hessian) implies that classical notions of basins of attraction may be quite misleading. And that the discussion of wide/narrow basins may be in need of a new perspective around over-parametrization and redundancy that are able to create large connected components at the bottom of the landscape. Second, the dependence of small number of large eigenvalues to the data distribution can be linked to the spectrum of the covariance matrix of gradients of model outputs. With this in mind, we may reevaluate the connections within the data-architecture-algorithm framework of a model, hoping that it would shed light into the geometry of high-dimensional and non-convex spaces in modern applications. In particular, we present a case that links the two observations: small and large batch gradient descent appear to converge to different basins of attraction but we show that they are in fact connected through their flat region and so belong to the same basin.
Tasks
Published	2017-06-14
URL	http://arxiv.org/abs/1706.04454v3
PDF	http://arxiv.org/pdf/1706.04454v3.pdf
PWC	https://paperswithcode.com/paper/empirical-analysis-of-the-hessian-of-over
Repo
Framework

Efficient Convolutional Neural Network For Audio Event Detection


Title	Efficient Convolutional Neural Network For Audio Event Detection
Authors	Matthias Meyer, Lukas Cavigelli, Lothar Thiele
Abstract	Wireless distributed systems as used in sensor networks, Internet-of-Things and cyber-physical systems, impose high requirements on resource efficiency. Advanced preprocessing and classification of data at the network edge can help to decrease the communication demand and to reduce the amount of data to be processed centrally. In the area of distributed acoustic sensing, the combination of algorithms with a high classification rate and resource-constraint embedded systems is essential. Unfortunately, algorithms for acoustic event detection have a high memory and computational demand and are not suited for execution at the network edge. This paper addresses these aspects by applying structural optimizations to a convolutional neural network for audio event detection to reduce the memory requirement by a factor of more than 500 and the computational effort by a factor of 2.1 while performing 9.2% better.
Tasks
Published	2017-09-28
URL	http://arxiv.org/abs/1709.09888v1
PDF	http://arxiv.org/pdf/1709.09888v1.pdf
PWC	https://paperswithcode.com/paper/efficient-convolutional-neural-network-for
Repo
Framework

Dynamic Mortality Risk Predictions in Pediatric Critical Care Using Recurrent Neural Networks


Title	Dynamic Mortality Risk Predictions in Pediatric Critical Care Using Recurrent Neural Networks
Authors	M Aczon, D Ledbetter, L Ho, A Gunny, A Flynn, J Williams, R Wetzel
Abstract	Viewing the trajectory of a patient as a dynamical system, a recurrent neural network was developed to learn the course of patient encounters in the Pediatric Intensive Care Unit (PICU) of a major tertiary care center. Data extracted from Electronic Medical Records (EMR) of about 12000 patients who were admitted to the PICU over a period of more than 10 years were leveraged. The RNN model ingests a sequence of measurements which include physiologic observations, laboratory results, administered drugs and interventions, and generates temporally dynamic predictions for in-ICU mortality at user-specified times. The RNN’s ICU mortality predictions offer significant improvements over those from two clinically-used scores and static machine learning algorithms.
Tasks
Published	2017-01-23
URL	http://arxiv.org/abs/1701.06675v1
PDF	http://arxiv.org/pdf/1701.06675v1.pdf
PWC	https://paperswithcode.com/paper/dynamic-mortality-risk-predictions-in
Repo
Framework

ConvNet Architecture Search for Spatiotemporal Feature Learning


Title	ConvNet Architecture Search for Spatiotemporal Feature Learning
Authors	Du Tran, Jamie Ray, Zheng Shou, Shih-Fu Chang, Manohar Paluri
Abstract	Learning image representations with ConvNets by pre-training on ImageNet has proven useful across many visual understanding tasks including object detection, semantic segmentation, and image captioning. Although any image representation can be applied to video frames, a dedicated spatiotemporal representation is still vital in order to incorporate motion patterns that cannot be captured by appearance based models alone. This paper presents an empirical ConvNet architecture search for spatiotemporal feature learning, culminating in a deep 3-dimensional (3D) Residual ConvNet. Our proposed architecture outperforms C3D by a good margin on Sports-1M, UCF101, HMDB51, THUMOS14, and ASLAN while being 2 times faster at inference time, 2 times smaller in model size, and having a more compact representation.
Tasks	Image Captioning, Neural Architecture Search, Object Detection, Semantic Segmentation
Published	2017-08-16
URL	http://arxiv.org/abs/1708.05038v1
PDF	http://arxiv.org/pdf/1708.05038v1.pdf
PWC	https://paperswithcode.com/paper/convnet-architecture-search-for
Repo
Framework

Balancing Selection Pressures, Multiple Objectives, and Neural Modularity to Coevolve Cooperative Agent Behavior


Title	Balancing Selection Pressures, Multiple Objectives, and Neural Modularity to Coevolve Cooperative Agent Behavior
Authors	Alex C. Rollins, Jacob Schrum
Abstract	Previous research using evolutionary computation in Multi-Agent Systems indicates that assigning fitness based on team vs.\ individual behavior has a strong impact on the ability of evolved teams of artificial agents to exhibit teamwork in challenging tasks. However, such research only made use of single-objective evolution. In contrast, when a multiobjective evolutionary algorithm is used, populations can be subject to individual-level objectives, team-level objectives, or combinations of the two. This paper explores the performance of cooperatively coevolved teams of agents controlled by artificial neural networks subject to these types of objectives. Specifically, predator agents are evolved to capture scripted prey agents in a torus-shaped grid world. Because of the tension between individual and team behaviors, multiple modes of behavior can be useful, and thus the effect of modular neural networks is also explored. Results demonstrate that fitness rewarding individual behavior is superior to fitness rewarding team behavior, despite being applied to a cooperative task. However, the use of networks with multiple modules allows predators to discover intelligent behavior, regardless of which type of objectives are used.
Tasks
Published	2017-03-24
URL	http://arxiv.org/abs/1703.08577v1
PDF	http://arxiv.org/pdf/1703.08577v1.pdf
PWC	https://paperswithcode.com/paper/balancing-selection-pressures-multiple
Repo
Framework

Deep Lattice Networks and Partial Monotonic Functions


Title	Deep Lattice Networks and Partial Monotonic Functions
Authors	Seungil You, David Ding, Kevin Canini, Jan Pfeifer, Maya Gupta
Abstract	We propose learning deep models that are monotonic with respect to a user-specified set of inputs by alternating layers of linear embeddings, ensembles of lattices, and calibrators (piecewise linear functions), with appropriate constraints for monotonicity, and jointly training the resulting network. We implement the layers and projections with new computational graph nodes in TensorFlow and use the ADAM optimizer and batched stochastic gradients. Experiments on benchmark and real-world datasets show that six-layer monotonic deep lattice networks achieve state-of-the art performance for classification and regression with monotonicity guarantees.
Tasks
Published	2017-09-19
URL	http://arxiv.org/abs/1709.06680v1
PDF	http://arxiv.org/pdf/1709.06680v1.pdf
PWC	https://paperswithcode.com/paper/deep-lattice-networks-and-partial-monotonic
Repo
Framework

Systematic Testing of Convolutional Neural Networks for Autonomous Driving


Title	Systematic Testing of Convolutional Neural Networks for Autonomous Driving
Authors	Tommaso Dreossi, Shromona Ghosh, Alberto Sangiovanni-Vincentelli, Sanjit A. Seshia
Abstract	We present a framework to systematically analyze convolutional neural networks (CNNs) used in classification of cars in autonomous vehicles. Our analysis procedure comprises an image generator that produces synthetic pictures by sampling in a lower dimension image modification subspace and a suite of visualization tools. The image generator produces images which can be used to test the CNN and hence expose its vulnerabilities. The presented framework can be used to extract insights of the CNN classifier, compare across classification models, or generate training and validation datasets.
Tasks	Autonomous Driving, Autonomous Vehicles
Published	2017-08-10
URL	http://arxiv.org/abs/1708.03309v2
PDF	http://arxiv.org/pdf/1708.03309v2.pdf
PWC	https://paperswithcode.com/paper/systematic-testing-of-convolutional-neural
Repo
Framework

A probabilistic data-driven model for planar pushing


Title	A probabilistic data-driven model for planar pushing
Authors	Maria Bauza, Alberto Rodriguez
Abstract	This paper presents a data-driven approach to model planar pushing interaction to predict both the most likely outcome of a push and its expected variability. The learned models rely on a variation of Gaussian processes with input-dependent noise called Variational Heteroscedastic Gaussian processes (VHGP) that capture the mean and variance of a stochastic function. We show that we can learn accurate models that outperform analytical models after less than 100 samples and saturate in performance with less than 1000 samples. We validate the results against a collected dataset of repeated trajectories, and use the learned models to study questions such as the nature of the variability in pushing, and the validity of the quasi-static assumption.
Tasks	Gaussian Processes
Published	2017-04-10
URL	http://arxiv.org/abs/1704.03033v2
PDF	http://arxiv.org/pdf/1704.03033v2.pdf
PWC	https://paperswithcode.com/paper/a-probabilistic-data-driven-model-for-planar
Repo
Framework