October 20, 2019

2898 words 14 mins read

Paper Group AWR 326

Non-bifurcating phylogenetic tree inference via the adaptive LASSO. A Novel Learning-based Global Path Planning Algorithm for Planetary Rovers. Mining within-trial oscillatory brain dynamics to address the variability of optimized spatial filters. Towards Efficient Large-Scale Graph Neural Network Computing. Beamformed Fingerprint Learning for Accu …

Non-bifurcating phylogenetic tree inference via the adaptive LASSO


Title	Non-bifurcating phylogenetic tree inference via the adaptive LASSO
Authors	Cheng Zhang, Vu Dinh, Frederick A. Matsen IV
Abstract	Phylogenetic tree inference using deep DNA sequencing is reshaping our understanding of rapidly evolving systems, such as the within-host battle between viruses and the immune system. Densely sampled phylogenetic trees can contain special features, including “sampled ancestors” in which we sequence a genotype along with its direct descendants, and “polytomies” in which multiple descendants arise simultaneously. These features are apparent after identifying zero-length branches in the tree. However, current maximum-likelihood based approaches are not capable of revealing such zero-length branches. In this paper, we find these zero-length branches by introducing adaptive-LASSO-type regularization estimators to phylogenetics, deriving their properties, and showing regularization to be a practically useful approach for phylogenetics.
Tasks
Published	2018-05-28
URL	http://arxiv.org/abs/1805.11073v1
PDF	http://arxiv.org/pdf/1805.11073v1.pdf
PWC	https://paperswithcode.com/paper/non-bifurcating-phylogenetic-tree-inference
Repo	https://github.com/matsengrp/adaLASSO-phylo
Framework	none

A Novel Learning-based Global Path Planning Algorithm for Planetary Rovers


Title	A Novel Learning-based Global Path Planning Algorithm for Planetary Rovers
Authors	Jiang Zhang, Yuanqing Xia, Ganghui Shen
Abstract	Autonomous path planning algorithms are significant to planetary exploration rovers, since relying on commands from Earth will heavily reduce their efficiency of executing exploration missions. This paper proposes a novel learning-based algorithm to deal with global path planning problem for planetary exploration rovers. Specifically, a novel deep convolutional neural network with double branches (DB-CNN) is designed and trained, which can plan path directly from orbital images of planetary surfaces without implementing environment mapping. Moreover, the planning procedure requires no prior knowledge about planetary surface terrains. Finally, experimental results demonstrate that DB-CNN achieves better performance on global path planning and faster convergence during training compared with the existing Value Iteration Network (VIN).
Tasks
Published	2018-11-23
URL	http://arxiv.org/abs/1811.10437v1
PDF	http://arxiv.org/pdf/1811.10437v1.pdf
PWC	https://paperswithcode.com/paper/a-novel-learning-based-global-path-planning
Repo	https://github.com/bitzj2015/DB-CNN
Framework	tf

Mining within-trial oscillatory brain dynamics to address the variability of optimized spatial filters


Title	Mining within-trial oscillatory brain dynamics to address the variability of optimized spatial filters
Authors	Andreas Meinel, Henrich Kolkhorst, Michael Tangermann
Abstract	Data-driven spatial filtering algorithms optimize scores such as the contrast between two conditions to extract oscillatory brain signal components. Most machine learning approaches for filter estimation, however, disregard within-trial temporal dynamics and are extremely sensitive to changes in training data and involved hyperparameters. This leads to highly variable solutions and impedes the selection of a suitable candidate for, e.g.,~neurotechnological applications. Fostering component introspection, we propose to embrace this variability by condensing the functional signatures of a large set of oscillatory components into homogeneous clusters, each representing specific within-trial envelope dynamics. The proposed method is exemplified by and evaluated on a complex hand force task with a rich within-trial structure. Based on electroencephalography data of 18 healthy subjects, we found that the components’ distinct temporal envelope dynamics are highly subject-specific. On average, we obtained seven clusters per subject, which were strictly confined regarding their underlying frequency bands. As the analysis method is not limited to a specific spatial filtering algorithm, it could be utilized for a wide range of neurotechnological applications, e.g., to select and monitor functionally relevant features for brain-computer interface protocols in stroke rehabilitation.
Tasks
Published	2018-04-27
URL	http://arxiv.org/abs/1804.10454v2
PDF	http://arxiv.org/pdf/1804.10454v2.pdf
PWC	https://paperswithcode.com/paper/mining-within-trial-oscillatory-brain
Repo	https://github.com/bsdlab/func_mining
Framework	none

Towards Efficient Large-Scale Graph Neural Network Computing


Title	Towards Efficient Large-Scale Graph Neural Network Computing
Authors	Lingxiao Ma, Zhi Yang, Youshan Miao, Jilong Xue, Ming Wu, Lidong Zhou, Yafei Dai
Abstract	Recent deep learning models have moved beyond low-dimensional regular grids such as image, video, and speech, to high-dimensional graph-structured data, such as social networks, brain connections, and knowledge graphs. This evolution has led to large graph-based irregular and sparse models that go beyond what existing deep learning frameworks are designed for. Further, these models are not easily amenable to efficient, at scale, acceleration on parallel hardwares (e.g. GPUs). We introduce NGra, the first parallel processing framework for graph-based deep neural networks (GNNs). NGra presents a new SAGA-NN model for expressing deep neural networks as vertex programs with each layer in well-defined (Scatter, ApplyEdge, Gather, ApplyVertex) graph operation stages. This model not only allows GNNs to be expressed intuitively, but also facilitates the mapping to an efficient dataflow representation. NGra addresses the scalability challenge transparently through automatic graph partitioning and chunk-based stream processing out of GPU core or over multiple GPUs, which carefully considers data locality, data movement, and overlapping of parallel processing and data movement. NGra further achieves efficiency through highly optimized Scatter/Gather operators on GPUs despite its sparsity. Our evaluation shows that NGra scales to large real graphs that none of the existing frameworks can handle directly, while achieving up to about 4 times speedup even at small scales over the multiple-baseline design on TensorFlow.
Tasks	graph partitioning, Knowledge Graphs
Published	2018-10-19
URL	http://arxiv.org/abs/1810.08403v1
PDF	http://arxiv.org/pdf/1810.08403v1.pdf
PWC	https://paperswithcode.com/paper/towards-efficient-large-scale-graph-neural
Repo	https://github.com/xchadesi/GraphNeuralNetwork
Framework	pytorch

Beamformed Fingerprint Learning for Accurate Millimeter Wave Positioning


Title	Beamformed Fingerprint Learning for Accurate Millimeter Wave Positioning
Authors	João Gante, Gabriel Falcão, Leonel Sousa
Abstract	With millimeter wave wireless communications, the resulting radiation reflects on most visible objects, creating rich multipath environments, namely in urban scenarios. The radiation captured by a listening device is thus shaped by the obstacles encountered, which carry latent information regarding their relative positions. In this paper, a system to convert the received millimeter wave radiation into the device’s position is proposed, making use of the aforementioned hidden information. Using deep learning techniques and a pre-established codebook of beamforming patterns transmitted by a base station, the simulations show that average estimation errors below 10 meters are achievable in realistic outdoors scenarios that contain mostly non-line-of-sight positions, paving the way for new positioning systems.
Tasks	Outdoor Positioning
Published	2018-04-11
URL	http://arxiv.org/abs/1804.04112v1
PDF	http://arxiv.org/pdf/1804.04112v1.pdf
PWC	https://paperswithcode.com/paper/beamformed-fingerprint-learning-for-accurate
Repo	https://github.com/gante/mmWave-localization-learning
Framework	tf

Neural Tangent Kernel: Convergence and Generalization in Neural Networks


Title	Neural Tangent Kernel: Convergence and Generalization in Neural Networks
Authors	Arthur Jacot, Franck Gabriel, Clément Hongler
Abstract	At initialization, artificial neural networks (ANNs) are equivalent to Gaussian processes in the infinite-width limit, thus connecting them to kernel methods. We prove that the evolution of an ANN during training can also be described by a kernel: during gradient descent on the parameters of an ANN, the network function $f_\theta$ (which maps input vectors to output vectors) follows the kernel gradient of the functional cost (which is convex, in contrast to the parameter cost) w.r.t. a new kernel: the Neural Tangent Kernel (NTK). This kernel is central to describe the generalization features of ANNs. While the NTK is random at initialization and varies during training, in the infinite-width limit it converges to an explicit limiting kernel and it stays constant during training. This makes it possible to study the training of ANNs in function space instead of parameter space. Convergence of the training can then be related to the positive-definiteness of the limiting NTK. We prove the positive-definiteness of the limiting NTK when the data is supported on the sphere and the non-linearity is non-polynomial. We then focus on the setting of least-squares regression and show that in the infinite-width limit, the network function $f_\theta$ follows a linear differential equation during training. The convergence is fastest along the largest kernel principal components of the input data with respect to the NTK, hence suggesting a theoretical motivation for early stopping. Finally we study the NTK numerically, observe its behavior for wide networks, and compare it to the infinite-width limit.
Tasks	Gaussian Processes
Published	2018-06-20
URL	https://arxiv.org/abs/1806.07572v4
PDF	https://arxiv.org/pdf/1806.07572v4.pdf
PWC	https://paperswithcode.com/paper/neural-tangent-kernel-convergence-and
Repo	https://github.com/thegregyang/NNspectra
Framework	pytorch

Model-Based Active Exploration


Title	Model-Based Active Exploration
Authors	Pranav Shyam, Wojciech Jaśkowski, Faustino Gomez
Abstract	Efficient exploration is an unsolved problem in Reinforcement Learning which is usually addressed by reactively rewarding the agent for fortuitously encountering novel situations. This paper introduces an efficient active exploration algorithm, Model-Based Active eXploration (MAX), which uses an ensemble of forward models to plan to observe novel events. This is carried out by optimizing agent behaviour with respect to a measure of novelty derived from the Bayesian perspective of exploration, which is estimated using the disagreement between the futures predicted by the ensemble members. We show empirically that in semi-random discrete environments where directed exploration is critical to make progress, MAX is at least an order of magnitude more efficient than strong baselines. MAX scales to high-dimensional continuous environments where it builds task-agnostic models that can be used for any downstream task.
Tasks	Efficient Exploration
Published	2018-10-29
URL	https://arxiv.org/abs/1810.12162v5
PDF	https://arxiv.org/pdf/1810.12162v5.pdf
PWC	https://paperswithcode.com/paper/model-based-active-exploration
Repo	https://github.com/nnaisense/max
Framework	pytorch

Differentiable Compositional Kernel Learning for Gaussian Processes


Title	Differentiable Compositional Kernel Learning for Gaussian Processes
Authors	Shengyang Sun, Guodong Zhang, Chaoqi Wang, Wenyuan Zeng, Jiaman Li, Roger Grosse
Abstract	The generalization properties of Gaussian processes depend heavily on the choice of kernel, and this choice remains a dark art. We present the Neural Kernel Network (NKN), a flexible family of kernels represented by a neural network. The NKN architecture is based on the composition rules for kernels, so that each unit of the network corresponds to a valid kernel. It can compactly approximate compositional kernel structures such as those used by the Automatic Statistician (Lloyd et al., 2014), but because the architecture is differentiable, it is end-to-end trainable with gradient-based optimization. We show that the NKN is universal for the class of stationary kernels. Empirically we demonstrate pattern discovery and extrapolation abilities of NKN on several tasks that depend crucially on identifying the underlying structure, including time series and texture extrapolation, as well as Bayesian optimization.
Tasks	Gaussian Processes, Time Series
Published	2018-06-12
URL	http://arxiv.org/abs/1806.04326v3
PDF	http://arxiv.org/pdf/1806.04326v3.pdf
PWC	https://paperswithcode.com/paper/differentiable-compositional-kernel-learning
Repo	https://github.com/thjashin/spectral-stein-grad
Framework	tf

Variational Implicit Processes


Title	Variational Implicit Processes
Authors	Chao Ma, Yingzhen Li, José Miguel Hernández-Lobato
Abstract	We introduce the implicit processes (IPs), a stochastic process that places implicitly defined multivariate distributions over any finite collections of random variables. IPs are therefore highly flexible implicit priors over functions, with examples including data simulators, Bayesian neural networks and non-linear transformations of stochastic processes. A novel and efficient approximate inference algorithm for IPs, namely the variational implicit processes (VIPs), is derived using generalised wake-sleep updates. This method returns simple update equations and allows scalable hyper-parameter learning with stochastic optimization. Experiments show that VIPs return better uncertainty estimates and lower errors over existing inference methods for challenging models such as Bayesian neural networks, and Gaussian processes.
Tasks	Gaussian Processes, Stochastic Optimization
Published	2018-06-06
URL	https://arxiv.org/abs/1806.02390v2
PDF	https://arxiv.org/pdf/1806.02390v2.pdf
PWC	https://paperswithcode.com/paper/variational-implicit-processes
Repo	https://github.com/LaurantChao/VIP
Framework	tf

Deep Mixed Effect Model using Gaussian Processes: A Personalized and Reliable Prediction for Healthcare


Title	Deep Mixed Effect Model using Gaussian Processes: A Personalized and Reliable Prediction for Healthcare
Authors	Ingyo Chung, Saehoon Kim, Juho Lee, Kwang Joon Kim, Sung Ju Hwang, Eunho Yang
Abstract	We present a personalized and reliable prediction model for healthcare, which can provide individually tailored medical services such as diagnosis, disease treatment, and prevention. Our proposed framework targets at making personalized and reliable predictions from time-series data, such as Electronic Health Records (EHR), by modeling two complementary components: i) a shared component that captures global trend across diverse patients and ii) a patient-specific component that models idiosyncratic variability for each patient. To this end, we propose a composite model of a deep neural network to learn complex global trends from the large number of patients, and Gaussian Processes (GP) to probabilistically model individual time-series given relatively small number of visits per patient. We evaluate our model on diverse and heterogeneous tasks from EHR datasets and show practical advantages over standard time-series deep models such as pure Recurrent Neural Network (RNN).
Tasks	Gaussian Processes, Time Series
Published	2018-06-05
URL	https://arxiv.org/abs/1806.01551v3
PDF	https://arxiv.org/pdf/1806.01551v3.pdf
PWC	https://paperswithcode.com/paper/mixed-effect-composite-rnn-gp-a-personalized
Repo	https://github.com/OpenXAIProject/Mixed-Effect-Composite-RNN-Gaussian-Process
Framework	tf

Minimal I-MAP MCMC for Scalable Structure Discovery in Causal DAG Models


Title	Minimal I-MAP MCMC for Scalable Structure Discovery in Causal DAG Models
Authors	Raj Agrawal, Tamara Broderick, Caroline Uhler
Abstract	Learning a Bayesian network (BN) from data can be useful for decision-making or discovering causal relationships. However, traditional methods often fail in modern applications, which exhibit a larger number of observed variables than data points. The resulting uncertainty about the underlying network as well as the desire to incorporate prior information recommend a Bayesian approach to learning the BN, but the highly combinatorial structure of BNs poses a striking challenge for inference. The current state-of-the-art methods such as order MCMC are faster than previous methods but prevent the use of many natural structural priors and still have running time exponential in the maximum indegree of the true directed acyclic graph (DAG) of the BN. We here propose an alternative posterior approximation based on the observation that, if we incorporate empirical conditional independence tests, we can focus on a high-probability DAG associated with each order of the vertices. We show that our method allows the desired flexibility in prior specification, removes timing dependence on the maximum indegree and yields provably good posterior approximations; in addition, we show that it achieves superior accuracy, scalability, and sampler mixing on several datasets.
Tasks	Decision Making
Published	2018-03-15
URL	http://arxiv.org/abs/1803.05554v3
PDF	http://arxiv.org/pdf/1803.05554v3.pdf
PWC	https://paperswithcode.com/paper/minimal-i-map-mcmc-for-scalable-structure
Repo	https://github.com/miraep8/Minimal_IMAP_MCMC
Framework	none

High Quality Monocular Depth Estimation via Transfer Learning


Title	High Quality Monocular Depth Estimation via Transfer Learning
Authors	Ibraheem Alhashim, Peter Wonka
Abstract	Accurate depth estimation from images is a fundamental task in many applications including scene understanding and reconstruction. Existing solutions for depth estimation often produce blurry approximations of low resolution. This paper presents a convolutional neural network for computing a high-resolution depth map given a single RGB image with the help of transfer learning. Following a standard encoder-decoder architecture, we leverage features extracted using high performing pre-trained networks when initializing our encoder along with augmentation and training strategies that lead to more accurate results. We show how, even for a very simple decoder, our method is able to achieve detailed high-resolution depth maps. Our network, with fewer parameters and training iterations, outperforms state-of-the-art on two datasets and also produces qualitatively better results that capture object boundaries more faithfully. Code and corresponding pre-trained weights are made publicly available.
Tasks	Depth Estimation, Monocular Depth Estimation, Transfer Learning
Published	2018-12-31
URL	http://arxiv.org/abs/1812.11941v2
PDF	http://arxiv.org/pdf/1812.11941v2.pdf
PWC	https://paperswithcode.com/paper/high-quality-monocular-depth-estimation-via
Repo	https://github.com/alinstein/Depth_estimation
Framework	pytorch


Title	modAL: A modular active learning framework for Python
Authors	Tivadar Danka, Peter Horvath
Abstract	modAL is a modular active learning framework for Python, aimed to make active learning research and practice simpler. Its distinguishing features are (i) clear and modular object oriented design (ii) full compatibility with scikit-learn models and workflows. These features make fast prototyping and easy extensibility possible, aiding the development of real-life active learning pipelines and novel algorithms as well. modAL is fully open source, hosted on GitHub at https://github.com/cosmic-cortex/modAL. To assure code quality, extensive unit tests are provided and continuous integration is applied. In addition, a detailed documentation with several tutorials are also available for ease of use. The framework is available in PyPI and distributed under the MIT license.
Tasks	Active Learning
Published	2018-05-02
URL	http://arxiv.org/abs/1805.00979v2
PDF	http://arxiv.org/pdf/1805.00979v2.pdf
PWC	https://paperswithcode.com/paper/modal-a-modular-active-learning-framework-for
Repo	https://github.com/modAL-python/modAL
Framework	none

Using State Predictions for Value Regularization in Curiosity Driven Deep Reinforcement Learning


Title	Using State Predictions for Value Regularization in Curiosity Driven Deep Reinforcement Learning
Authors	Gino Brunner, Manuel Fritsche, Oliver Richter, Roger Wattenhofer
Abstract	Learning in sparse reward settings remains a challenge in Reinforcement Learning, which is often addressed by using intrinsic rewards. One promising strategy is inspired by human curiosity, requiring the agent to learn to predict the future. In this paper a curiosity-driven agent is extended to use these predictions directly for training. To achieve this, the agent predicts the value function of the next state at any point in time. Subsequently, the consistency of this prediction with the current value function is measured, which is then used as a regularization term in the loss function of the algorithm. Experiments were made on grid-world environments as well as on a 3D navigation task, both with sparse rewards. In the first case the extended agent is able to learn significantly faster than the baselines.
Tasks
Published	2018-09-30
URL	http://arxiv.org/abs/1810.00361v1
PDF	http://arxiv.org/pdf/1810.00361v1.pdf
PWC	https://paperswithcode.com/paper/using-state-predictions-for-value
Repo	https://github.com/ManuelFritsche/vpc
Framework	tf

Spell Once, Summon Anywhere: A Two-Level Open-Vocabulary Language Model


Title	Spell Once, Summon Anywhere: A Two-Level Open-Vocabulary Language Model
Authors	Sabrina J. Mielke, Jason Eisner
Abstract	We show how the spellings of known words can help us deal with unknown words in open-vocabulary NLP tasks. The method we propose can be used to extend any closed-vocabulary generative model, but in this paper we specifically consider the case of neural language modeling. Our Bayesian generative story combines a standard RNN language model (generating the word tokens in each sentence) with an RNN-based spelling model (generating the letters in each word type). These two RNNs respectively capture sentence structure and word structure, and are kept separate as in linguistics. By invoking the second RNN to generate spellings for novel words in context, we obtain an open-vocabulary language model. For known words, embeddings are naturally inferred by combining evidence from type spelling and token context. Comparing to baselines (including a novel strong baseline), we beat previous work and establish state-of-the-art results on multiple datasets.
Tasks	Language Modelling
Published	2018-04-23
URL	https://arxiv.org/abs/1804.08205v4
PDF	https://arxiv.org/pdf/1804.08205v4.pdf
PWC	https://paperswithcode.com/paper/spell-once-summon-anywhere-a-two-level-open
Repo	https://github.com/sjmielke/spell-once
Framework	pytorch