October 19, 2019

3145 words 15 mins read

Paper Group ANR 190

Directed Exploration in PAC Model-Free Reinforcement Learning. Query2Vec: An Evaluation of NLP Techniques for Generalized Workload Analytics. Learning agent’s spatial configuration from sensorimotor invariants. Reinforcement learning for non-prehensile manipulation: Transfer from simulation to physical system. Penalized matrix decomposition for den …

Directed Exploration in PAC Model-Free Reinforcement Learning


Title	Directed Exploration in PAC Model-Free Reinforcement Learning
Authors	Min-hwan Oh, Garud Iyengar
Abstract	We study an exploration method for model-free RL that generalizes the counter-based exploration bonus methods and takes into account long term exploratory value of actions rather than a single step look-ahead. We propose a model-free RL method that modifies Delayed Q-learning and utilizes the long-term exploration bonus with provable efficiency. We show that our proposed method finds a near-optimal policy in polynomial time (PAC-MDP), and also provide experimental evidence that our proposed algorithm is an efficient exploration method.
Tasks	Efficient Exploration, Q-Learning
Published	2018-08-31
URL	http://arxiv.org/abs/1808.10552v1
PDF	http://arxiv.org/pdf/1808.10552v1.pdf
PWC	https://paperswithcode.com/paper/directed-exploration-in-pac-model-free
Repo
Framework

Query2Vec: An Evaluation of NLP Techniques for Generalized Workload Analytics


Title	Query2Vec: An Evaluation of NLP Techniques for Generalized Workload Analytics
Authors	Shrainik Jain, Bill Howe, Jiaqi Yan, Thierry Cruanes
Abstract	We consider methods for learning vector representations of SQL queries to support generalized workload analytics tasks, including workload summarization for index selection and predicting queries that will trigger memory errors. We consider vector representations of both raw SQL text and optimized query plans, and evaluate these methods on synthetic and real SQL workloads. We find that general algorithms based on vector representations can outperform existing approaches that rely on specialized features. For index recommendation, we cluster the vector representations to compress large workloads with no loss in performance from the recommended index. For error prediction, we train a classifier over learned vectors that can automatically relate subtle syntactic patterns with specific errors raised during query execution. Surprisingly, we also find that these methods enable transfer learning, where a model trained on one SQL corpus can be applied to an unrelated corpus and still enable good performance. We find that these general approaches, when trained on a large corpus of SQL queries, provides a robust foundation for a variety of workload analysis tasks and database features, without requiring application-specific feature engineering.
Tasks	Feature Engineering, Transfer Learning
Published	2018-01-17
URL	http://arxiv.org/abs/1801.05613v2
PDF	http://arxiv.org/pdf/1801.05613v2.pdf
PWC	https://paperswithcode.com/paper/query2vec-an-evaluation-of-nlp-techniques-for
Repo
Framework

Learning agent’s spatial configuration from sensorimotor invariants


Title	Learning agent’s spatial configuration from sensorimotor invariants
Authors	Alban Laflaquière, J. Kevin O’Regan, Sylvain Argentieri, Bruno Gas, Alexander V. Terekhov
Abstract	The design of robotic systems is largely dictated by our purely human intuition about how we perceive the world. This intuition has been proven incorrect with regard to a number of critical issues, such as visual change blindness. In order to develop truly autonomous robots, we must step away from this intuition and let robotic agents develop their own way of perceiving. The robot should start from scratch and gradually develop perceptual notions, under no prior assumptions, exclusively by looking into its sensorimotor experience and identifying repetitive patterns and invariants. One of the most fundamental perceptual notions, space, cannot be an exception to this requirement. In this paper we look into the prerequisites for the emergence of simplified spatial notions on the basis of a robot’s sensorimotor flow. We show that the notion of space as environment-independent cannot be deduced solely from exteroceptive information, which is highly variable and is mainly determined by the contents of the environment. The environment-independent definition of space can be approached by looking into the functions that link the motor commands to changes in exteroceptive inputs. In a sufficiently rich environment, the kernels of these functions correspond uniquely to the spatial configuration of the agent’s exteroceptors. We simulate a redundant robotic arm with a retina installed at its end-point and show how this agent can learn the configuration space of its retina. The resulting manifold has the topology of the Cartesian product of a plane and a circle, and corresponds to the planar position and orientation of the retina.
Tasks
Published	2018-10-03
URL	http://arxiv.org/abs/1810.01872v1
PDF	http://arxiv.org/pdf/1810.01872v1.pdf
PWC	https://paperswithcode.com/paper/learning-agents-spatial-configuration-from
Repo
Framework

Reinforcement learning for non-prehensile manipulation: Transfer from simulation to physical system


Title	Reinforcement learning for non-prehensile manipulation: Transfer from simulation to physical system
Authors	Kendall Lowrey, Svetoslav Kolev, Jeremy Dao, Aravind Rajeswaran, Emanuel Todorov
Abstract	Reinforcement learning has emerged as a promising methodology for training robot controllers. However, most results have been limited to simulation due to the need for a large number of samples and the lack of automated-yet-safe data collection methods. Model-based reinforcement learning methods provide an avenue to circumvent these challenges, but the traditional concern has been the mismatch between the simulator and the real world. Here, we show that control policies learned in simulation can successfully transfer to a physical system, composed of three Phantom robots pushing an object to various desired target positions. We use a modified form of the natural policy gradient algorithm for learning, applied to a carefully identified simulation model. The resulting policies, trained entirely in simulation, work well on the physical system without additional training. In addition, we show that training with an ensemble of models makes the learned policies more robust to modeling errors, thus compensating for difficulties in system identification.
Tasks
Published	2018-03-28
URL	http://arxiv.org/abs/1803.10371v1
PDF	http://arxiv.org/pdf/1803.10371v1.pdf
PWC	https://paperswithcode.com/paper/reinforcement-learning-for-non-prehensile
Repo
Framework

Penalized matrix decomposition for denoising, compression, and improved demixing of functional imaging data


Title	Penalized matrix decomposition for denoising, compression, and improved demixing of functional imaging data
Authors	E. Kelly Buchanan, Ian Kinsella, Ding Zhou, Rong Zhu, Pengcheng Zhou, Felipe Gerhard, John Ferrante, Ying Ma, Sharon Kim, Mohammed Shaik, Yajie Liang, Rongwen Lu, Jacob Reimer, Paul Fahey, Taliah Muhammad, Graham Dempsey, Elizabeth Hillman, Na Ji, Andreas Tolias, Liam Paninski
Abstract	Calcium imaging has revolutionized systems neuroscience, providing the ability to image large neural populations with single-cell resolution. The resulting datasets are quite large, which has presented a barrier to routine open sharing of this data, slowing progress in reproducible research. State of the art methods for analyzing this data are based on non-negative matrix factorization (NMF); these approaches solve a non-convex optimization problem, and are effective when good initializations are available, but can break down in low-SNR settings where common initialization approaches fail. Here we introduce an approach to compressing and denoising functional imaging data. The method is based on a spatially-localized penalized matrix decomposition (PMD) of the data to separate (low-dimensional) signal from (temporally-uncorrelated) noise. This approach can be applied in parallel on local spatial patches and is therefore highly scalable, does not impose non-negativity constraints or require stringent identifiability assumptions (leading to significantly more robust results compared to NMF), and estimates all parameters directly from the data, so no hand-tuning is required. We have applied the method to a wide range of functional imaging data (including one-photon, two-photon, three-photon, widefield, somatic, axonal, dendritic, calcium, and voltage imaging datasets): in all cases, we observe ~2-4x increases in SNR and compression rates of 20-300x with minimal visible loss of signal, with no adjustment of hyperparameters; this in turn facilitates the process of demixing the observed activity into contributions from individual neurons. We focus on two challenging applications: dendritic calcium imaging data and voltage imaging data in the context of optogenetic stimulation. In both cases, we show that our new approach leads to faster and much more robust extraction of activity from the data.
Tasks	Denoising
Published	2018-07-17
URL	http://arxiv.org/abs/1807.06203v1
PDF	http://arxiv.org/pdf/1807.06203v1.pdf
PWC	https://paperswithcode.com/paper/penalized-matrix-decomposition-for-denoising
Repo
Framework

Learning Filter Scale and Orientation In CNNs


Title	Learning Filter Scale and Orientation In CNNs
Authors	Ilker Cam, F. Boray Tek
Abstract	Convolutional neural networks have many hyperparameters such as the filter size, number of filters, and pooling size, which require manual tuning. Though deep stacked structures are able to create multi-scale and hierarchical representations, manually fixed filter sizes limit the scale of representations that can be learned in a single convolutional layer. This paper introduces a new adaptive filter model that allows variable scale and orientation. The scale and orientation parameters of filters can be learned using back propagation. Therefore, in a single convolution layer, we can create filters of different scale and orientation that can adapt to small or large features and objects. The proposed model uses a relatively large base size (grid) for filters. In the grid, a differentiable function acts as an envelope for the filters. The envelope function guides effective filter scale and shape/orientation by masking the filter weights before the convolution. Therefore, only the weights in the envelope are updated during training. In this work, we employed a multivariate (2D) Gaussian as the envelope function and showed that it can grow, shrink, or rotate by updating its covariance matrix during back propagation training . We tested the new filter model on MNIST, MNIST-cluttered, and CIFAR-10 and compared the results with the networks that used conventional convolution layers. The results demonstrate that the new model can effectively learn and produce filters of different scales and orientations in a single layer. Moreover, the experiments show that the adaptive convolution layers perform equally; or better, especially when data includes objects of varying scale and noisy backgrounds.
Tasks
Published	2018-02-13
URL	http://arxiv.org/abs/1803.00388v1
PDF	http://arxiv.org/pdf/1803.00388v1.pdf
PWC	https://paperswithcode.com/paper/learning-filter-scale-and-orientation-in-cnns
Repo
Framework

From CDF to PDF — A Density Estimation Method for High Dimensional Data


Title	From CDF to PDF — A Density Estimation Method for High Dimensional Data
Authors	Shengdong Zhang
Abstract	CDF2PDF is a method of PDF estimation by approximating CDF. The original idea of it was previously proposed in [1] called SIC. However, SIC requires additional hyper-parameter tunning, and no algorithms for computing higher order derivative from a trained NN are provided in [1]. CDF2PDF improves SIC by avoiding the time-consuming hyper-parameter tuning part and enabling higher order derivative computation to be done in polynomial time. Experiments of this method for one-dimensional data shows promising results.
Tasks	Density Estimation
Published	2018-04-15
URL	http://arxiv.org/abs/1804.05316v1
PDF	http://arxiv.org/pdf/1804.05316v1.pdf
PWC	https://paperswithcode.com/paper/from-cdf-to-pdf-a-density-estimation-method
Repo
Framework

Boosting Combinatorial Problem Modeling with Machine Learning


Title	Boosting Combinatorial Problem Modeling with Machine Learning
Authors	Michele Lombardi, Michela Milano
Abstract	In the past few years, the area of Machine Learning (ML) has witnessed tremendous advancements, becoming a pervasive technology in a wide range of applications. One area that can significantly benefit from the use of ML is Combinatorial Optimization. The three pillars of constraint satisfaction and optimization problem solving, i.e., modeling, search, and optimization, can exploit ML techniques to boost their accuracy, efficiency and effectiveness. In this survey we focus on the modeling component, whose effectiveness is crucial for solving the problem. The modeling activity has been traditionally shaped by optimization and domain experts, interacting to provide realistic results. Machine Learning techniques can tremendously ease the process, and exploit the available data to either create models or refine expert-designed ones. In this survey we cover approaches that have been recently proposed to enhance the modeling process by learning either single constraints, objective functions, or the whole model. We highlight common themes to multiple approaches and draw connections with related fields of research.
Tasks	Combinatorial Optimization
Published	2018-07-15
URL	http://arxiv.org/abs/1807.05517v1
PDF	http://arxiv.org/pdf/1807.05517v1.pdf
PWC	https://paperswithcode.com/paper/boosting-combinatorial-problem-modeling-with
Repo
Framework

Axiomatizations of inconsistency indices for triads


Title	Axiomatizations of inconsistency indices for triads
Authors	László Csató
Abstract	Pairwise comparison matrices often exhibit inconsistency, therefore many indices have been suggested to measure their deviation from a consistent matrix. A set of axioms has been proposed recently that is required to be satisfied by any reasonable inconsistency index. This set seems to be not exhaustive as illustrated by an example, hence it is expanded by adding two new properties. All axioms are considered on the set of triads, pairwise comparison matrices with three alternatives, which is the simplest case of inconsistency. We choose the logically independent properties and prove that they characterize, that is, uniquely determine the inconsistency ranking induced by most inconsistency indices that coincide on this restricted domain. Since triads play a prominent role in a number of inconsistency indices, our results can also contribute to the measurement of inconsistency for pairwise comparison matrices with more than three alternatives.
Tasks
Published	2018-01-10
URL	http://arxiv.org/abs/1801.03355v2
PDF	http://arxiv.org/pdf/1801.03355v2.pdf
PWC	https://paperswithcode.com/paper/axiomatizations-of-inconsistency-indices-for
Repo
Framework

Factorized Adversarial Networks for Unsupervised Domain Adaptation


Title	Factorized Adversarial Networks for Unsupervised Domain Adaptation
Authors	Jian Ren, Jianchao Yang, Ning Xu, David J. Foran
Abstract	In this paper, we propose Factorized Adversarial Networks (FAN) to solve unsupervised domain adaptation problems for image classification tasks. Our networks map the data distribution into a latent feature space, which is factorized into a domain-specific subspace that contains domain-specific characteristics and a task-specific subspace that retains category information, for both source and target domains, respectively. Unsupervised domain adaptation is achieved by adversarial training to minimize the discrepancy between the distributions of two task-specific subspaces from source and target domains. We demonstrate that the proposed approach outperforms state-of-the-art methods on multiple benchmark datasets used in the literature for unsupervised domain adaptation. Furthermore, we collect two real-world tagging datasets that are much larger than existing benchmark datasets, and get significant improvement upon baselines, proving the practical value of our approach.
Tasks	Domain Adaptation, Image Classification, Unsupervised Domain Adaptation
Published	2018-06-04
URL	http://arxiv.org/abs/1806.01376v1
PDF	http://arxiv.org/pdf/1806.01376v1.pdf
PWC	https://paperswithcode.com/paper/factorized-adversarial-networks-for
Repo
Framework

Semantic Interoperability Middleware Architecture for Heterogeneous Environmental Data Sources


Title	Semantic Interoperability Middleware Architecture for Heterogeneous Environmental Data Sources
Authors	A. K. Akanbi, M. Masinde
Abstract	Data heterogeneity hampers the effort to integrate and infer knowledge from vast heterogeneous data sources. An application case study is described, in which the objective was to semantically represent and integrate structured data from sensor devices with unstructured data in the form of local indigenous knowledge. However, the semantic representation of these heterogeneous data sources for environmental monitoring systems is not well supported yet. To combat the incompatibility issues, a dedicated semantic middleware solution is required. In this paper, we describe and evaluate a cross-domain middleware architecture that semantically integrates and generate inference from heterogeneous data sources. These use of semantic technology for predicting and forecasting complex environmental phenomenon will increase the degree of accuracy of environmental monitoring systems.
Tasks
Published	2018-09-16
URL	http://arxiv.org/abs/1809.05890v1
PDF	http://arxiv.org/pdf/1809.05890v1.pdf
PWC	https://paperswithcode.com/paper/semantic-interoperability-middleware
Repo
Framework

Spatiotemporal KSVD Dictionary Learning for Online Multi-target Tracking


Title	Spatiotemporal KSVD Dictionary Learning for Online Multi-target Tracking
Authors	Huynh Manh, Gita Alaghband
Abstract	In this paper, we present a new spatial discriminative KSVD dictionary algorithm (STKSVD) for learning target appearance in online multi-target tracking. Different from other classification/recognition tasks (e.g. face, image recognition), learning target’s appearance in online multi-target tracking is impacted by factors such as posture/articulation changes, partial occlusion by background scene or other targets, background changes (human detection bounding box covers human parts and part of the scene), etc. However, we observe that these variations occur gradually relative to spatial and temporal dynamics. We characterize the spatial and temporal information between target’s samples through a new STKSVD appearance learning algorithm to better discriminate sparse code, linear classifier parameters and minimize reconstruction error in a single optimization system. Our appearance learning algorithm and tracking framework employ two different methods of calculating appearance similarity score in each stage of a two-stage association: a linear classifier in the first stage, and minimum residual errors in the second stage. The results tested using 2DMOT2015 dataset and its public Aggregated Channel features (ACF) human detection for all comparisons show that our method outperforms the existing related learning methods.
Tasks	Dictionary Learning, Human Detection
Published	2018-07-05
URL	http://arxiv.org/abs/1807.02143v1
PDF	http://arxiv.org/pdf/1807.02143v1.pdf
PWC	https://paperswithcode.com/paper/spatiotemporal-ksvd-dictionary-learning-for
Repo
Framework

Multi-function Convolutional Neural Networks for Improving Image Classification Performance


Title	Multi-function Convolutional Neural Networks for Improving Image Classification Performance
Authors	Luna M. Zhang
Abstract	Traditional Convolutional Neural Networks (CNNs) typically use the same activation function (usually ReLU) for all neurons with non-linear mapping operations. For example, the deep convolutional architecture Inception-v4 uses ReLU. To improve the classification performance of traditional CNNs, a new “Multi-function Convolutional Neural Network” (MCNN) is created by using different activation functions for different neurons. For $n$ neurons and $m$ different activation functions, there are a total of $m^n-m$ MCNNs and only $m$ traditional CNNs. Therefore, the best model is very likely to be chosen from MCNNs because there are $m^n-2m$ more MCNNs than traditional CNNs. For performance analysis, two different datasets for two applications (classifying handwritten digits from the MNIST database and classifying brain MRI images into one of the four stages of Alzheimer’s disease (AD)) are used. For both applications, an activation function is randomly selected for each layer of a MCNN. For the AD diagnosis application, MCNNs using a newly created multi-function Inception-v4 architecture are constructed. Overall, simulations show that MCNNs can outperform traditional CNNs in terms of multi-class classification accuracy for both applications. An important future research work will be to efficiently select the best MCNN from $m^n-m$ candidate MCNNs. Current CNN software only provides users with partial functionality of MCNNs since different layers can use different activation functions but not individual neurons in the same layer. Thus, modifying current CNN software systems such as ResNets, DenseNets, and Dual Path Networks by using multiple activation functions and developing more effective and faster MCNN software systems and tools would be very useful to solve difficult practical image classification problems.
Tasks	Image Classification
Published	2018-05-30
URL	http://arxiv.org/abs/1805.11788v1
PDF	http://arxiv.org/pdf/1805.11788v1.pdf
PWC	https://paperswithcode.com/paper/multi-function-convolutional-neural-networks
Repo
Framework

Averaging Stochastic Gradient Descent on Riemannian Manifolds


Title	Averaging Stochastic Gradient Descent on Riemannian Manifolds
Authors	Nilesh Tripuraneni, Nicolas Flammarion, Francis Bach, Michael I. Jordan
Abstract	We consider the minimization of a function defined on a Riemannian manifold $\mathcal{M}$ accessible only through unbiased estimates of its gradients. We develop a geometric framework to transform a sequence of slowly converging iterates generated from stochastic gradient descent (SGD) on $\mathcal{M}$ to an averaged iterate sequence with a robust and fast $O(1/n)$ convergence rate. We then present an application of our framework to geodesically-strongly-convex (and possibly Euclidean non-convex) problems. Finally, we demonstrate how these ideas apply to the case of streaming $k$-PCA, where we show how to accelerate the slow rate of the randomized power method (without requiring knowledge of the eigengap) into a robust algorithm achieving the optimal rate of convergence.
Tasks
Published	2018-02-26
URL	http://arxiv.org/abs/1802.09128v2
PDF	http://arxiv.org/pdf/1802.09128v2.pdf
PWC	https://paperswithcode.com/paper/averaging-stochastic-gradient-descent-on
Repo
Framework

Visual Analytics for Explainable Deep Learning


Title	Visual Analytics for Explainable Deep Learning
Authors	Jaegul Choo, Shixia Liu
Abstract	Recently, deep learning has been advancing the state of the art in artificial intelligence to a new level, and humans rely on artificial intelligence techniques more than ever. However, even with such unprecedented advancements, the lack of explanation regarding the decisions made by deep learning models and absence of control over their internal processes act as major drawbacks in critical decision-making processes, such as precision medicine and law enforcement. In response, efforts are being made to make deep learning interpretable and controllable by humans. In this paper, we review visual analytics, information visualization, and machine learning perspectives relevant to this aim, and discuss potential challenges and future research directions.
Tasks	Decision Making
Published	2018-04-07
URL	http://arxiv.org/abs/1804.02527v1
PDF	http://arxiv.org/pdf/1804.02527v1.pdf
PWC	https://paperswithcode.com/paper/visual-analytics-for-explainable-deep
Repo
Framework