October 18, 2019

3261 words 16 mins read

Paper Group ANR 610

A Feature-Driven Active Framework for Ultrasound-Based Brain Shift Compensation. MatchBench: An Evaluation of Feature Matchers. An Efficient, Generalized Bellman Update For Cooperative Inverse Reinforcement Learning. Symbol Emergence in Cognitive Developmental Systems: a Survey. Tight Analyses for Non-Smooth Stochastic Gradient Descent. Folksonomic …

A Feature-Driven Active Framework for Ultrasound-Based Brain Shift Compensation


Title	A Feature-Driven Active Framework for Ultrasound-Based Brain Shift Compensation
Authors	Jie Luo, Matt Toews, Ines Machado, Sarah Frisken, Miaomiao Zhang, Frank Preiswerk, Alireza Sedghi, Hongyi Ding, Steve Pieper, Polina Golland, Alexandra Golby, Masashi Sugiyama, William M. Wells III
Abstract	A reliable Ultrasound (US)-to-US registration method to compensate for brain shift would substantially improve Image-Guided Neurological Surgery. Developing such a registration method is very challenging, due to factors such as missing correspondence in images, the complexity of brain pathology and the demand for fast computation. We propose a novel feature-driven active framework. Here, landmarks and their displacement are first estimated from a pair of US images using corresponding local image features. Subsequently, a Gaussian Process (GP) model is used to interpolate a dense deformation field from the sparse landmarks. Kernels of the GP are estimated by using variograms and a discrete grid search method. If necessary, the user can actively add new landmarks based on the image context and visualization of the uncertainty measure provided by the GP to further improve the result. We retrospectively demonstrate our registration framework as a robust and accurate brain shift compensation solution on clinical data acquired during neurosurgery.
Tasks
Published	2018-03-20
URL	http://arxiv.org/abs/1803.07682v1
PDF	http://arxiv.org/pdf/1803.07682v1.pdf
PWC	https://paperswithcode.com/paper/a-feature-driven-active-framework-for
Repo
Framework

MatchBench: An Evaluation of Feature Matchers


Title	MatchBench: An Evaluation of Feature Matchers
Authors	JiaWang Bian, Ruihan Yang, Yun Liu, Le Zhang, Ming-Ming Cheng, Ian Reid, WenHai Wu
Abstract	Feature matching is one of the most fundamental and active research areas in computer vision. A comprehensive evaluation of feature matchers is necessary, since it would advance both the development of this field and also high-level applications such as Structure-from-Motion or Visual SLAM. However, to the best of our knowledge, no previous work targets the evaluation of feature matchers while they only focus on evaluating feature detectors and descriptors. This leads to a critical absence in this field that there is no standard datasets and evaluation metrics to evaluate different feature matchers fairly. To this end, we present the first uniform feature matching benchmark to facilitate the evaluation of feature matchers. In the proposed benchmark, matchers are evaluated in different aspects, involving matching ability, correspondence sufficiency, and efficiency. Also, their performances are investigated in different scenes and in different matching types. Subsequently, we carry out an extensive evaluation of different state-of-the-art matchers on the benchmark and make in-depth analyses based on the reported results. This can be used to design practical matching systems in real applications and also advocates the potential future research directions in the field of feature matching.
Tasks
Published	2018-08-07
URL	http://arxiv.org/abs/1808.02267v1
PDF	http://arxiv.org/pdf/1808.02267v1.pdf
PWC	https://paperswithcode.com/paper/matchbench-an-evaluation-of-feature-matchers
Repo
Framework

An Efficient, Generalized Bellman Update For Cooperative Inverse Reinforcement Learning


Title	An Efficient, Generalized Bellman Update For Cooperative Inverse Reinforcement Learning
Authors	Dhruv Malik, Malayandi Palaniappan, Jaime F. Fisac, Dylan Hadfield-Menell, Stuart Russell, Anca D. Dragan
Abstract	Our goal is for AI systems to correctly identify and act according to their human user’s objectives. Cooperative Inverse Reinforcement Learning (CIRL) formalizes this value alignment problem as a two-player game between a human and robot, in which only the human knows the parameters of the reward function: the robot needs to learn them as the interaction unfolds. Previous work showed that CIRL can be solved as a POMDP, but with an action space size exponential in the size of the reward parameter space. In this work, we exploit a specific property of CIRL—the human is a full information agent—to derive an optimality-preserving modification to the standard Bellman update; this reduces the complexity of the problem by an exponential factor and allows us to relax CIRL’s assumption of human rationality. We apply this update to a variety of POMDP solvers and find that it enables us to scale CIRL to non-trivial problems, with larger reward parameter spaces, and larger action spaces for both robot and human. In solutions to these larger problems, the human exhibits pedagogic (teaching) behavior, while the robot interprets it as such and attains higher value for the human.
Tasks
Published	2018-06-11
URL	http://arxiv.org/abs/1806.03820v1
PDF	http://arxiv.org/pdf/1806.03820v1.pdf
PWC	https://paperswithcode.com/paper/an-efficient-generalized-bellman-update-for
Repo
Framework

Symbol Emergence in Cognitive Developmental Systems: a Survey


Title	Symbol Emergence in Cognitive Developmental Systems: a Survey
Authors	Tadahiro Taniguchi, Emre Ugur, Matej Hoffmann, Lorenzo Jamone, Takayuki Nagai, Benjamin Rosman, Toshihiko Matsuka, Naoto Iwahashi, Erhan Oztop, Justus Piater, Florentin Wörgötter
Abstract	Humans use signs, e.g., sentences in a spoken language, for communication and thought. Hence, symbol systems like language are crucial for our communication with other agents and adaptation to our real-world environment. The symbol systems we use in our human society adaptively and dynamically change over time. In the context of artificial intelligence (AI) and cognitive systems, the symbol grounding problem has been regarded as one of the central problems related to {\it symbols}. However, the symbol grounding problem was originally posed to connect symbolic AI and sensorimotor information and did not consider many interdisciplinary phenomena in human communication and dynamic symbol systems in our society, which semiotics considered. In this paper, we focus on the symbol emergence problem, addressing not only cognitive dynamics but also the dynamics of symbol systems in society, rather than the symbol grounding problem. We first introduce the notion of a symbol in semiotics from the humanities, to leave the very narrow idea of symbols in symbolic AI. Furthermore, over the years, it became more and more clear that symbol emergence has to be regarded as a multifaceted problem. Therefore, secondly, we review the history of the symbol emergence problem in different fields, including both biological and artificial systems, showing their mutual relations. We summarize the discussion and provide an integrative viewpoint and comprehensive overview of symbol emergence in cognitive systems. Additionally, we describe the challenges facing the creation of cognitive systems that can be part of symbol emergence systems.
Tasks
Published	2018-01-26
URL	http://arxiv.org/abs/1801.08829v2
PDF	http://arxiv.org/pdf/1801.08829v2.pdf
PWC	https://paperswithcode.com/paper/symbol-emergence-in-cognitive-developmental
Repo
Framework

Tight Analyses for Non-Smooth Stochastic Gradient Descent


Title	Tight Analyses for Non-Smooth Stochastic Gradient Descent
Authors	Nicholas J. A. Harvey, Christopher Liaw, Yaniv Plan, Sikander Randhawa
Abstract	Consider the problem of minimizing functions that are Lipschitz and strongly convex, but not necessarily differentiable. We prove that after $T$ steps of stochastic gradient descent, the error of the final iterate is $O(\log(T)/T)$ with high probability. We also construct a function from this class for which the error of the final iterate of deterministic gradient descent is $\Omega(\log(T)/T)$. This shows that the upper bound is tight and that, in this setting, the last iterate of stochastic gradient descent has the same general error rate (with high probability) as deterministic gradient descent. This resolves both open questions posed by Shamir (2012). An intermediate step of our analysis proves that the suffix averaging method achieves error $O(1/T)$ with high probability, which is optimal (for any first-order optimization method). This improves results of Rakhlin (2012) and Hazan and Kale (2014), both of which achieved error $O(1/T)$, but only in expectation, and achieved a high probability error bound of $O(\log \log(T)/T)$, which is suboptimal. We prove analogous results for functions that are Lipschitz and convex, but not necessarily strongly convex or differentiable. After $T$ steps of stochastic gradient descent, the error of the final iterate is $O(\log(T)/\sqrt{T})$ with high probability, and there exists a function for which the error of the final iterate of deterministic gradient descent is $\Omega(\log(T)/\sqrt{T})$.
Tasks
Published	2018-12-13
URL	http://arxiv.org/abs/1812.05217v1
PDF	http://arxiv.org/pdf/1812.05217v1.pdf
PWC	https://paperswithcode.com/paper/tight-analyses-for-non-smooth-stochastic
Repo
Framework

Folksonomication: Predicting Tags for Movies from Plot Synopses Using Emotion Flow Encoded Neural Network


Title	Folksonomication: Predicting Tags for Movies from Plot Synopses Using Emotion Flow Encoded Neural Network
Authors	Sudipta Kar, Suraj Maharjan, Thamar Solorio
Abstract	Folksonomy of movies covers a wide range of heterogeneous information about movies, like the genre, plot structure, visual experiences, soundtracks, metadata, and emotional experiences from watching a movie. Being able to automatically generate or predict tags for movies can help recommendation engines improve retrieval of similar movies, and help viewers know what to expect from a movie in advance. In this work, we explore the problem of creating tags for movies from plot synopses. We propose a novel neural network model that merges information from synopses and emotion flows throughout the plots to predict a set of tags for movies. We compare our system with multiple baselines and found that the addition of emotion flows boosts the performance of the network by learning ~18% more tags than a traditional machine learning system.
Tasks
Published	2018-08-15
URL	http://arxiv.org/abs/1808.04943v1
PDF	http://arxiv.org/pdf/1808.04943v1.pdf
PWC	https://paperswithcode.com/paper/folksonomication-predicting-tags-for-movies
Repo
Framework

Supervised Kernel PCA For Longitudinal Data


Title	Supervised Kernel PCA For Longitudinal Data
Authors	Patrick Staples, Min Ouyang, Robert F. Dougherty, Gregory A. Ryslik, Paul Dagum
Abstract	In statistical learning, high covariate dimensionality poses challenges for robust prediction and inference. To address this challenge, supervised dimension reduction is often performed, where dependence on the outcome is maximized for a selected covariate subspace with smaller dimensionality. Prevalent dimension reduction techniques assume data are $i.i.d.$, which is not appropriate for longitudinal data comprising multiple subjects with repeated measurements over time. In this paper, we derive a decomposition of the Hilbert-Schmidt Independence Criterion as a supervised loss function for longitudinal data, enabling dimension reduction between and within clusters separately, and propose a dimensionality-reduction technique, $sklPCA$, that performs this decomposed dimension reduction. We also show that this technique yields superior model accuracy compared to the model it extends.
Tasks	Dimensionality Reduction
Published	2018-08-20
URL	http://arxiv.org/abs/1808.06638v3
PDF	http://arxiv.org/pdf/1808.06638v3.pdf
PWC	https://paperswithcode.com/paper/supervised-kernel-pca-for-longitudinal-data
Repo
Framework

ORIGAMI: A Heterogeneous Split Architecture for In-Memory Acceleration of Learning


Title	ORIGAMI: A Heterogeneous Split Architecture for In-Memory Acceleration of Learning
Authors	Hajar Falahati, Pejman Lotfi-Kamran, Mohammad Sadrosadati, Hamid Sarbazi-Azad
Abstract	Memory bandwidth bottleneck is a major challenges in processing machine learning (ML) algorithms. In-memory acceleration has potential to address this problem; however, it needs to address two challenges. First, in-memory accelerator should be general enough to support a large set of different ML algorithms. Second, it should be efficient enough to utilize bandwidth while meeting limited power and area budgets of logic layer of a 3D-stacked memory. We observe that previous work fails to simultaneously address both challenges. We propose ORIGAMI, a heterogeneous set of in-memory accelerators, to support compute demands of different ML algorithms, and also uses an off-the-shelf compute platform (e.g.,FPGA,GPU,TPU,etc.) to utilize bandwidth without violating strict area and power budgets. ORIGAMI offers a pattern-matching technique to identify similar computation patterns of ML algorithms and extracts a compute engine for each pattern. These compute engines constitute heterogeneous accelerators integrated on logic layer of a 3D-stacked memory. Combination of these compute engines can execute any type of ML algorithms. To utilize available bandwidth without violating area and power budgets of logic layer, ORIGAMI comes with a computation-splitting compiler that divides an ML algorithm between in-memory accelerators and an out-of-the-memory platform in a balanced way and with minimum inter-communications. Combination of pattern matching and split execution offers a new design point for acceleration of ML algorithms. Evaluation results across 12 popular ML algorithms show that ORIGAMI outperforms state-of-the-art accelerator with 3D-stacked memory in terms of performance and energy-delay product (EDP) by 1.5x and 29x (up to 1.6x and 31x), respectively. Furthermore, results are within a 1% margin of an ideal system that has unlimited compute resources on logic layer of a 3D-stacked memory.
Tasks
Published	2018-12-30
URL	http://arxiv.org/abs/1812.11473v2
PDF	http://arxiv.org/pdf/1812.11473v2.pdf
PWC	https://paperswithcode.com/paper/origami-a-heterogeneous-split-architecture
Repo
Framework

An Efficient Approximation Algorithm for Multi-criteria Indoor Route Planning Queries


Title	An Efficient Approximation Algorithm for Multi-criteria Indoor Route Planning Queries
Authors	Chaluka Salgado, Muhammad Aamir Cheema, David Taniar
Abstract	A route planning query has many real-world applications and has been studied extensively in outdoor spaces such as road networks or Euclidean space. Despite its many applications in indoor venues (e.g., shopping centres, libraries, airports), almost all existing studies are specifically designed for outdoor spaces and do not take into account unique properties of the indoor spaces such as hallways, stairs, escalators, rooms etc. We identify this research gap and formally define the problem of category aware multi-criteria route planning query, denoted by CAM, which returns the optimal route from an indoor source point to an indoor target point that passes through at least one indoor point from each given category while minimizing the total cost of the route in terms of travel distance and other relevant attributes. We show that CAM query is NP-hard. Based on a novel dominance-based pruning, we propose an efficient algorithm which generates high-quality results. We provide an extensive experimental study conducted on the largest shopping centre in Australia and compare our algorithm with alternative approaches. The experiments demonstrate that our algorithm is highly efficient and produces quality results.
Tasks
Published	2018-09-18
URL	http://arxiv.org/abs/1809.07614v1
PDF	http://arxiv.org/pdf/1809.07614v1.pdf
PWC	https://paperswithcode.com/paper/an-efficient-approximation-algorithm-for
Repo
Framework

Stochastic Hyperparameter Optimization through Hypernetworks


Title	Stochastic Hyperparameter Optimization through Hypernetworks
Authors	Jonathan Lorraine, David Duvenaud
Abstract	Machine learning models are often tuned by nesting optimization of model weights inside the optimization of hyperparameters. We give a method to collapse this nested optimization into joint stochastic optimization of weights and hyperparameters. Our process trains a neural network to output approximately optimal weights as a function of hyperparameters. We show that our technique converges to locally optimal weights and hyperparameters for sufficiently large hypernetworks. We compare this method to standard hyperparameter optimization strategies and demonstrate its effectiveness for tuning thousands of hyperparameters.
Tasks	Hyperparameter Optimization, Stochastic Optimization
Published	2018-02-26
URL	http://arxiv.org/abs/1802.09419v2
PDF	http://arxiv.org/pdf/1802.09419v2.pdf
PWC	https://paperswithcode.com/paper/stochastic-hyperparameter-optimization
Repo
Framework

Understanding Learning Dynamics Of Language Models with SVCCA


Title	Understanding Learning Dynamics Of Language Models with SVCCA
Authors	Naomi Saphra, Adam Lopez
Abstract	Research has shown that neural models implicitly encode linguistic features, but there has been no research showing \emph{how} these encodings arise as the models are trained. We present the first study on the learning dynamics of neural language models, using a simple and flexible analysis method called Singular Vector Canonical Correlation Analysis (SVCCA), which enables us to compare learned representations across time and across models, without the need to evaluate directly on annotated data. We probe the evolution of syntactic, semantic, and topic representations and find that part-of-speech is learned earlier than topic; that recurrent layers become more similar to those of a tagger during training; and embedding layers less similar. Our results and methods could inform better learning algorithms for NLP models, possibly to incorporate linguistic information more effectively.
Tasks	Language Modelling
Published	2018-11-01
URL	http://arxiv.org/abs/1811.00225v3
PDF	http://arxiv.org/pdf/1811.00225v3.pdf
PWC	https://paperswithcode.com/paper/understanding-learning-dynamics-of-language
Repo
Framework

Scalable attribute-aware network embedding with locality


Title	Scalable attribute-aware network embedding with locality
Authors	Weiyi Liu, Zhining Liu, Toyotaro Suzumura, Guangmin Hu
Abstract	Adding attributes for nodes to network embedding helps to improve the ability of the learned joint representation to depict features from topology and attributes simultaneously. Recent research on the joint embedding has exhibited a promising performance on a variety of tasks by jointly embedding the two spaces. However, due to the indispensable requirement of globality based information, present approaches contain a flaw of in-scalability. Here we propose \emph{SANE}, a scalable attribute-aware network embedding algorithm with locality, to learn the joint representation from topology and attributes. By enforcing the alignment of a local linear relationship between each node and its K-nearest neighbors in topology and attribute space, the joint embedding representations are more informative comparing with a single representation from topology or attributes alone. And we argue that the locality in \emph{SANE} is the key to learning the joint representation at scale. By using several real-world networks from diverse domains, We demonstrate the efficacy of \emph{SANE} in performance and scalability aspect. Overall, for performance on label classification, SANE successfully reaches up to the highest F1-score on most datasets, and even closer to the baseline method that needs label information as extra inputs, compared with other state-of-the-art joint representation algorithms. What’s more, \emph{SANE} has an up to 71.4% performance gain compared with the single topology-based algorithm. For scalability, we have demonstrated the linearly time complexity of \emph{SANE}. In addition, we intuitively observe that when the network size scales to 100,000 nodes, the “learning joint embedding” step of \emph{SANE} only takes $\approx10$ seconds.
Tasks	Network Embedding
Published	2018-04-17
URL	http://arxiv.org/abs/1804.07152v2
PDF	http://arxiv.org/pdf/1804.07152v2.pdf
PWC	https://paperswithcode.com/paper/scalable-attribute-aware-network-embedding
Repo
Framework

Entropy-Isomap: Manifold Learning for High-dimensional Dynamic Processes


Title	Entropy-Isomap: Manifold Learning for High-dimensional Dynamic Processes
Authors	Frank Schoeneman, Varun Chandola, Nils Napp, Olga Wodo, Jaroslaw Zola
Abstract	Scientific and engineering processes deliver massive high-dimensional data sets that are generated as non-linear transformations of an initial state and few process parameters. Mapping such data to a low-dimensional manifold facilitates better understanding of the underlying processes, and enables their optimization. In this paper, we first show that off-the-shelf non-linear spectral dimensionality reduction methods, e.g., Isomap, fail for such data, primarily due to the presence of strong temporal correlations. Then, we propose a novel method, Entropy-Isomap, to address the issue. The proposed method is successfully applied to large data describing a fabrication process of organic materials. The resulting low-dimensional representation correctly captures process control variables, allows for low-dimensional visualization of the material morphology evolution, and provides key insights to improve the process.
Tasks	Dimensionality Reduction
Published	2018-02-19
URL	http://arxiv.org/abs/1802.06823v2
PDF	http://arxiv.org/pdf/1802.06823v2.pdf
PWC	https://paperswithcode.com/paper/entropy-isomap-manifold-learning-for-high
Repo
Framework

Error Estimation for Randomized Least-Squares Algorithms via the Bootstrap


Title	Error Estimation for Randomized Least-Squares Algorithms via the Bootstrap
Authors	Miles E. Lopes, Shusen Wang, Michael W. Mahoney
Abstract	Over the course of the past decade, a variety of randomized algorithms have been proposed for computing approximate least-squares (LS) solutions in large-scale settings. A longstanding practical issue is that, for any given input, the user rarely knows the actual error of an approximate solution (relative to the exact solution). Likewise, it is difficult for the user to know precisely how much computation is needed to achieve the desired error tolerance. Consequently, the user often appeals to worst-case error bounds that tend to offer only qualitative guidance. As a more practical alternative, we propose a bootstrap method to compute a posteriori error estimates for randomized LS algorithms. These estimates permit the user to numerically assess the error of a given solution, and to predict how much work is needed to improve a “preliminary” solution. In addition, we provide theoretical consistency results for the method, which are the first such results in this context (to the best of our knowledge). From a practical standpoint, the method also has considerable flexibility, insofar as it can be applied to several popular sketching algorithms, as well as a variety of error metrics. Moreover, the extra step of error estimation does not add much cost to an underlying sketching algorithm. Finally, we demonstrate the effectiveness of the method with empirical results.
Tasks
Published	2018-03-21
URL	http://arxiv.org/abs/1803.08021v2
PDF	http://arxiv.org/pdf/1803.08021v2.pdf
PWC	https://paperswithcode.com/paper/error-estimation-for-randomized-least-squares
Repo
Framework

Estimating Gradual-Emotional Behavior in One-Minute Videos with ESNs


Title	Estimating Gradual-Emotional Behavior in One-Minute Videos with ESNs
Authors	Tianlin Liu, Arvid Kappas
Abstract	In this paper, we describe our approach for the OMG- Emotion Challenge 2018. The goal is to produce utterance-level valence and arousal estimations for videos of approximately 1 minute length. We tackle this problem by first extracting facial expressions features of videos as time series data, and then using Recurrent Neural Networks of the Echo State Network type to model the correspondence between the time series data and valence-arousal values. Experimentally we show that the proposed approach surpasses the baseline methods provided by the organizers.
Tasks	Time Series
Published	2018-05-02
URL	http://arxiv.org/abs/1805.08690v1
PDF	http://arxiv.org/pdf/1805.08690v1.pdf
PWC	https://paperswithcode.com/paper/estimating-gradual-emotional-behavior-in-one
Repo
Framework