July 29, 2019

3184 words 15 mins read

Paper Group ANR 50

Sequential detection of low-rank changes using extreme eigenvalues. Neuro-RAM Unit with Applications to Similarity Testing and Compression in Spiking Neural Networks. Improving LSTM-CTC based ASR performance in domains with limited training data. Deep Diagnostics: Applying Convolutional Neural Networks for Vessels Defects Detection. Linguistic Mark …

Sequential detection of low-rank changes using extreme eigenvalues


Title	Sequential detection of low-rank changes using extreme eigenvalues
Authors	Liyan Xie, Yao Xie
Abstract	We study the problem of detecting an abrupt change to the signal covariance matrix. In particular, the covariance changes from a “white” identity matrix to an unknown spiked or low-rank matrix. Two sequential change-point detection procedures are presented, based on the largest and the smallest eigenvalues of the sample covariance matrix. To control false-alarm-rate, we present an accurate theoretical approximation to the average-run-length (ARL) and expected detection delay (EDD) of the detection, leveraging the extreme eigenvalue distributions from random matrix theory and by capturing a non-negligible temporal correlation in the sequence of scan statistics due to the sliding window approach. Real data examples demonstrate the good performance of our method for detecting behavior change of a swarm.
Tasks	Change Point Detection
Published	2017-06-15
URL	http://arxiv.org/abs/1706.04729v1
PDF	http://arxiv.org/pdf/1706.04729v1.pdf
PWC	https://paperswithcode.com/paper/sequential-detection-of-low-rank-changes
Repo
Framework

Neuro-RAM Unit with Applications to Similarity Testing and Compression in Spiking Neural Networks


Title	Neuro-RAM Unit with Applications to Similarity Testing and Compression in Spiking Neural Networks
Authors	Nancy Lynch, Cameron Musco, Merav Parter
Abstract	We study distributed algorithms implemented in a simplified biologically inspired model for stochastic spiking neural networks. We focus on tradeoffs between computation time and network complexity, along with the role of randomness in efficient neural computation. It is widely accepted that neural computation is inherently stochastic. In recent work, we explored how this stochasticity could be leveraged to solve the `winner-take-all’ leader election task. Here, we focus on using randomness in neural algorithms for similarity testing and compression. In the most basic setting, given two $n$-length patterns of firing neurons, we wish to distinguish if the patterns are equal or $\epsilon$-far from equal. Randomization allows us to solve this task with a very compact network, using $O \left (\frac{\sqrt{n}\log n}{\epsilon}\right)$ auxiliary neurons, which is sublinear in the input size. At the heart of our solution is the design of a $t$-round neural random access memory, or indexing network, which we call a neuro-RAM. This module can be implemented with $O(n/t)$ auxiliary neurons and is useful in many applications beyond similarity testing. Using a VC dimension-based argument, we show that the tradeoff between runtime and network size in our neuro-RAM is nearly optimal. Our result has several implications – since our neuro-RAM can be implemented with deterministic threshold gates, it shows that, in contrast to similarity testing, randomness does not provide significant computational advantages for this problem. It also establishes a separation between feedforward networks whose gates spike with sigmoidal probability functions, and well-studied deterministic sigmoidal networks, whose gates output real number sigmoidal values, and which can implement a neuro-RAM much more efficiently. \|
Tasks
Published	2017-06-05
URL	http://arxiv.org/abs/1706.01382v2
PDF	http://arxiv.org/pdf/1706.01382v2.pdf
PWC	https://paperswithcode.com/paper/neuro-ram-unit-with-applications-to
Repo
Framework

Improving LSTM-CTC based ASR performance in domains with limited training data


Title	Improving LSTM-CTC based ASR performance in domains with limited training data
Authors	Jayadev Billa
Abstract	This paper addresses the observed performance gap between automatic speech recognition (ASR) systems based on Long Short Term Memory (LSTM) neural networks trained with the connectionist temporal classification (CTC) loss function and systems based on hybrid Deep Neural Networks (DNNs) trained with the cross entropy (CE) loss function on domains with limited data. We step through a number of experiments that show incremental improvements on a baseline EESEN toolkit based LSTM-CTC ASR system trained on the Librispeech 100hr (train-clean-100) corpus. Our results show that with effective combination of data augmentation and regularization, a LSTM-CTC based system can exceed the performance of a strong Kaldi based baseline trained on the same data.
Tasks	Data Augmentation, Speech Recognition
Published	2017-07-03
URL	http://arxiv.org/abs/1707.00722v2
PDF	http://arxiv.org/pdf/1707.00722v2.pdf
PWC	https://paperswithcode.com/paper/improving-lstm-ctc-based-asr-performance-in
Repo
Framework

Deep Diagnostics: Applying Convolutional Neural Networks for Vessels Defects Detection


Title	Deep Diagnostics: Applying Convolutional Neural Networks for Vessels Defects Detection
Authors	Stanislav Filippov, Arsenii Moiseev, Andronenko Andrey
Abstract	Coronary angiography is considered to be a safe tool for the evaluation of coronary artery disease and perform in approximately 12 million patients each year worldwide. [1] In most cases, angiograms are manually analyzed by a cardiologist. Actually, there are no clinical practice algorithms which could improve and automate this work. Neural networks show high efficiency in tasks of image analysis and they can be used for the analysis of angiograms and facilitate diagnostics. We have developed an algorithm based on Convolutional Neural Network and Neural Network U-Net [2] for vessels segmentation and defects detection such as stenosis. For our research we used anonymized angiography data obtained from one of the city’s hospitals and augmented them to improve learning efficiency. U-Net usage provided high quality segmentation and the combination of our algorithm with an ensemble of classifiers shows a good accuracy in the task of ischemia evaluation on test data. Subsequently, this approach can be served as a basis for the creation of an analytical system that could speed up the diagnosis of cardiovascular diseases and greatly facilitate the work of a specialist.
Tasks
Published	2017-05-17
URL	http://arxiv.org/abs/1705.06264v2
PDF	http://arxiv.org/pdf/1705.06264v2.pdf
PWC	https://paperswithcode.com/paper/deep-diagnostics-applying-convolutional
Repo
Framework

Linguistic Markers of Influence in Informal Interactions


Title	Linguistic Markers of Influence in Informal Interactions
Authors	Shrimai Prabhumoye, Samridhi Choudhary, Evangelia Spiliopoulou, Christopher Bogart, Carolyn Penstein Rose, Alan W Black
Abstract	There has been a long standing interest in understanding `Social Influence’ both in Social Sciences and in Computational Linguistics. In this paper, we present a novel approach to study and measure interpersonal influence in daily interactions. Motivated by the basic principles of influence, we attempt to identify indicative linguistic features of the posts in an online knitting community. We present the scheme used to operationalize and label the posts with indicator features. Experiments with the identified features show an improvement in the classification accuracy of influence by 3.15%. Our results illustrate the important correlation between the characteristics of the language and its potential to influence others. \|
Tasks
Published	2017-07-14
URL	http://arxiv.org/abs/1707.04546v1
PDF	http://arxiv.org/pdf/1707.04546v1.pdf
PWC	https://paperswithcode.com/paper/linguistic-markers-of-influence-in-informal
Repo
Framework

The Value of Inferring the Internal State of Traffic Participants for Autonomous Freeway Driving


Title	The Value of Inferring the Internal State of Traffic Participants for Autonomous Freeway Driving
Authors	Zachary Sunberg, Christopher Ho, Mykel Kochenderfer
Abstract	Safe interaction with human drivers is one of the primary challenges for autonomous vehicles. In order to plan driving maneuvers effectively, the vehicle’s control system must infer and predict how humans will behave based on their latent internal state (e.g., intentions and aggressiveness). This research uses a simple model for human behavior with unknown parameters that make up the internal states of the traffic participants and presents a method for quantifying the value of estimating these states and planning with their uncertainty explicitly modeled. An upper performance bound is established by an omniscient Monte Carlo Tree Search (MCTS) planner that has perfect knowledge of the internal states. A baseline lower bound is established by planning with MCTS assuming that all drivers have the same internal state. MCTS variants are then used to solve a partially observable Markov decision process (POMDP) that models the internal state uncertainty to determine whether inferring the internal state offers an advantage over the baseline. Applying this method to a freeway lane changing scenario reveals that there is a significant performance gap between the upper bound and baseline. POMDP planning techniques come close to closing this gap, especially when important hidden model parameters are correlated with measurable parameters.
Tasks	Autonomous Vehicles
Published	2017-02-02
URL	http://arxiv.org/abs/1702.00858v1
PDF	http://arxiv.org/pdf/1702.00858v1.pdf
PWC	https://paperswithcode.com/paper/the-value-of-inferring-the-internal-state-of
Repo
Framework

Learning Person Trajectory Representations for Team Activity Analysis


Title	Learning Person Trajectory Representations for Team Activity Analysis
Authors	Nazanin Mehrasa, Yatao Zhong, Frederick Tung, Luke Bornn, Greg Mori
Abstract	Activity analysis in which multiple people interact across a large space is challenging due to the interplay of individual actions and collective group dynamics. We propose an end-to-end approach for learning person trajectory representations for group activity analysis. The learned representations encode rich spatio-temporal dependencies and capture useful motion patterns for recognizing individual events, as well as characteristic group dynamics that can be used to identify groups from their trajectories alone. We develop our deep learning approach in the context of team sports, which provide well-defined sets of events (e.g. pass, shot) and groups of people (teams). Analysis of events and team formations using NHL hockey and NBA basketball datasets demonstrate the generality of our approach.
Tasks
Published	2017-06-03
URL	http://arxiv.org/abs/1706.00893v1
PDF	http://arxiv.org/pdf/1706.00893v1.pdf
PWC	https://paperswithcode.com/paper/learning-person-trajectory-representations
Repo
Framework

Long Short-Term Memory Kalman Filters:Recurrent Neural Estimators for Pose Regularization


Title	Long Short-Term Memory Kalman Filters:Recurrent Neural Estimators for Pose Regularization
Authors	Huseyin Coskun, Felix Achilles, Robert DiPietro, Nassir Navab, Federico Tombari
Abstract	One-shot pose estimation for tasks such as body joint localization, camera pose estimation, and object tracking are generally noisy, and temporal filters have been extensively used for regularization. One of the most widely-used methods is the Kalman filter, which is both extremely simple and general. However, Kalman filters require a motion model and measurement model to be specified a priori, which burdens the modeler and simultaneously demands that we use explicit models that are often only crude approximations of reality. For example, in the pose-estimation tasks mentioned above, it is common to use motion models that assume constant velocity or constant acceleration, and we believe that these simplified representations are severely inhibitive. In this work, we propose to instead learn rich, dynamic representations of the motion and noise models. In particular, we propose learning these models from data using long short term memory, which allows representations that depend on all previous observations and all previous states. We evaluate our method using three of the most popular pose estimation tasks in computer vision, and in all cases we obtain state-of-the-art performance.
Tasks	Object Tracking, Pose Estimation
Published	2017-08-06
URL	http://arxiv.org/abs/1708.01885v1
PDF	http://arxiv.org/pdf/1708.01885v1.pdf
PWC	https://paperswithcode.com/paper/long-short-term-memory-kalman
Repo
Framework

AnchorNet: A Weakly Supervised Network to Learn Geometry-sensitive Features For Semantic Matching


Title	AnchorNet: A Weakly Supervised Network to Learn Geometry-sensitive Features For Semantic Matching
Authors	David Novotny, Diane Larlus, Andrea Vedaldi
Abstract	Despite significant progress of deep learning in recent years, state-of-the-art semantic matching methods still rely on legacy features such as SIFT or HoG. We argue that the strong invariance properties that are key to the success of recent deep architectures on the classification task make them unfit for dense correspondence tasks, unless a large amount of supervision is used. In this work, we propose a deep network, termed AnchorNet, that produces image representations that are well-suited for semantic matching. It relies on a set of filters whose response is geometrically consistent across different object instances, even in the presence of strong intra-class, scale, or viewpoint variations. Trained only with weak image-level labels, the final representation successfully captures information about the object structure and improves results of state-of-the-art semantic matching methods such as the deformable spatial pyramid or the proposal flow methods. We show positive results on the cross-instance matching task where different instances of the same object category are matched as well as on a new cross-category semantic matching task aligning pairs of instances each from a different object class.
Tasks
Published	2017-04-16
URL	http://arxiv.org/abs/1704.04749v1
PDF	http://arxiv.org/pdf/1704.04749v1.pdf
PWC	https://paperswithcode.com/paper/anchornet-a-weakly-supervised-network-to
Repo
Framework

Improvement of training set structure in fusion data cleaning using Time-Domain Global Similarity method


Title	Improvement of training set structure in fusion data cleaning using Time-Domain Global Similarity method
Authors	Jian Liu, Ting Lan, Hong Qin
Abstract	Traditional data cleaning identifies dirty data by classifying original data sequences, which is a class$-$imbalanced problem since the proportion of incorrect data is much less than the proportion of correct ones for most diagnostic systems in Magnetic Confinement Fusion (MCF) devices. When using machine learning algorithms to classify diagnostic data based on class$-$imbalanced training set, most classifiers are biased towards the major class and show very poor classification rates on the minor class. By transforming the direct classification problem about original data sequences into a classification problem about the physical similarity between data sequences, the class$-$balanced effect of Time$-$Domain Global Similarity (TDGS) method on training set structure is investigated in this paper. Meanwhile, the impact of improved training set structure on data cleaning performance of TDGS method is demonstrated with an application example in EAST POlarimetry$-$INTerferometry (POINT) system.
Tasks
Published	2017-06-30
URL	http://arxiv.org/abs/1706.10018v1
PDF	http://arxiv.org/pdf/1706.10018v1.pdf
PWC	https://paperswithcode.com/paper/improvement-of-training-set-structure-in
Repo
Framework

A generalized parsing framework for Abstract Grammars


Title	A generalized parsing framework for Abstract Grammars
Authors	Daniel Harasim, Chris Bruno, Eva Portelance, Martin Rohrmeier, Timothy J. O’Donnell
Abstract	This technical report presents a general framework for parsing a variety of grammar formalisms. We develop a grammar formalism, called an Abstract Grammar, which is general enough to represent grammars at many levels of the hierarchy, including Context Free Grammars, Minimalist Grammars, and Generalized Context-free Grammars. We then develop a single parsing framework which is capable of parsing grammars which are at least up to GCFGs on the hierarchy. Our parsing framework exposes a grammar interface, so that it can parse any particular grammar formalism that can be reduced to an Abstract Grammar.
Tasks
Published	2017-10-31
URL	http://arxiv.org/abs/1710.11301v3
PDF	http://arxiv.org/pdf/1710.11301v3.pdf
PWC	https://paperswithcode.com/paper/a-generalized-parsing-framework-for-abstract
Repo
Framework

Multi-Observation Elicitation


Title	Multi-Observation Elicitation
Authors	Sebastian Casalaina-Martin, Rafael Frongillo, Tom Morgan, Bo Waggoner
Abstract	We study loss functions that measure the accuracy of a prediction based on multiple data points simultaneously. To our knowledge, such loss functions have not been studied before in the area of property elicitation or in machine learning more broadly. As compared to traditional loss functions that take only a single data point, these multi-observation loss functions can in some cases drastically reduce the dimensionality of the hypothesis required. In elicitation, this corresponds to requiring many fewer reports; in empirical risk minimization, it corresponds to algorithms on a hypothesis space of much smaller dimension. We explore some examples of the tradeoff between dimensionality and number of observations, give some geometric characterizations and intuition for relating loss functions and the properties that they elicit, and discuss some implications for both elicitation and machine-learning contexts.
Tasks
Published	2017-06-05
URL	http://arxiv.org/abs/1706.01394v1
PDF	http://arxiv.org/pdf/1706.01394v1.pdf
PWC	https://paperswithcode.com/paper/multi-observation-elicitation
Repo
Framework

In-Datacenter Performance Analysis of a Tensor Processing Unit


Title	In-Datacenter Performance Analysis of a Tensor Processing Unit
Authors	Norman P. Jouppi, Cliff Young, Nishant Patil, David Patterson, Gaurav Agrawal, Raminder Bajwa, Sarah Bates, Suresh Bhatia, Nan Boden, Al Borchers, Rick Boyle, Pierre-luc Cantin, Clifford Chao, Chris Clark, Jeremy Coriell, Mike Daley, Matt Dau, Jeffrey Dean, Ben Gelb, Tara Vazir Ghaemmaghami, Rajendra Gottipati, William Gulland, Robert Hagmann, C. Richard Ho, Doug Hogberg, John Hu, Robert Hundt, Dan Hurt, Julian Ibarz, Aaron Jaffey, Alek Jaworski, Alexander Kaplan, Harshit Khaitan, Andy Koch, Naveen Kumar, Steve Lacy, James Laudon, James Law, Diemthu Le, Chris Leary, Zhuyuan Liu, Kyle Lucke, Alan Lundin, Gordon MacKean, Adriana Maggiore, Maire Mahony, Kieran Miller, Rahul Nagarajan, Ravi Narayanaswami, Ray Ni, Kathy Nix, Thomas Norrie, Mark Omernick, Narayana Penukonda, Andy Phelps, Jonathan Ross, Matt Ross, Amir Salek, Emad Samadiani, Chris Severn, Gregory Sizikov, Matthew Snelham, Jed Souter, Dan Steinberg, Andy Swing, Mercedes Tan, Gregory Thorson, Bo Tian, Horia Toma, Erick Tuttle, Vijay Vasudevan, Richard Walter, Walter Wang, Eric Wilcox, Doe Hyun Yoon
Abstract	Many architects believe that major improvements in cost-energy-performance must now come from domain-specific hardware. This paper evaluates a custom ASIC—called a Tensor Processing Unit (TPU)—deployed in datacenters since 2015 that accelerates the inference phase of neural networks (NN). The heart of the TPU is a 65,536 8-bit MAC matrix multiply unit that offers a peak throughput of 92 TeraOps/second (TOPS) and a large (28 MiB) software-managed on-chip memory. The TPU’s deterministic execution model is a better match to the 99th-percentile response-time requirement of our NN applications than are the time-varying optimizations of CPUs and GPUs (caches, out-of-order execution, multithreading, multiprocessing, prefetching, …) that help average throughput more than guaranteed latency. The lack of such features helps explain why, despite having myriad MACs and a big memory, the TPU is relatively small and low power. We compare the TPU to a server-class Intel Haswell CPU and an Nvidia K80 GPU, which are contemporaries deployed in the same datacenters. Our workload, written in the high-level TensorFlow framework, uses production NN applications (MLPs, CNNs, and LSTMs) that represent 95% of our datacenters’ NN inference demand. Despite low utilization for some applications, the TPU is on average about 15X - 30X faster than its contemporary GPU or CPU, with TOPS/Watt about 30X - 80X higher. Moreover, using the GPU’s GDDR5 memory in the TPU would triple achieved TOPS and raise TOPS/Watt to nearly 70X the GPU and 200X the CPU.
Tasks
Published	2017-04-16
URL	http://arxiv.org/abs/1704.04760v1
PDF	http://arxiv.org/pdf/1704.04760v1.pdf
PWC	https://paperswithcode.com/paper/in-datacenter-performance-analysis-of-a
Repo
Framework

Few-shot Autoregressive Density Estimation: Towards Learning to Learn Distributions


Title	Few-shot Autoregressive Density Estimation: Towards Learning to Learn Distributions
Authors	Scott Reed, Yutian Chen, Thomas Paine, Aäron van den Oord, S. M. Ali Eslami, Danilo Rezende, Oriol Vinyals, Nando de Freitas
Abstract	Deep autoregressive models have shown state-of-the-art performance in density estimation for natural images on large-scale datasets such as ImageNet. However, such models require many thousands of gradient-based weight updates and unique image examples for training. Ideally, the models would rapidly learn visual concepts from only a handful of examples, similar to the manner in which humans learns across many vision tasks. In this paper, we show how 1) neural attention and 2) meta learning techniques can be used in combination with autoregressive models to enable effective few-shot density estimation. Our proposed modifications to PixelCNN result in state-of-the art few-shot density estimation on the Omniglot dataset. Furthermore, we visualize the learned attention policy and find that it learns intuitive algorithms for simple tasks such as image mirroring on ImageNet and handwriting on Omniglot without supervision. Finally, we extend the model to natural images and demonstrate few-shot image generation on the Stanford Online Products dataset.
Tasks	Density Estimation, Image Generation, Meta-Learning, Omniglot
Published	2017-10-27
URL	http://arxiv.org/abs/1710.10304v4
PDF	http://arxiv.org/pdf/1710.10304v4.pdf
PWC	https://paperswithcode.com/paper/few-shot-autoregressive-density-estimation
Repo
Framework

Robust Bayesian Optimization with Student-t Likelihood


Title	Robust Bayesian Optimization with Student-t Likelihood
Authors	Ruben Martinez-Cantin, Michael McCourt, Kevin Tee
Abstract	Bayesian optimization has recently attracted the attention of the automatic machine learning community for its excellent results in hyperparameter tuning. BO is characterized by the sample efficiency with which it can optimize expensive black-box functions. The efficiency is achieved in a similar fashion to the learning to learn methods: surrogate models (typically in the form of Gaussian processes) learn the target function and perform intelligent sampling. This surrogate model can be applied even in the presence of noise; however, as with most regression methods, it is very sensitive to outlier data. This can result in erroneous predictions and, in the case of BO, biased and inefficient exploration. In this work, we present a GP model that is robust to outliers which uses a Student-t likelihood to segregate outliers and robustly conduct Bayesian optimization. We present numerical results evaluating the proposed method in both artificial functions and real problems.
Tasks	Gaussian Processes
Published	2017-07-18
URL	http://arxiv.org/abs/1707.05729v1
PDF	http://arxiv.org/pdf/1707.05729v1.pdf
PWC	https://paperswithcode.com/paper/robust-bayesian-optimization-with-student-t
Repo
Framework