May 7, 2019

2890 words 14 mins read

Paper Group AWR 22

A U-statistic Approach to Hypothesis Testing for Structure Discovery in Undirected Graphical Models. Word Embeddings for the Construction Domain. Generative and Discriminative Voxel Modeling with Convolutional Neural Networks. MultiCol-SLAM - A Modular Real-Time Multi-Camera SLAM System. Value Iteration Networks. Training Deep Nets with Sublinear M …

A U-statistic Approach to Hypothesis Testing for Structure Discovery in Undirected Graphical Models


Title	A U-statistic Approach to Hypothesis Testing for Structure Discovery in Undirected Graphical Models
Authors	Wacha Bounliphone, Matthew Blaschko
Abstract	Structure discovery in graphical models is the determination of the topology of a graph that encodes conditional independence properties of the joint distribution of all variables in the model. For some class of probability distributions, an edge between two variables is present if and only if the corresponding entry in the precision matrix is non-zero. For a finite sample estimate of the precision matrix, entries close to zero may be due to low sample effects, or due to an actual association between variables; these two cases are not readily distinguishable. %Fisher provided a hypothesis test based on a parametric approximation to the distribution of an entry in the precision matrix of a Gaussian distribution, but this may not provide valid upper bounds on $p$-values for non-Gaussian distributions. Many related works on this topic consider potentially restrictive distributional or sparsity assumptions that may not apply to a data sample of interest, and direct estimation of the uncertainty of an estimate of the precision matrix for general distributions remains challenging. Consequently, we make use of results for $U$-statistics and apply them to the covariance matrix. By probabilistically bounding the distortion of the covariance matrix, we can apply Weyl’s theorem to bound the distortion of the precision matrix, yielding a conservative, but sound test threshold for a much wider class of distributions than considered in previous works. The resulting test enables one to answer with statistical significance whether an edge is present in the graph, and convergence results are known for a wide range of distributions. The computational complexities is linear in the sample size enabling the application of the test to large data samples for which computation time becomes a limiting factor. We experimentally validate the correctness and scalability of the test on multivariate distributions for which the distributional assumptions of competing tests result in underestimates of the false positive ratio. By contrast, the proposed test remains sound, promising to be a useful tool for hypothesis testing for diverse real-world problems.
Tasks
Published	2016-04-06
URL	http://arxiv.org/abs/1604.01733v1
PDF	http://arxiv.org/pdf/1604.01733v1.pdf
PWC	https://paperswithcode.com/paper/a-u-statistic-approach-to-hypothesis-testing
Repo	https://github.com/wbounliphone/Ustatistics_Approach_For_SD
Framework	none

Word Embeddings for the Construction Domain


Title	Word Embeddings for the Construction Domain
Authors	Antoine J. -P. Tixier, Michalis Vazirgiannis, Matthew R. Hallowell
Abstract	We introduce word vectors for the construction domain. Our vectors were obtained by running word2vec on an 11M-word corpus that we created from scratch by leveraging freely-accessible online sources of construction-related text. We first explore the embedding space and show that our vectors capture meaningful construction-specific concepts. We then evaluate the performance of our vectors against that of ones trained on a 100B-word corpus (Google News) within the framework of an injury report classification task. Without any parameter tuning, our embeddings give competitive results, and outperform the Google News vectors in many cases. Using a keyword-based compression of the reports also leads to a significant speed-up with only a limited loss in performance. We release our corpus and the data set we created for the classification task as publicly available, in the hope that they will be used by future studies for benchmarking and building on our work.
Tasks	Word Embeddings
Published	2016-10-28
URL	http://arxiv.org/abs/1610.09333v1
PDF	http://arxiv.org/pdf/1610.09333v1.pdf
PWC	https://paperswithcode.com/paper/word-embeddings-for-the-construction-domain
Repo	https://github.com/Tixierae/WECD
Framework	none

Generative and Discriminative Voxel Modeling with Convolutional Neural Networks


Title	Generative and Discriminative Voxel Modeling with Convolutional Neural Networks
Authors	Andrew Brock, Theodore Lim, J. M. Ritchie, Nick Weston
Abstract	When working with three-dimensional data, choice of representation is key. We explore voxel-based models, and present evidence for the viability of voxellated representations in applications including shape modeling and object classification. Our key contributions are methods for training voxel-based variational autoencoders, a user interface for exploring the latent space learned by the autoencoder, and a deep convolutional neural network architecture for object classification. We address challenges unique to voxel-based representations, and empirically evaluate our models on the ModelNet benchmark, where we demonstrate a 51.5% relative improvement in the state of the art for object classification.
Tasks	Object Classification
Published	2016-08-15
URL	http://arxiv.org/abs/1608.04236v2
PDF	http://arxiv.org/pdf/1608.04236v2.pdf
PWC	https://paperswithcode.com/paper/generative-and-discriminative-voxel-modeling
Repo	https://github.com/CPUFronz/CapsVoxGAN
Framework	pytorch

MultiCol-SLAM - A Modular Real-Time Multi-Camera SLAM System


Title	MultiCol-SLAM - A Modular Real-Time Multi-Camera SLAM System
Authors	Steffen Urban, Stefan Hinz
Abstract	The basis for most vision based applications like robotics, self-driving cars and potentially augmented and virtual reality is a robust, continuous estimation of the position and orientation of a camera system w.r.t the observed environment (scene). In recent years many vision based systems that perform simultaneous localization and mapping (SLAM) have been presented and released as open source. In this paper, we extend and improve upon a state-of-the-art SLAM to make it applicable to arbitrary, rigidly coupled multi-camera systems (MCS) using the MultiCol model. In addition, we include a performance evaluation on accurate ground truth and compare the robustness of the proposed method to a single camera version of the SLAM system. An open source implementation of the proposed multi-fisheye camera SLAM system can be found on-line https://github.com/urbste/MultiCol-SLAM.
Tasks	Self-Driving Cars, Simultaneous Localization and Mapping
Published	2016-10-24
URL	http://arxiv.org/abs/1610.07336v1
PDF	http://arxiv.org/pdf/1610.07336v1.pdf
PWC	https://paperswithcode.com/paper/multicol-slam-a-modular-real-time-multi
Repo	https://github.com/ns15417/multicol-slam2
Framework	none

Value Iteration Networks


Title	Value Iteration Networks
Authors	Aviv Tamar, Yi Wu, Garrett Thomas, Sergey Levine, Pieter Abbeel
Abstract	We introduce the value iteration network (VIN): a fully differentiable neural network with a `planning module’ embedded within. VINs can learn to plan, and are suitable for predicting outcomes that involve planning-based reasoning, such as policies for reinforcement learning. Key to our approach is a novel differentiable approximation of the value-iteration algorithm, which can be represented as a convolutional neural network, and trained end-to-end using standard backpropagation. We evaluate VIN based policies on discrete and continuous path-planning domains, and on a natural-language based search task. We show that by learning an explicit planning computation, VIN policies generalize better to new, unseen domains. \|
Tasks
Published	2016-02-09
URL	http://arxiv.org/abs/1602.02867v4
PDF	http://arxiv.org/pdf/1602.02867v4.pdf
PWC	https://paperswithcode.com/paper/value-iteration-networks
Repo	https://github.com/TheAbhiKumar/tensorflow-value-iteration-networks
Framework	tf

Training Deep Nets with Sublinear Memory Cost


Title	Training Deep Nets with Sublinear Memory Cost
Authors	Tianqi Chen, Bing Xu, Chiyuan Zhang, Carlos Guestrin
Abstract	We propose a systematic approach to reduce the memory consumption of deep neural network training. Specifically, we design an algorithm that costs O(sqrt(n)) memory to train a n layer network, with only the computational cost of an extra forward pass per mini-batch. As many of the state-of-the-art models hit the upper bound of the GPU memory, our algorithm allows deeper and more complex models to be explored, and helps advance the innovations in deep learning research. We focus on reducing the memory cost to store the intermediate feature maps and gradients during training. Computation graph analysis is used for automatic in-place operation and memory sharing optimizations. We show that it is possible to trade computation for memory - giving a more memory efficient training algorithm with a little extra computation cost. In the extreme case, our analysis also shows that the memory consumption can be reduced to O(log n) with as little as O(n log n) extra cost for forward computation. Our experiments show that we can reduce the memory cost of a 1,000-layer deep residual network from 48G to 7G with only 30 percent additional running time cost on ImageNet problems. Similarly, significant memory cost reduction is observed in training complex recurrent neural networks on very long sequences.
Tasks
Published	2016-04-21
URL	http://arxiv.org/abs/1604.06174v2
PDF	http://arxiv.org/pdf/1604.06174v2.pdf
PWC	https://paperswithcode.com/paper/training-deep-nets-with-sublinear-memory-cost
Repo	https://github.com/dmlc/mxnet-memonger
Framework	tf

Distributed Optimization of Multi-Class SVMs


Title	Distributed Optimization of Multi-Class SVMs
Authors	Maximilian Alber, Julian Zimmert, Urun Dogan, Marius Kloft
Abstract	Training of one-vs.-rest SVMs can be parallelized over the number of classes in a straight forward way. Given enough computational resources, one-vs.-rest SVMs can thus be trained on data involving a large number of classes. The same cannot be stated, however, for the so-called all-in-one SVMs, which require solving a quadratic program of size quadratically in the number of classes. We develop distributed algorithms for two all-in-one SVM formulations (Lee et al. and Weston and Watkins) that parallelize the computation evenly over the number of classes. This allows us to compare these models to one-vs.-rest SVMs on unprecedented scale. The results indicate superior accuracy on text classification data.
Tasks	Distributed Optimization, Text Classification
Published	2016-11-25
URL	http://arxiv.org/abs/1611.08480v2
PDF	http://arxiv.org/pdf/1611.08480v2.pdf
PWC	https://paperswithcode.com/paper/distributed-optimization-of-multi-class-svms
Repo	https://github.com/albermax/xcsvm
Framework	none

Generative Models and Model Criticism via Optimized Maximum Mean Discrepancy


Title	Generative Models and Model Criticism via Optimized Maximum Mean Discrepancy
Authors	Dougal J. Sutherland, Hsiao-Yu Tung, Heiko Strathmann, Soumyajit De, Aaditya Ramdas, Alex Smola, Arthur Gretton
Abstract	We propose a method to optimize the representation and distinguishability of samples from two probability distributions, by maximizing the estimated power of a statistical test based on the maximum mean discrepancy (MMD). This optimized MMD is applied to the setting of unsupervised learning by generative adversarial networks (GAN), in which a model attempts to generate realistic samples, and a discriminator attempts to tell these apart from data samples. In this context, the MMD may be used in two roles: first, as a discriminator, either directly on the samples, or on features of the samples. Second, the MMD can be used to evaluate the performance of a generative model, by testing the model’s samples against a reference data set. In the latter role, the optimized MMD is particularly helpful, as it gives an interpretable indication of how the model and data distributions differ, even in cases where individual model samples are not easily distinguished either by eye or by classifier.
Tasks
Published	2016-11-14
URL	https://arxiv.org/abs/1611.04488v5
PDF	https://arxiv.org/pdf/1611.04488v5.pdf
PWC	https://paperswithcode.com/paper/generative-models-and-model-criticism-via
Repo	https://github.com/dougalsutherland/opt-mmd
Framework	tf

The xyz algorithm for fast interaction search in high-dimensional data


Title	The xyz algorithm for fast interaction search in high-dimensional data
Authors	Gian-Andrea Thanei, Nicolai Meinshausen, Rajen D. Shah
Abstract	When performing regression on a dataset with $p$ variables, it is often of interest to go beyond using main linear effects and include interactions as products between individual variables. For small-scale problems, these interactions can be computed explicitly but this leads to a computational complexity of at least $\mathcal{O}(p^2)$ if done naively. This cost can be prohibitive if $p$ is very large. We introduce a new randomised algorithm that is able to discover interactions with high probability and under mild conditions has a runtime that is subquadratic in $p$. We show that strong interactions can be discovered in almost linear time, whilst finding weaker interactions requires $\mathcal{O}(p^\alpha)$ operations for $1 < \alpha < 2$ depending on their strength. The underlying idea is to transform interaction search into a closestpair problem which can be solved efficiently in subquadratic time. The algorithm is called $\mathit{xyz}$ and is implemented in the language R. We demonstrate its efficiency for application to genome-wide association studies, where more than $10^{11}$ interactions can be screened in under $280$ seconds with a single-core $1.2$ GHz CPU.
Tasks
Published	2016-10-17
URL	http://arxiv.org/abs/1610.05108v4
PDF	http://arxiv.org/pdf/1610.05108v4.pdf
PWC	https://paperswithcode.com/paper/the-xyz-algorithm-for-fast-interaction-search
Repo	https://github.com/gathanei/xyz
Framework	none

Deep Spiking Networks


Title	Deep Spiking Networks
Authors	Peter O’Connor, Max Welling
Abstract	We introduce an algorithm to do backpropagation on a spiking network. Our network is “spiking” in the sense that our neurons accumulate their activation into a potential over time, and only send out a signal (a “spike”) when this potential crosses a threshold and the neuron is reset. Neurons only update their states when receiving signals from other neurons. Total computation of the network thus scales with the number of spikes caused by an input rather than network size. We show that the spiking Multi-Layer Perceptron behaves identically, during both prediction and training, to a conventional deep network of rectified-linear units, in the limiting case where we run the spiking network for a long time. We apply this architecture to a conventional classification problem (MNIST) and achieve performance very close to that of a conventional Multi-Layer Perceptron with the same architecture. Our network is a natural architecture for learning based on streaming event-based data, and is a stepping stone towards using spiking neural networks to learn efficiently on streaming data.
Tasks
Published	2016-02-26
URL	http://arxiv.org/abs/1602.08323v2
PDF	http://arxiv.org/pdf/1602.08323v2.pdf
PWC	https://paperswithcode.com/paper/deep-spiking-networks
Repo	https://github.com/petered/spiking-mlp
Framework	none

A Gentle Tutorial of Recurrent Neural Network with Error Backpropagation


Title	A Gentle Tutorial of Recurrent Neural Network with Error Backpropagation
Authors	Gang Chen
Abstract	We describe recurrent neural networks (RNNs), which have attracted great attention on sequential tasks, such as handwriting recognition, speech recognition and image to text. However, compared to general feedforward neural networks, RNNs have feedback loops, which makes it a little hard to understand the backpropagation step. Thus, we focus on basics, especially the error backpropagation to compute gradients with respect to model parameters. Further, we go into detail on how error backpropagation algorithm is applied on long short-term memory (LSTM) by unfolding the memory unit.
Tasks	Speech Recognition
Published	2016-10-08
URL	http://arxiv.org/abs/1610.02583v3
PDF	http://arxiv.org/pdf/1610.02583v3.pdf
PWC	https://paperswithcode.com/paper/a-gentle-tutorial-of-recurrent-neural-network
Repo	https://github.com/disooqi/DNN
Framework	none

A Discriminatively Learned CNN Embedding for Person Re-identification


Title	A Discriminatively Learned CNN Embedding for Person Re-identification
Authors	Zhedong Zheng, Liang Zheng, Yi Yang
Abstract	We revisit two popular convolutional neural networks (CNN) in person re-identification (re-ID), i.e, verification and classification models. The two models have their respective advantages and limitations due to different loss functions. In this paper, we shed light on how to combine the two models to learn more discriminative pedestrian descriptors. Specifically, we propose a new siamese network that simultaneously computes identification loss and verification loss. Given a pair of training images, the network predicts the identities of the two images and whether they belong to the same identity. Our network learns a discriminative embedding and a similarity measurement at the same time, thus making full usage of the annotations. Albeit simple, the learned embedding improves the state-of-the-art performance on two public person re-ID benchmarks. Further, we show our architecture can also be applied in image retrieval.
Tasks	Image Retrieval, Person Re-Identification
Published	2016-11-17
URL	http://arxiv.org/abs/1611.05666v2
PDF	http://arxiv.org/pdf/1611.05666v2.pdf
PWC	https://paperswithcode.com/paper/a-discriminatively-learned-cnn-embedding-for
Repo	https://github.com/ahangchen/rank-reid
Framework	pytorch

Cloud Dictionary: Sparse Coding and Modeling for Point Clouds


Title	Cloud Dictionary: Sparse Coding and Modeling for Point Clouds
Authors	Or Litany, Tal Remez, Alex Bronstein
Abstract	With the development of range sensors such as LIDAR and time-of-flight cameras, 3D point cloud scans have become ubiquitous in computer vision applications, the most prominent ones being gesture recognition and autonomous driving. Parsimony-based algorithms have shown great success on images and videos where data points are sampled on a regular Cartesian grid. We propose an adaptation of these techniques to irregularly sampled signals by using continuous dictionaries. We present an example application in the form of point cloud denoising.
Tasks	Autonomous Driving, Denoising, Gesture Recognition
Published	2016-12-15
URL	http://arxiv.org/abs/1612.04956v2
PDF	http://arxiv.org/pdf/1612.04956v2.pdf
PWC	https://paperswithcode.com/paper/cloud-dictionary-sparse-coding-and-modeling
Repo	https://github.com/PeterBeben/continuous-K-SVD
Framework	none

Recurrent Neural Networks With Limited Numerical Precision


Title	Recurrent Neural Networks With Limited Numerical Precision
Authors	Joachim Ott, Zhouhan Lin, Ying Zhang, Shih-Chii Liu, Yoshua Bengio
Abstract	Recurrent Neural Networks (RNNs) produce state-of-art performance on many machine learning tasks but their demand on resources in terms of memory and computational power are often high. Therefore, there is a great interest in optimizing the computations performed with these models especially when considering development of specialized low-power hardware for deep networks. One way of reducing the computational needs is to limit the numerical precision of the network weights and biases, and this will be addressed for the case of RNNs. We present results from the use of different stochastic and deterministic reduced precision training methods applied to two major RNN types, which are then tested on three datasets. The results show that the stochastic and deterministic ternarization, pow2- ternarization, and exponential quantization methods gave rise to low-precision RNNs that produce similar and even higher accuracy on certain datasets, therefore providing a path towards training more efficient implementations of RNNs in specialized hardware.
Tasks	Quantization
Published	2016-11-21
URL	http://arxiv.org/abs/1611.07065v2
PDF	http://arxiv.org/pdf/1611.07065v2.pdf
PWC	https://paperswithcode.com/paper/recurrent-neural-networks-with-limited
Repo	https://github.com/ottj/QuantizedRNN
Framework	none

Dynamic Hierarchical Dirichlet Process for Abnormal Behaviour Detection in Video


Title	Dynamic Hierarchical Dirichlet Process for Abnormal Behaviour Detection in Video
Authors	Olga Isupova, Danil Kuzin, Lyudmila Mihaylova
Abstract	This paper proposes a novel dynamic Hierarchical Dirichlet Process topic model that considers the dependence between successive observations. Conventional posterior inference algorithms for this kind of models require processing of the whole data through several passes. It is computationally intractable for massive or sequential data. We design the batch and online inference algorithms, based on the Gibbs sampling, for the proposed model. It allows to process sequential data, incrementally updating the model by a new observation. The model is applied to abnormal behaviour detection in video sequences. A new abnormality measure is proposed for decision making. The proposed method is compared with the method based on the non- dynamic Hierarchical Dirichlet Process, for which we also derive the online Gibbs sampler and the abnormality measure. The results with synthetic and real data show that the consideration of the dynamics in a topic model improves the classification performance for abnormal behaviour detection.
Tasks	Decision Making
Published	2016-06-27
URL	http://arxiv.org/abs/1606.08476v1
PDF	http://arxiv.org/pdf/1606.08476v1.pdf
PWC	https://paperswithcode.com/paper/dynamic-hierarchical-dirichlet-process-for
Repo	https://github.com/OlgaIsupova/dynamic-hdp
Framework	none