July 27, 2019

2769 words 13 mins read

Paper Group ANR 699

Inference via low-dimensional couplings. Scaling up the Automatic Statistician: Scalable Structure Discovery using Gaussian Processes. Superpixels Based Marker Tracking Vs. Hue Thresholding In Rodent Biomechanics Application. Exploring Heritability of Functional Brain Networks with Inexact Graph Matching. Nearest-Neighbor Sample Compression: Effici …

Inference via low-dimensional couplings


Title	Inference via low-dimensional couplings
Authors	Alessio Spantini, Daniele Bigoni, Youssef Marzouk
Abstract	We investigate the low-dimensional structure of deterministic transformations between random variables, i.e., transport maps between probability measures. In the context of statistics and machine learning, these transformations can be used to couple a tractable “reference” measure (e.g., a standard Gaussian) with a target measure of interest. Direct simulation from the desired measure can then be achieved by pushing forward reference samples through the map. Yet characterizing such a map—e.g., representing and evaluating it—grows challenging in high dimensions. The central contribution of this paper is to establish a link between the Markov properties of the target measure and the existence of low-dimensional couplings, induced by transport maps that are sparse and/or decomposable. Our analysis not only facilitates the construction of transformations in high-dimensional settings, but also suggests new inference methodologies for continuous non-Gaussian graphical models. For instance, in the context of nonlinear state-space models, we describe new variational algorithms for filtering, smoothing, and sequential parameter inference. These algorithms can be understood as the natural generalization—to the non-Gaussian case—of the square-root Rauch-Tung-Striebel Gaussian smoother.
Tasks
Published	2017-03-17
URL	http://arxiv.org/abs/1703.06131v4
PDF	http://arxiv.org/pdf/1703.06131v4.pdf
PWC	https://paperswithcode.com/paper/inference-via-low-dimensional-couplings
Repo
Framework

Scaling up the Automatic Statistician: Scalable Structure Discovery using Gaussian Processes


Title	Scaling up the Automatic Statistician: Scalable Structure Discovery using Gaussian Processes
Authors	Hyunjik Kim, Yee Whye Teh
Abstract	Automating statistical modelling is a challenging problem in artificial intelligence. The Automatic Statistician takes a first step in this direction, by employing a kernel search algorithm with Gaussian Processes (GP) to provide interpretable statistical models for regression problems. However this does not scale due to its $O(N^3)$ running time for the model selection. We propose Scalable Kernel Composition (SKC), a scalable kernel search algorithm that extends the Automatic Statistician to bigger data sets. In doing so, we derive a cheap upper bound on the GP marginal likelihood that sandwiches the marginal likelihood with the variational lower bound . We show that the upper bound is significantly tighter than the lower bound and thus useful for model selection.
Tasks	Gaussian Processes, Model Selection
Published	2017-06-08
URL	http://arxiv.org/abs/1706.02524v2
PDF	http://arxiv.org/pdf/1706.02524v2.pdf
PWC	https://paperswithcode.com/paper/scaling-up-the-automatic-statistician
Repo
Framework

Superpixels Based Marker Tracking Vs. Hue Thresholding In Rodent Biomechanics Application


Title	Superpixels Based Marker Tracking Vs. Hue Thresholding In Rodent Biomechanics Application
Authors	Omid Haji Maghsoudi, Annie Vahedipour Tabrizi, Benjamin Robertson, Andrew Spence
Abstract	Examining locomotion has improved our basic understanding of motor control and aided in treating motor impairment. Mice and rats are premier models of human disease and increasingly the model systems of choice for basic neuroscience. High frame rates (250 Hz) are needed to quantify the kinematics of these running rodents. Manual tracking, especially for multiple markers, becomes time-consuming and impossible for large sample sizes. Therefore, the need for automatic segmentation of these markers has grown in recent years. We propose two methods to segment and track these markers: first, using SLIC superpixels segmentation with a tracker based on position, speed, shape, and color information of the segmented region in the previous frame; second, using a thresholding on hue channel following up with the same tracker. The comparison showed that the SLIC superpixels method was superior because the segmentation was more reliable and based on both color and spatial information.
Tasks
Published	2017-10-17
URL	http://arxiv.org/abs/1710.06473v4
PDF	http://arxiv.org/pdf/1710.06473v4.pdf
PWC	https://paperswithcode.com/paper/superpixels-based-marker-tracking-vs-hue
Repo
Framework

Exploring Heritability of Functional Brain Networks with Inexact Graph Matching


Title	Exploring Heritability of Functional Brain Networks with Inexact Graph Matching
Authors	Sofia Ira Ktena, Salim Arslan, Sarah Parisot, Daniel Rueckert
Abstract	Data-driven brain parcellations aim to provide a more accurate representation of an individual’s functional connectivity, since they are able to capture individual variability that arises due to development or disease. This renders comparisons between the emerging brain connectivity networks more challenging, since correspondences between their elements are not preserved. Unveiling these correspondences is of major importance to keep track of local functional connectivity changes. We propose a novel method based on graph edit distance for the comparison of brain graphs directly in their domain, that can accurately reflect similarities between individual networks while providing the network element correspondences. This method is validated on a dataset of 116 twin subjects provided by the Human Connectome Project.
Tasks	Graph Matching
Published	2017-03-29
URL	http://arxiv.org/abs/1703.10062v1
PDF	http://arxiv.org/pdf/1703.10062v1.pdf
PWC	https://paperswithcode.com/paper/exploring-heritability-of-functional-brain
Repo
Framework

Nearest-Neighbor Sample Compression: Efficiency, Consistency, Infinite Dimensions


Title	Nearest-Neighbor Sample Compression: Efficiency, Consistency, Infinite Dimensions
Authors	Aryeh Kontorovich, Sivan Sabato, Roi Weiss
Abstract	We examine the Bayes-consistency of a recently proposed 1-nearest-neighbor-based multiclass learning algorithm. This algorithm is derived from sample compression bounds and enjoys the statistical advantages of tight, fully empirical generalization bounds, as well as the algorithmic advantages of a faster runtime and memory savings. We prove that this algorithm is strongly Bayes-consistent in metric spaces with finite doubling dimension — the first consistency result for an efficient nearest-neighbor sample compression scheme. Rather surprisingly, we discover that this algorithm continues to be Bayes-consistent even in a certain infinite-dimensional setting, in which the basic measure-theoretic conditions on which classic consistency proofs hinge are violated. This is all the more surprising, since it is known that $k$-NN is not Bayes-consistent in this setting. We pose several challenging open problems for future research.
Tasks
Published	2017-05-23
URL	https://arxiv.org/abs/1705.08184v3
PDF	https://arxiv.org/pdf/1705.08184v3.pdf
PWC	https://paperswithcode.com/paper/nearest-neighbor-sample-compression
Repo
Framework

The Relative Performance of Ensemble Methods with Deep Convolutional Neural Networks for Image Classification


Title	The Relative Performance of Ensemble Methods with Deep Convolutional Neural Networks for Image Classification
Authors	Cheng Ju, Aurélien Bibaut, Mark J. van der Laan
Abstract	Artificial neural networks have been successfully applied to a variety of machine learning tasks, including image recognition, semantic segmentation, and machine translation. However, few studies fully investigated ensembles of artificial neural networks. In this work, we investigated multiple widely used ensemble methods, including unweighted averaging, majority voting, the Bayes Optimal Classifier, and the (discrete) Super Learner, for image recognition tasks, with deep neural networks as candidate algorithms. We designed several experiments, with the candidate algorithms being the same network structure with different model checkpoints within a single training process, networks with same structure but trained multiple times stochastically, and networks with different structure. In addition, we further studied the over-confidence phenomenon of the neural networks, as well as its impact on the ensemble methods. Across all of our experiments, the Super Learner achieved best performance among all the ensemble methods in this study.
Tasks	Image Classification, Machine Translation, Semantic Segmentation
Published	2017-04-05
URL	http://arxiv.org/abs/1704.01664v1
PDF	http://arxiv.org/pdf/1704.01664v1.pdf
PWC	https://paperswithcode.com/paper/the-relative-performance-of-ensemble-methods
Repo
Framework

Zeroth-Order Online Alternating Direction Method of Multipliers: Convergence Analysis and Applications


Title	Zeroth-Order Online Alternating Direction Method of Multipliers: Convergence Analysis and Applications
Authors	Sijia Liu, Jie Chen, Pin-Yu Chen, Alfred O. Hero
Abstract	In this paper, we design and analyze a new zeroth-order online algorithm, namely, the zeroth-order online alternating direction method of multipliers (ZOO-ADMM), which enjoys dual advantages of being gradient-free operation and employing the ADMM to accommodate complex structured regularizers. Compared to the first-order gradient-based online algorithm, we show that ZOO-ADMM requires $\sqrt{m}$ times more iterations, leading to a convergence rate of $O(\sqrt{m}/\sqrt{T})$, where $m$ is the number of optimization variables, and $T$ is the number of iterations. To accelerate ZOO-ADMM, we propose two minibatch strategies: gradient sample averaging and observation averaging, resulting in an improved convergence rate of $O(\sqrt{1+q^{-1}m}/\sqrt{T})$, where $q$ is the minibatch size. In addition to convergence analysis, we also demonstrate ZOO-ADMM to applications in signal processing, statistics, and machine learning.
Tasks
Published	2017-10-21
URL	http://arxiv.org/abs/1710.07804v2
PDF	http://arxiv.org/pdf/1710.07804v2.pdf
PWC	https://paperswithcode.com/paper/zeroth-order-online-alternating-direction
Repo
Framework

Beyond the Pixel-Wise Loss for Topology-Aware Delineation


Title	Beyond the Pixel-Wise Loss for Topology-Aware Delineation
Authors	Agata Mosinska, Pablo Marquez-Neila, Mateusz Kozinski, Pascal Fua
Abstract	Delineation of curvilinear structures is an important problem in Computer Vision with multiple practical applications. With the advent of Deep Learning, many current approaches on automatic delineation have focused on finding more powerful deep architectures, but have continued using the habitual pixel-wise losses such as binary cross-entropy. In this paper we claim that pixel-wise losses alone are unsuitable for this problem because of their inability to reflect the topological impact of mistakes in the final prediction. We propose a new loss term that is aware of the higher-order topological features of linear structures. We also introduce a refinement pipeline that iteratively applies the same model over the previous delineation to refine the predictions at each step while keeping the number of parameters and the complexity of the model constant. When combined with the standard pixel-wise loss, both our new loss term and our iterative refinement boost the quality of the predicted delineations, in some cases almost doubling the accuracy as compared to the same classifier trained with the binary cross-entropy alone. We show that our approach outperforms state-of-the-art methods on a wide range of data, from microscopy to aerial images.
Tasks
Published	2017-12-06
URL	http://arxiv.org/abs/1712.02190v1
PDF	http://arxiv.org/pdf/1712.02190v1.pdf
PWC	https://paperswithcode.com/paper/beyond-the-pixel-wise-loss-for-topology-aware
Repo
Framework

Robust Depth-based Person Re-identification


Title	Robust Depth-based Person Re-identification
Authors	Ancong Wu, Wei-Shi Zheng, Jianhuang Lai
Abstract	Person re-identification (re-id) aims to match people across non-overlapping camera views. So far the RGB-based appearance is widely used in most existing works. However, when people appeared in extreme illumination or changed clothes, the RGB appearance-based re-id methods tended to fail. To overcome this problem, we propose to exploit depth information to provide more invariant body shape and skeleton information regardless of illumination and color change. More specifically, we exploit depth voxel covariance descriptor and further propose a locally rotation invariant depth shape descriptor called Eigen-depth feature to describe pedestrian body shape. We prove that the distance between any two covariance matrices on the Riemannian manifold is equivalent to the Euclidean distance between the corresponding Eigen-depth features. Furthermore, we propose a kernelized implicit feature transfer scheme to estimate Eigen-depth feature implicitly from RGB image when depth information is not available. We find that combining the estimated depth features with RGB-based appearance features can sometimes help to better reduce visual ambiguities of appearance features caused by illumination and similar clothes. The effectiveness of our models was validated on publicly available depth pedestrian datasets as compared to related methods for person re-identification.
Tasks	Person Re-Identification
Published	2017-03-28
URL	http://arxiv.org/abs/1703.09474v1
PDF	http://arxiv.org/pdf/1703.09474v1.pdf
PWC	https://paperswithcode.com/paper/robust-depth-based-person-re-identification
Repo
Framework

Unsupervised Morphological Expansion of Small Datasets for Improving Word Embeddings


Title	Unsupervised Morphological Expansion of Small Datasets for Improving Word Embeddings
Authors	Syed Sarfaraz Akhtar, Arihant Gupta, Avijit Vajpayee, Arjit Srivastava, Manish Shrivastava
Abstract	We present a language independent, unsupervised method for building word embeddings using morphological expansion of text. Our model handles the problem of data sparsity and yields improved word embeddings by relying on training word embeddings on artificially generated sentences. We evaluate our method using small sized training sets on eleven test sets for the word similarity task across seven languages. Further, for English, we evaluated the impacts of our approach using a large training set on three standard test sets. Our method improved results across all languages.
Tasks	Word Embeddings
Published	2017-11-15
URL	http://arxiv.org/abs/1711.05678v1
PDF	http://arxiv.org/pdf/1711.05678v1.pdf
PWC	https://paperswithcode.com/paper/unsupervised-morphological-expansion-of-small
Repo
Framework

Sequential Lifted Bayesian Filtering in Multiset Rewriting Systems


Title	Sequential Lifted Bayesian Filtering in Multiset Rewriting Systems
Authors	Max Schröder, Stefan Lüdtke, Sebastian Bader, Frank Krüger, Thomas Kirste
Abstract	Bayesian Filtering for plan and activity recognition is challenging for scenarios that contain many observation equivalent entities (i.e. entities that produce the same observations). This is due to the combinatorial explosion in the number of hypotheses that need to be tracked. However, this class of problems exhibits a certain symmetry that can be exploited for state space representation and inference. We analyze current state of the art methods and find that none of them completely fits the requirements arising in this problem class. We sketch a novel inference algorithm that provides a solution by incorporating concepts from Lifted Inference algorithms, Probabilistic Multiset Rewriting Systems, and Computational State Space Models. Two experiments confirm that this novel algorithm has the potential to perform efficient probabilistic inference on this problem class.
Tasks	Activity Recognition
Published	2017-07-20
URL	http://arxiv.org/abs/1707.06446v2
PDF	http://arxiv.org/pdf/1707.06446v2.pdf
PWC	https://paperswithcode.com/paper/sequential-lifted-bayesian-filtering-in
Repo
Framework

Information Extraction in Illicit Domains


Title	Information Extraction in Illicit Domains
Authors	Mayank Kejriwal, Pedro Szekely
Abstract	Extracting useful entities and attribute values from illicit domains such as human trafficking is a challenging problem with the potential for widespread social impact. Such domains employ atypical language models, have `long tails’ and suffer from the problem of concept drift. In this paper, we propose a lightweight, feature-agnostic Information Extraction (IE) paradigm specifically designed for such domains. Our approach uses raw, unlabeled text from an initial corpus, and a few (12-120) seed annotations per domain-specific attribute, to learn robust IE models for unobserved pages and websites. Empirically, we demonstrate that our approach can outperform feature-centric Conditional Random Field baselines by over 18% F-Measure on five annotated sets of real-world human trafficking datasets in both low-supervision and high-supervision settings. We also show that our approach is demonstrably robust to concept drift, and can be efficiently bootstrapped even in a serial computing environment. \|
Tasks
Published	2017-03-09
URL	http://arxiv.org/abs/1703.03097v1
PDF	http://arxiv.org/pdf/1703.03097v1.pdf
PWC	https://paperswithcode.com/paper/information-extraction-in-illicit-domains
Repo
Framework

Linguistic Knowledge as Memory for Recurrent Neural Networks


Title	Linguistic Knowledge as Memory for Recurrent Neural Networks
Authors	Bhuwan Dhingra, Zhilin Yang, William W. Cohen, Ruslan Salakhutdinov
Abstract	Training recurrent neural networks to model long term dependencies is difficult. Hence, we propose to use external linguistic knowledge as an explicit signal to inform the model which memories it should utilize. Specifically, external knowledge is used to augment a sequence with typed edges between arbitrarily distant elements, and the resulting graph is decomposed into directed acyclic subgraphs. We introduce a model that encodes such graphs as explicit memory in recurrent neural networks, and use it to model coreference relations in text. We apply our model to several text comprehension tasks and achieve new state-of-the-art results on all considered benchmarks, including CNN, bAbi, and LAMBADA. On the bAbi QA tasks, our model solves 15 out of the 20 tasks with only 1000 training examples per task. Analysis of the learned representations further demonstrates the ability of our model to encode fine-grained entity information across a document.
Tasks	Reading Comprehension
Published	2017-03-07
URL	http://arxiv.org/abs/1703.02620v1
PDF	http://arxiv.org/pdf/1703.02620v1.pdf
PWC	https://paperswithcode.com/paper/linguistic-knowledge-as-memory-for-recurrent
Repo
Framework

Robust Seed Mask Generation for Interactive Image Segmentation


Title	Robust Seed Mask Generation for Interactive Image Segmentation
Authors	Mario Amrehn, Stefan Steidl, Markus Kowarschik, Andreas Maier
Abstract	In interactive medical image segmentation, anatomical structures are extracted from reconstructed volumetric images. The first iterations of user interaction traditionally consist of drawing pictorial hints as an initial estimate of the object to extract. Only after this time consuming first phase, the efficient selective refinement of current segmentation results begins. Erroneously labeled seeds, especially near the border of the object, are challenging to detect and replace for a human and may substantially impact the overall segmentation quality. We propose an automatic seeding pipeline as well as a configuration based on saliency recognition, in order to skip the time-consuming initial interaction phase during segmentation. A median Dice score of 68.22% is reached before the first user interaction on the test data set with an error rate in seeding of only 0.088%.
Tasks	Medical Image Segmentation, Semantic Segmentation
Published	2017-11-20
URL	http://arxiv.org/abs/1711.07419v1
PDF	http://arxiv.org/pdf/1711.07419v1.pdf
PWC	https://paperswithcode.com/paper/robust-seed-mask-generation-for-interactive
Repo
Framework

Compressing DMA Engine: Leveraging Activation Sparsity for Training Deep Neural Networks


Title	Compressing DMA Engine: Leveraging Activation Sparsity for Training Deep Neural Networks
Authors	Minsoo Rhu, Mike O’Connor, Niladrish Chatterjee, Jeff Pool, Stephen W. Keckler
Abstract	Popular deep learning frameworks require users to fine-tune their memory usage so that the training data of a deep neural network (DNN) fits within the GPU physical memory. Prior work tries to address this restriction by virtualizing the memory usage of DNNs, enabling both CPU and GPU memory to be utilized for memory allocations. Despite its merits, virtualizing memory can incur significant performance overheads when the time needed to copy data back and forth from CPU memory is higher than the latency to perform the computations required for DNN forward and backward propagation. We introduce a high-performance virtualization strategy based on a “compressing DMA engine” (cDMA) that drastically reduces the size of the data structures that are targeted for CPU-side allocations. The cDMA engine offers an average 2.6x (maximum 13.8x) compression ratio by exploiting the sparsity inherent in offloaded data, improving the performance of virtualized DNNs by an average 32% (maximum 61%).
Tasks
Published	2017-05-03
URL	http://arxiv.org/abs/1705.01626v1
PDF	http://arxiv.org/pdf/1705.01626v1.pdf
PWC	https://paperswithcode.com/paper/compressing-dma-engine-leveraging-activation
Repo
Framework