October 17, 2019

2962 words 14 mins read

Paper Group ANR 811

Unsupervised monocular stereo matching. SynonymNet: Multi-context Bilateral Matching for Entity Synonyms. SuperDepth: Self-Supervised, Super-Resolved Monocular Depth Estimation. Adaptive Stochastic Gradient Langevin Dynamics: Taming Convergence and Saddle Point Escape Time. A Tale of Three Probabilistic Families: Discriminative, Descriptive and Gen …

Unsupervised monocular stereo matching


Title	Unsupervised monocular stereo matching
Authors	Zhimin Zhang, Jianzhong Qiao, Shukuan Lin
Abstract	At present, deep learning has been applied more and more in monocular image depth estimation and has shown promising results. The current more ideal method for monocular depth estimation is the supervised learning based on ground truth depth, but this method requires an abundance of expensive ground truth depth as the supervised labels. Therefore, researchers began to work on unsupervised depth estimation methods. Although the accuracy of unsupervised depth estimation method is still lower than that of supervised method, it is a promising research direction. In this paper, Based on the experimental results that the stereo matching models outperforms monocular depth estimation models under the same unsupervised depth estimation model, we proposed an unsupervised monocular vision stereo matching method. In order to achieve the monocular stereo matching, we constructed two unsupervised deep convolution network models, one was to reconstruct the right view from the left view, and the other was to estimate the depth map using the reconstructed right view and the original left view. The two network models are piped together during the test phase. The output results of this method outperforms the current mainstream unsupervised depth estimation method in the challenging KITTI dataset.
Tasks	Depth Estimation, Monocular Depth Estimation, Stereo Matching, Stereo Matching Hand
Published	2018-12-31
URL	http://arxiv.org/abs/1812.11671v1
PDF	http://arxiv.org/pdf/1812.11671v1.pdf
PWC	https://paperswithcode.com/paper/unsupervised-monocular-stereo-matching
Repo
Framework

SynonymNet: Multi-context Bilateral Matching for Entity Synonyms


Title	SynonymNet: Multi-context Bilateral Matching for Entity Synonyms
Authors	Chenwei Zhang, Yaliang Li, Nan Du, Wei Fan, Philip S. Yu
Abstract	Being able to automatically discover synonymous entities from a large free-text corpus has transformative effects on structured knowledge discovery. Existing works either require structured annotations, or fail to incorporate context information effectively, which lower the efficiency of information usage. In this paper, we propose a framework for synonym discovery from free-text corpus without structured annotation. As one of the key components in synonym discovery, we introduce a novel neural network model SynonymNet to determine whether or not two given entities are synonym with each other. Instead of using entities features, SynonymNet makes use of multiple pieces of contexts in which the entity is mentioned, and compares the context-level similarity via a bilateral matching schema to determine synonymity. Experimental results demonstrate that the proposed model achieves state-of-the-art results on both generic and domain-specific synonym datasets: Wiki+Freebase, PubMed+UMLS and MedBook+MKG, with up to 4.16% improvement in terms of Area Under the Curve (AUC) and 3.19% in terms of Mean Average Precision (MAP) compare to the best baseline method.
Tasks
Published	2018-12-31
URL	http://arxiv.org/abs/1901.00056v1
PDF	http://arxiv.org/pdf/1901.00056v1.pdf
PWC	https://paperswithcode.com/paper/synonymnet-multi-context-bilateral-matching
Repo
Framework

SuperDepth: Self-Supervised, Super-Resolved Monocular Depth Estimation


Title	SuperDepth: Self-Supervised, Super-Resolved Monocular Depth Estimation
Authors	Sudeep Pillai, Rares Ambrus, Adrien Gaidon
Abstract	Recent techniques in self-supervised monocular depth estimation are approaching the performance of supervised methods, but operate in low resolution only. We show that high resolution is key towards high-fidelity self-supervised monocular depth prediction. Inspired by recent deep learning methods for Single-Image Super-Resolution, we propose a sub-pixel convolutional layer extension for depth super-resolution that accurately synthesizes high-resolution disparities from their corresponding low-resolution convolutional features. In addition, we introduce a differentiable flip-augmentation layer that accurately fuses predictions from the image and its horizontally flipped version, reducing the effect of left and right shadow regions generated in the disparity map due to occlusions. Both contributions provide significant performance gains over the state-of-the-art in self-supervised depth and pose estimation on the public KITTI benchmark. A video of our approach can be found at https://youtu.be/jKNgBeBMx0I.
Tasks	Depth Estimation, Image Super-Resolution, Monocular Depth Estimation, Pose Estimation, Super-Resolution
Published	2018-10-03
URL	http://arxiv.org/abs/1810.01849v1
PDF	http://arxiv.org/pdf/1810.01849v1.pdf
PWC	https://paperswithcode.com/paper/superdepth-self-supervised-super-resolved
Repo
Framework

Adaptive Stochastic Gradient Langevin Dynamics: Taming Convergence and Saddle Point Escape Time


Title	Adaptive Stochastic Gradient Langevin Dynamics: Taming Convergence and Saddle Point Escape Time
Authors	Hejian Sang, Jia Liu
Abstract	In this paper, we propose a new adaptive stochastic gradient Langevin dynamics (ASGLD) algorithmic framework and its two specialized versions, namely adaptive stochastic gradient (ASG) and adaptive gradient Langevin dynamics(AGLD), for non-convex optimization problems. All proposed algorithms can escape from saddle points with at most $O(\log d)$ iterations, which is nearly dimension-free. Further, we show that ASGLD and ASG converge to a local minimum with at most $O(\log d/\epsilon^4)$ iterations. Also, ASGLD with full gradients or ASGLD with a slowly linearly increasing batch size converge to a local minimum with iterations bounded by $O(\log d/\epsilon^2)$, which outperforms existing first-order methods.
Tasks
Published	2018-05-23
URL	http://arxiv.org/abs/1805.09416v1
PDF	http://arxiv.org/pdf/1805.09416v1.pdf
PWC	https://paperswithcode.com/paper/adaptive-stochastic-gradient-langevin
Repo
Framework

A Tale of Three Probabilistic Families: Discriminative, Descriptive and Generative Models


Title	A Tale of Three Probabilistic Families: Discriminative, Descriptive and Generative Models
Authors	Ying Nian Wu, Ruiqi Gao, Tian Han, Song-Chun Zhu
Abstract	The pattern theory of Grenander is a mathematical framework where patterns are represented by probability models on random variables of algebraic structures. In this paper, we review three families of probability models, namely, the discriminative models, the descriptive models, and the generative models. A discriminative model is in the form of a classifier. It specifies the conditional probability of the class label given the input signal. A descriptive model specifies the probability distribution of the signal, based on an energy function defined on the signal. A generative model assumes that the signal is generated by some latent variables via a transformation. We shall review these models within a common framework and explore their connections. We shall also review the recent developments that take advantage of the high approximation capacities of deep neural networks.
Tasks
Published	2018-10-09
URL	http://arxiv.org/abs/1810.04261v2
PDF	http://arxiv.org/pdf/1810.04261v2.pdf
PWC	https://paperswithcode.com/paper/a-tale-of-three-probabilistic-families
Repo
Framework

Lightweight Stochastic Optimization for Minimizing Finite Sums with Infinite Data


Title	Lightweight Stochastic Optimization for Minimizing Finite Sums with Infinite Data
Authors	Shuai Zheng, James T. Kwok
Abstract	Variance reduction has been commonly used in stochastic optimization. It relies crucially on the assumption that the data set is finite. However, when the data are imputed with random noise as in data augmentation, the perturbed data set be- comes essentially infinite. Recently, the stochastic MISO (S-MISO) algorithm is introduced to address this expected risk minimization problem. Though it converges faster than SGD, a significant amount of memory is required. In this pa- per, we propose two SGD-like algorithms for expected risk minimization with random perturbation, namely, stochastic sample average gradient (SSAG) and stochastic SAGA (S-SAGA). The memory cost of SSAG does not depend on the sample size, while that of S-SAGA is the same as those of variance reduction methods on un- perturbed data. Theoretical analysis and experimental results on logistic regression and AUC maximization show that SSAG has faster convergence rate than SGD with comparable space requirement, while S-SAGA outperforms S-MISO in terms of both iteration complexity and storage.
Tasks	Data Augmentation, Stochastic Optimization
Published	2018-06-08
URL	http://arxiv.org/abs/1806.02927v1
PDF	http://arxiv.org/pdf/1806.02927v1.pdf
PWC	https://paperswithcode.com/paper/lightweight-stochastic-optimization-for
Repo
Framework

Object Tracking with Correlation Filters using Selective Single Background Patch


Title	Object Tracking with Correlation Filters using Selective Single Background Patch
Authors	Lasitha Mekkayil, Hariharan Ramasangu
Abstract	Correlation filter plays a major role in improved tracking performance compared to existing trackers. The tracker uses the adaptive correlation response to predict the location of the target. Many varieties of correlation trackers were proposed recently with high accuracy and frame rates. The paper proposes a method to select a single background patch to have a better tracking performance. The paper also contributes a variant of correlation filter by modifying the filter with image restoration filters. The approach is validated using Object Tracking Benchmark sequences.
Tasks	Image Restoration, Object Tracking
Published	2018-05-09
URL	http://arxiv.org/abs/1805.03453v1
PDF	http://arxiv.org/pdf/1805.03453v1.pdf
PWC	https://paperswithcode.com/paper/object-tracking-with-correlation-filters
Repo
Framework

STDP Learning of Image Patches with Convolutional Spiking Neural Networks


Title	STDP Learning of Image Patches with Convolutional Spiking Neural Networks
Authors	Daniel J. Saunders, Hava T. Siegelmann, Robert Kozma, Miklós Ruszinkó
Abstract	Spiking neural networks are motivated from principles of neural systems and may possess unexplored advantages in the context of machine learning. A class of \textit{convolutional spiking neural networks} is introduced, trained to detect image features with an unsupervised, competitive learning mechanism. Image features can be shared within subpopulations of neurons, or each may evolve independently to capture different features in different regions of input space. We analyze the time and memory requirements of learning with and operating such networks. The MNIST dataset is used as an experimental testbed, and comparisons are made between the performance and convergence speed of a baseline spiking neural network.
Tasks
Published	2018-08-24
URL	http://arxiv.org/abs/1808.08173v1
PDF	http://arxiv.org/pdf/1808.08173v1.pdf
PWC	https://paperswithcode.com/paper/stdp-learning-of-image-patches-with
Repo
Framework

Topic-Specific Sentiment Analysis Can Help Identify Political Ideology


Title	Topic-Specific Sentiment Analysis Can Help Identify Political Ideology
Authors	Sumit Bhatia, Deepak P
Abstract	Ideological leanings of an individual can often be gauged by the sentiment one expresses about different issues. We propose a simple framework that represents a political ideology as a distribution of sentiment polarities towards a set of topics. This representation can then be used to detect ideological leanings of documents (speeches, news articles, etc.) based on the sentiments expressed towards different topics. Experiments performed using a widely used dataset show the promise of our proposed approach that achieves comparable performance to other methods despite being much simpler and more interpretable.
Tasks	Sentiment Analysis
Published	2018-10-30
URL	http://arxiv.org/abs/1810.12897v1
PDF	http://arxiv.org/pdf/1810.12897v1.pdf
PWC	https://paperswithcode.com/paper/topic-specific-sentiment-analysis-can-help
Repo
Framework

Evaluation Measures for Quantification: An Axiomatic Approach


Title	Evaluation Measures for Quantification: An Axiomatic Approach
Authors	Fabrizio Sebastiani
Abstract	Quantification is the task of estimating, given a set $\sigma$ of unlabelled items and a set of classes $\mathcal{C}={c_{1}, \ldots, c_{\mathcal{C}}}$, the prevalence (or `relative frequency') in $\sigma$ of each class $c_{i}\in \mathcal{C}$. While quantification may in principle be solved by classifying each item in $\sigma$ and counting how many such items have been labelled with $c_{i}$, it has long been shown that this` classify and count’ (CC) method yields suboptimal quantification accuracy. As a result, quantification is no longer considered a mere byproduct of classification, and has evolved as a task of its own. While the scientific community has devoted a lot of attention to devising more accurate quantification methods, it has not devoted much to discussing what properties an \emph{evaluation measure for quantification} (EMQ) should enjoy, and which EMQs should be adopted as a result. This paper lies down a number of interesting properties that an EMQ may or may not enjoy, discusses if (and when) each of these properties is desirable, surveys the EMQs that have been used so far, and discusses whether they enjoy or not the above properties. As a result of this investigation, some of the EMQs that have been used in the literature turn out to be severely unfit, while others emerge as closer to what the quantification community actually needs. However, a significant result is that no existing EMQ satisfies all the properties identified as desirable, thus indicating that more research is needed in order to identify (or synthesize) a truly adequate EMQ.
Tasks
Published	2018-09-06
URL	http://arxiv.org/abs/1809.01991v1
PDF	http://arxiv.org/pdf/1809.01991v1.pdf
PWC	https://paperswithcode.com/paper/evaluation-measures-for-quantification-an
Repo
Framework

Combined Static and Motion Features for Deep-Networks Based Activity Recognition in Videos


Title	Combined Static and Motion Features for Deep-Networks Based Activity Recognition in Videos
Authors	Sameera Ramasinghe, Jathushan Rajasegaran, Vinoj Jayasundara, Kanchana Ranasinghe, Ranga Rodrigo, Ajith A. Pasqual
Abstract	Activity recognition in videos in a deep-learning setting—or otherwise—uses both static and pre-computed motion components. The method of combining the two components, whilst keeping the burden on the deep network less, still remains uninvestigated. Moreover, it is not clear what the level of contribution of individual components is, and how to control the contribution. In this work, we use a combination of CNN-generated static features and motion features in the form of motion tubes. We propose three schemas for combining static and motion components: based on a variance ratio, principal components, and Cholesky decomposition. The Cholesky decomposition based method allows the control of contributions. The ratio given by variance analysis of static and motion features match well with the experimental optimal ratio used in the Cholesky decomposition based method. The resulting activity recognition system is better or on par with existing state-of-the-art when tested with three popular datasets. The findings also enable us to characterize a dataset with respect to its richness in motion information.
Tasks	Activity Recognition, Activity Recognition In Videos
Published	2018-10-16
URL	http://arxiv.org/abs/1810.06827v1
PDF	http://arxiv.org/pdf/1810.06827v1.pdf
PWC	https://paperswithcode.com/paper/combined-static-and-motion-features-for-deep
Repo
Framework

Stochastic Nested Variance Reduction for Nonconvex Optimization


Title	Stochastic Nested Variance Reduction for Nonconvex Optimization
Authors	Dongruo Zhou, Pan Xu, Quanquan Gu
Abstract	We study finite-sum nonconvex optimization problems, where the objective function is an average of $n$ nonconvex functions. We propose a new stochastic gradient descent algorithm based on nested variance reduction. Compared with conventional stochastic variance reduced gradient (SVRG) algorithm that uses two reference points to construct a semi-stochastic gradient with diminishing variance in each iteration, our algorithm uses $K+1$ nested reference points to build a semi-stochastic gradient to further reduce its variance in each iteration. For smooth nonconvex functions, the proposed algorithm converges to an $\epsilon$-approximate first-order stationary point (i.e., $\nabla F(\mathbf{x})_2\leq \epsilon$) within $\tilde{O}(n\land \epsilon^{-2}+\epsilon^{-3}\land n^{1/2}\epsilon^{-2})$ number of stochastic gradient evaluations. This improves the best known gradient complexity of SVRG $O(n+n^{2/3}\epsilon^{-2})$ and that of SCSG $O(n\land \epsilon^{-2}+\epsilon^{-10/3}\land n^{2/3}\epsilon^{-2})$. For gradient dominated functions, our algorithm also achieves a better gradient complexity than the state-of-the-art algorithms.
Tasks
Published	2018-06-20
URL	http://arxiv.org/abs/1806.07811v1
PDF	http://arxiv.org/pdf/1806.07811v1.pdf
PWC	https://paperswithcode.com/paper/stochastic-nested-variance-reduction-for
Repo
Framework

Combining neural and knowledge-based approaches to Named Entity Recognition in Polish


Title	Combining neural and knowledge-based approaches to Named Entity Recognition in Polish
Authors	Sławomir Dadas
Abstract	Named entity recognition (NER) is one of the tasks in natural language processing that can greatly benefit from the use of external knowledge sources. We propose a named entity recognition framework composed of knowledge-based feature extractors and a deep learning model including contextual word embeddings, long short-term memory (LSTM) layers and conditional random fields (CRF) inference layer. We use an entity linking module to integrate our system with Wikipedia. The combination of effective neural architecture and external resources allows us to obtain state-of-the-art results on recognition of Polish proper names. We evaluate our model on data from PolEval 2018 NER challenge on which it outperforms other methods, reducing the error rate by 22.4% compared to the winning solution. Our work shows that combining neural NER model and entity linking model with a knowledge base is more effective in recognizing named entities than using NER model alone.
Tasks	Entity Linking, Named Entity Recognition, Word Embeddings
Published	2018-11-26
URL	http://arxiv.org/abs/1811.10418v1
PDF	http://arxiv.org/pdf/1811.10418v1.pdf
PWC	https://paperswithcode.com/paper/combining-neural-and-knowledge-based
Repo
Framework

Using Attribution to Decode Dataset Bias in Neural Network Models for Chemistry


Title	Using Attribution to Decode Dataset Bias in Neural Network Models for Chemistry
Authors	Kevin McCloskey, Ankur Taly, Federico Monti, Michael P. Brenner, Lucy Colwell
Abstract	Deep neural networks have achieved state of the art accuracy at classifying molecules with respect to whether they bind to specific protein targets. A key breakthrough would occur if these models could reveal the fragment pharmacophores that are causally involved in binding. Extracting chemical details of binding from the networks could potentially lead to scientific discoveries about the mechanisms of drug actions. But doing so requires shining light into the black box that is the trained neural network model, a task that has proved difficult across many domains. Here we show how the binding mechanism learned by deep neural network models can be interrogated, using a recently described attribution method. We first work with carefully constructed synthetic datasets, in which the ‘fragment logic’ of binding is fully known. We find that networks that achieve perfect accuracy on held out test datasets still learn spurious correlations due to biases in the datasets, and we are able to exploit this non-robustness to construct adversarial examples that fool the model. The dataset bias makes these models unreliable for accurately revealing information about the mechanisms of protein-ligand binding. In light of our findings, we prescribe a test that checks for dataset bias given a hypothesis. If the test fails, it indicates that either the model must be simplified or regularized and/or that the training dataset requires augmentation.
Tasks
Published	2018-11-27
URL	https://arxiv.org/abs/1811.11310v3
PDF	https://arxiv.org/pdf/1811.11310v3.pdf
PWC	https://paperswithcode.com/paper/using-attribution-to-decode-dataset-bias-in
Repo
Framework

Feature Grouping as a Stochastic Regularizer for High-Dimensional Structured Data


Title	Feature Grouping as a Stochastic Regularizer for High-Dimensional Structured Data
Authors	Sergul Aydore, Bertrand Thirion, Gael Varoquaux
Abstract	In many applications where collecting data is expensive, for example neuroscience or medical imaging, the sample size is typically small compared to the feature dimension. It is challenging in this setting to train expressive, non-linear models without overfitting. These datasets call for intelligent regularization that exploits known structure, such as correlations between the features arising from the measurement device. However, existing structured regularizers need specially crafted solvers, which are difficult to apply to complex models. We propose a new regularizer specifically designed to leverage structure in the data in a way that can be applied efficiently to complex models. Our approach relies on feature grouping, using a fast clustering algorithm inside a stochastic gradient descent loop: given a family of feature groupings that capture feature covariations, we randomly select these groups at each iteration. We show that this approach amounts to enforcing a denoising regularizer on the solution. The method is easy to implement in many model architectures, such as fully connected neural networks, and has a linear computational cost. We apply this regularizer to a real-world fMRI dataset and the Olivetti Faces datasets. Experiments on both datasets demonstrate that the proposed approach produces models that generalize better than those trained with conventional regularizers, and also improves convergence speed.
Tasks	Denoising, Image Classification
Published	2018-07-31
URL	http://arxiv.org/abs/1807.11718v2
PDF	http://arxiv.org/pdf/1807.11718v2.pdf
PWC	https://paperswithcode.com/paper/using-feature-grouping-as-a-stochastic
Repo
Framework