January 28, 2020

3010 words 15 mins read

Paper Group ANR 875

Smooth Shells: Multi-Scale Shape Registration with Functional Maps. Properties of the Stochastic Approximation EM Algorithm with Mini-batch Sampling. Evolutionary techniques in lattice sieving algorithms. How are attributes expressed in face DCNNs?. Open Sesame: Getting Inside BERT’s Linguistic Knowledge. Efficient Multivariate Bandit Algorithm wit …

Smooth Shells: Multi-Scale Shape Registration with Functional Maps


Title	Smooth Shells: Multi-Scale Shape Registration with Functional Maps
Authors	Marvin Eisenberger, Zorah Lähner, Daniel Cremers
Abstract	We propose a novel 3D shape correspondence method based on the iterative alignment of so-called smooth shells. Smooth shells define a series of coarse-to-fine shape approximations designed to work well with multiscale algorithms. The main idea is to first align rough approximations of the geometry and then add more and more details to refine the correspondence. We fuse classical shape registration with Functional Maps by embedding the input shapes into an intrinsic-extrinsic product space. Moreover, we disambiguate intrinsic symmetries by applying a surrogate based Markov chain Monte Carlo initialization. Our method naturally handles various types of noise that commonly occur in real scans, like non-isometry or incompatible meshing. Finally, we demonstrate state-of-the-art quantitative results on several datasets and show that our pipeline produces smoother, more realistic results than other automatic matching methods in real world applications.
Tasks
Published	2019-05-29
URL	https://arxiv.org/abs/1905.12512v2
PDF	https://arxiv.org/pdf/1905.12512v2.pdf
PWC	https://paperswithcode.com/paper/smooth-shells-multi-scale-shape-registration
Repo
Framework

Properties of the Stochastic Approximation EM Algorithm with Mini-batch Sampling


Title	Properties of the Stochastic Approximation EM Algorithm with Mini-batch Sampling
Authors	Estelle Kuhn, Catherine Matias, Tabea Rebafka
Abstract	To deal with very large datasets a mini-batch version of the Monte Carlo Markov Chain Stochastic Approximation Expectation-Maximization algorithm for general latent variable models is proposed. For exponential models the algorithm is shown to be convergent under classicalconditions as the number of iterations increases. Numerical experiments illustrate the performance of the mini-batch algorithm in various models.In particular, we highlight that mini-batch sampling results in an important speed-up of the convergence of the sequence of estimators generated by the algorithm. Moreover, insights on the effect of the mini-batch size on the limit distribution are presented. Finally, we illustrate how to use mini-batch sampling in practice to improve results when a constraint on the computing time is given.
Tasks	Latent Variable Models
Published	2019-07-22
URL	https://arxiv.org/abs/1907.09164v2
PDF	https://arxiv.org/pdf/1907.09164v2.pdf
PWC	https://paperswithcode.com/paper/properties-of-the-stochastic-approximation-em
Repo
Framework

Evolutionary techniques in lattice sieving algorithms


Title	Evolutionary techniques in lattice sieving algorithms
Authors	Thijs Laarhoven
Abstract	Lattice-based cryptography has recently emerged as a prominent candidate for secure communication in the quantum age. Its security relies on the hardness of certain lattice problems, and the inability of known lattice algorithms, such as lattice sieving, to solve these problems efficiently. In this paper we investigate the similarities between lattice sieving and evolutionary algorithms, how various improvements to lattice sieving can be viewed as applications of known techniques from evolutionary computation, and how other evolutionary techniques can benefit lattice sieving in practice.
Tasks
Published	2019-07-10
URL	https://arxiv.org/abs/1907.04629v1
PDF	https://arxiv.org/pdf/1907.04629v1.pdf
PWC	https://paperswithcode.com/paper/evolutionary-techniques-in-lattice-sieving
Repo
Framework

How are attributes expressed in face DCNNs?


Title	How are attributes expressed in face DCNNs?
Authors	Prithviraj Dhar, Ankan Bansal, Carlos D. Castillo, Joshua Gleason, P. Jonathon Phillips, Rama Chellappa
Abstract	As deep networks become increasingly accurate at recognizing faces, it is vital to understand how these networks process faces. While these networks are solely trained to recognize identities, they also contain face related information such as sex, age, and pose of the face. The networks are not trained to learn these attributes. We introduce expressivity as a measure of how much a feature vector informs us about an attribute, where a feature vector can be from internal or final layers of a network. Expressivity is computed by a second neural network whose inputs are features and attributes. The output of the second neural network approximates the mutual information between feature vectors and an attribute. We investigate the expressivity for two different deep convolutional neural network (DCNN) architectures: a Resnet-101 and an Inception Resnet v2. In the final fully connected layer of the networks, we found the order of expressivity for facial attributes to be Age > Sex > Yaw. Additionally, we studied the changes in the encoding of facial attributes over training iterations. We found that as training progresses, expressivities of yaw, sex, and age decrease. Our technique can be a tool for investigating the sources of bias in a network and a step towards explaining the network’s identity decisions.
Tasks
Published	2019-10-12
URL	https://arxiv.org/abs/1910.05657v1
PDF	https://arxiv.org/pdf/1910.05657v1.pdf
PWC	https://paperswithcode.com/paper/how-are-attributes-expressed-in-face-dcnns
Repo
Framework

Open Sesame: Getting Inside BERT’s Linguistic Knowledge


Title	Open Sesame: Getting Inside BERT’s Linguistic Knowledge
Authors	Yongjie Lin, Yi Chern Tan, Robert Frank
Abstract	How and to what extent does BERT encode syntactically-sensitive hierarchical information or positionally-sensitive linear information? Recent work has shown that contextual representations like BERT perform well on tasks that require sensitivity to linguistic structure. We present here two studies which aim to provide a better understanding of the nature of BERT’s representations. The first of these focuses on the identification of structurally-defined elements using diagnostic classifiers, while the second explores BERT’s representation of subject-verb agreement and anaphor-antecedent dependencies through a quantitative assessment of self-attention vectors. In both cases, we find that BERT encodes positional information about word tokens well on its lower layers, but switches to a hierarchically-oriented encoding on higher layers. We conclude then that BERT’s representations do indeed model linguistically relevant aspects of hierarchical structure, though they do not appear to show the sharp sensitivity to hierarchical structure that is found in human processing of reflexive anaphora.
Tasks
Published	2019-06-04
URL	https://arxiv.org/abs/1906.01698v1
PDF	https://arxiv.org/pdf/1906.01698v1.pdf
PWC	https://paperswithcode.com/paper/open-sesame-getting-inside-berts-linguistic
Repo
Framework

Efficient Multivariate Bandit Algorithm with Path Planning


Title	Efficient Multivariate Bandit Algorithm with Path Planning
Authors	Keyu Nie, Zezhong Zhang, Ted Tao Yuan, Rong Song, Pauline Berry Burke
Abstract	In this paper, we solve the arms exponential exploding issue in multivariate Multi-Armed Bandit (Multivariate-MAB) problem when the arm dimension hierarchy is considered. We propose a framework called path planning (TS-PP) which utilizes decision graph/trees to model arm reward success rate with m-way dimension interaction, and adopts Thompson sampling (TS) for heuristic search of arm selection. Naturally, it is quite straightforward to combat the curse of dimensionality using a serial processes that operates sequentially by focusing on one dimension per each process. For our best acknowledge, we are the first to solve Multivariate-MAB problem using graph path planning strategy and deploying alike Monte-Carlo tree search ideas. Our proposed method utilizing tree models has advantages comparing with traditional models such as general linear regression. Simulation studies validate our claim by achieving faster convergence speed, better efficient optimal arm allocation and lower cumulative regret.
Tasks
Published	2019-09-06
URL	https://arxiv.org/abs/1909.02705v1
PDF	https://arxiv.org/pdf/1909.02705v1.pdf
PWC	https://paperswithcode.com/paper/efficient-multivariate-bandit-algorithm-with
Repo
Framework

On NMT Search Errors and Model Errors: Cat Got Your Tongue?


Title	On NMT Search Errors and Model Errors: Cat Got Your Tongue?
Authors	Felix Stahlberg, Bill Byrne
Abstract	We report on search errors and model errors in neural machine translation (NMT). We present an exact inference procedure for neural sequence models based on a combination of beam search and depth-first search. We use our exact search to find the global best model scores under a Transformer base model for the entire WMT15 English-German test set. Surprisingly, beam search fails to find these global best model scores in most cases, even with a very large beam size of 100. For more than 50% of the sentences, the model in fact assigns its global best score to the empty translation, revealing a massive failure of neural models in properly accounting for adequacy. We show by constraining search with a minimum translation length that at the root of the problem of empty translations lies an inherent bias towards shorter translations. We conclude that vanilla NMT in its current form requires just the right amount of beam search errors, which, from a modelling perspective, is a highly unsatisfactory conclusion indeed, as the model often prefers an empty translation.
Tasks	Machine Translation
Published	2019-08-27
URL	https://arxiv.org/abs/1908.10090v1
PDF	https://arxiv.org/pdf/1908.10090v1.pdf
PWC	https://paperswithcode.com/paper/on-nmt-search-errors-and-model-errors-cat-got
Repo
Framework

Probabilistic Regression of Rotations using Quaternion Averaging and a Deep Multi-Headed Network


Title	Probabilistic Regression of Rotations using Quaternion Averaging and a Deep Multi-Headed Network
Authors	Valentin Peretroukhin, Brandon Wagstaff, Matthew Giamou, Jonathan Kelly
Abstract	Accurate estimates of rotation are crucial to vision-based motion estimation in augmented reality and robotics. In this work, we present a method to extract probabilistic estimates of rotation from deep regression models. First, we build on prior work and argue that a multi-headed network structure we name HydraNet provides better calibrated uncertainty estimates than methods that rely on stochastic forward passes. Second, we extend HydraNet to targets that belong to the rotation group, SO(3), by regressing unit quaternions and using the tools of rotation averaging and uncertainty injection onto the manifold to produce three-dimensional covariances. Finally, we present results and analysis on a synthetic dataset, learn consistent orientation estimates on the 7-Scenes dataset, and show how we can use our learned covariances to fuse deep estimates of relative orientation with classical stereo visual odometry to improve localization on the KITTI dataset.
Tasks	Motion Estimation, Visual Odometry
Published	2019-04-01
URL	http://arxiv.org/abs/1904.03182v1
PDF	http://arxiv.org/pdf/1904.03182v1.pdf
PWC	https://paperswithcode.com/paper/probabilistic-regression-of-rotations-using
Repo
Framework

Quotienting Impertinent Camera Kinematics for 3D Video Stabilization


Title	Quotienting Impertinent Camera Kinematics for 3D Video Stabilization
Authors	Thomas W. Mitchel, Christian Wuelker, Jin Seob Kim, Sipu Ruan, Gregory S. Chirikjian
Abstract	With the recent advent of methods that allow for real-time computation, dense 3D flows have become a viable basis for fast camera motion estimation. Most importantly, dense flows are more robust than the sparse feature matching techniques used by existing 3D stabilization methods, able to better handle large camera displacements and occlusions similar to those often found in consumer videos. Here we introduce a framework for 3D video stabilization that relies on dense scene flow alone. The foundation of this approach is a novel camera motion model that allows for real-world camera poses to be recovered directly from 3D motion fields. Moreover, this model can be extended to describe certain types of non-rigid artifacts that are commonly found in videos, such as those resulting from zooms. This framework gives rise to several robust regimes that produce high-quality stabilization of the kind achieved by prior full 3D methods while avoiding the fragility typically present in feature-based approaches. As an added benefit, our framework is fast: the simplicity of our motion model and efficient flow calculations combine to enable stabilization at a high frame rate.
Tasks	Motion Estimation
Published	2019-03-21
URL	https://arxiv.org/abs/1903.09073v2
PDF	https://arxiv.org/pdf/1903.09073v2.pdf
PWC	https://paperswithcode.com/paper/quotienting-impertinent-camera-kinematics-for
Repo
Framework

Stochastic Generalized Adversarial Label Learning


Title	Stochastic Generalized Adversarial Label Learning
Authors	Chidubem Arachie, Bert Huang
Abstract	The usage of machine learning models has grown substantially and is spreading into several application domains. A common need in using machine learning models is collecting the data required to train these models. In some cases, labeling a massive dataset can be a crippling bottleneck, so there is need to develop models that work when training labels for large amounts of data are not easily obtained. A possible solution is weak supervision, which uses noisy labels that are easily obtained from multiple sources. The challenge is how best to combine these noisy labels and train a model to perform well given a task. In this paper, we propose stochastic generalized adversarial label learning (Stoch-GALL), a framework for training machine learning models that perform well when noisy and possibly correlated labels are provided. Our framework allows users to provide different weak labels and multiple constraints on these labels. Our model then attempts to learn parameters for the data by solving a non-zero sum game optimization. The game is between an adversary that chooses labels for the data and a model that minimizes the error made by the adversarial labels. We test our method on three datasets by training convolutional neural network models that learn to classify image objects with limited access to training labels. Our approach is able to learn even in settings where the weak supervision confounds state-of-the-art weakly supervised learning methods. The results of our experiments demonstrate the applicability of this approach to general classification tasks.
Tasks
Published	2019-06-03
URL	https://arxiv.org/abs/1906.00512v2
PDF	https://arxiv.org/pdf/1906.00512v2.pdf
PWC	https://paperswithcode.com/paper/190600512
Repo
Framework

Robustness Against Outliers For Deep Neural Networks By Gradient Conjugate Priors


Title	Robustness Against Outliers For Deep Neural Networks By Gradient Conjugate Priors
Authors	Pavel Gurevich, Hannes Stuke
Abstract	We analyze a new robust method for the reconstruction of probability distributions of observed data in the presence of output outliers. It is based on a so-called gradient conjugate prior (GCP) network which outputs the parameters of a prior. By rigorously studying the dynamics of the GCP learning process, we derive an explicit formula for correcting the obtained variance of the marginal distribution and removing the bias caused by outliers in the training set. Assuming a Gaussian (input-dependent) ground truth distribution contaminated with a proportion $\varepsilon$ of outliers, we show that the fitted mean is in a $c e^{-1/\varepsilon}$-neighborhood of the ground truth mean and the corrected variance is in a $b\varepsilon$-neighborhood of the ground truth variance, whereas the uncorrected variance of the marginal distribution can even be infinite. We explicitly find $b$ as a function of the output of the GCP network, without a priori knowledge of the outliers (possibly input-dependent) distribution. Experiments with synthetic and real-world data sets indicate that the GCP network fitted with a standard optimizer outperforms other robust methods for regression.
Tasks
Published	2019-05-21
URL	https://arxiv.org/abs/1905.08464v1
PDF	https://arxiv.org/pdf/1905.08464v1.pdf
PWC	https://paperswithcode.com/paper/robustness-against-outliers-for-deep-neural
Repo
Framework

Robust Video Background Identification by Dominant Rigid Motion Estimation


Title	Robust Video Background Identification by Dominant Rigid Motion Estimation
Authors	Kaimo Lin, Nianjuan Jiang, Loong Fah Cheong, Jiangbo Lu, Xun Xu
Abstract	The ability to identify the static background in videos captured by a moving camera is an important pre-requisite for many video applications (e.g. video stabilization, stitching, and segmentation). Existing methods usually face difficulties when the foreground objects occupy a larger area than the background in the image. Many methods also cannot scale up to handle densely sampled feature trajectories. In this paper, we propose an efficient local-to-global method to identify background, based on the assumption that as long as there is sufficient camera motion, the cumulative background features will have the largest amount of trajectories. Our motion model at the two-frame level is based on the epipolar geometry so that there will be no over-segmentation problem, another issue that plagues the 2D motion segmentation approach. Foreground objects erroneously labelled due to intermittent motions are also taken care of by checking their global consistency with the final estimated background motion. Lastly, by virtue of its efficiency, our method can deal with densely sampled trajectories. It outperforms several state-of-the-art motion segmentation methods on public datasets, both quantitatively and qualitatively.
Tasks	Motion Estimation, Motion Segmentation
Published	2019-03-06
URL	http://arxiv.org/abs/1903.02232v1
PDF	http://arxiv.org/pdf/1903.02232v1.pdf
PWC	https://paperswithcode.com/paper/robust-video-background-identification-by
Repo
Framework

GANspection


Title	GANspection
Authors	Hammad A. Ayyubi
Abstract	Generative Adversarial Networks (GANs) have been used extensively and quite successfully for unsupervised learning. As GANs don’t approximate an explicit probability distribution, it’s an interesting study to inspect the latent space representations learned by GANs. The current work seeks to push the boundaries of such inspection methods to further understand in more detail the manifold being learned by GANs. Various interpolation and extrapolation techniques along with vector arithmetic is used to understand the learned manifold. We show through experiments that GANs indeed learn a data probability distribution rather than memorize images/data. Further, we prove that GANs encode semantically relevant information in the learned probability distribution. The experiments have been performed on two publicly available datasets - Large Scale Scene Understanding (LSUN) and CelebA.
Tasks	Scene Understanding
Published	2019-10-21
URL	https://arxiv.org/abs/1910.09638v1
PDF	https://arxiv.org/pdf/1910.09638v1.pdf
PWC	https://paperswithcode.com/paper/ganspection
Repo
Framework

Why X rather than Y? Explaining Neural Model’ Predictions by Generating Intervention Counterfactual Samples


Title	Why X rather than Y? Explaining Neural Model’ Predictions by Generating Intervention Counterfactual Samples
Authors	Thai Le, Suhang Wang, Dongwon Lee
Abstract	Even though the topic of explainable AI/ML is very popular in text and computer vision domain, most of the previous literatures are not suitable for explaining black-box models’ predictions on general data mining datasets. This is because these datasets are usually in high-dimensional vectored features format that are not as friendly and comprehensible as texts and images to the end users. In this paper, we combine the best of both worlds: “explanations by intervention” from causality and “explanations are contrastive” from philosophy and social science domain to explain neural models’ predictions for tabular datasets. Specifically, given a model’s prediction as label X, we propose a novel idea to intervene and generate minimally modified contrastive sample to be classified as Y, that then results in a simple natural text giving answer to the question “Why X rather than Y?". We carry out experiments with several datasets of different scales and compare our approach with other baselines on three different areas: fidelity, reasonableness and explainability.
Tasks
Published	2019-11-05
URL	https://arxiv.org/abs/1911.02042v1
PDF	https://arxiv.org/pdf/1911.02042v1.pdf
PWC	https://paperswithcode.com/paper/why-x-rather-than-y-explaining-neural-model
Repo
Framework

Why Non-myopic Bayesian Optimization is Promising and How Far Should We Look-ahead? A Study via Rollout


Title	Why Non-myopic Bayesian Optimization is Promising and How Far Should We Look-ahead? A Study via Rollout
Authors	Xubo Yue, Raed Al Kontar
Abstract	Lookahead, also known as non-myopic, Bayesian optimization (BO) aims to find optimal sampling policies through solving a dynamic programming (DP) formulation that maximizes a long-term reward over a rolling horizon. Though promising, lookahead BO faces the risk of error propagation through its increased dependence on a possibly mis-specified model. In this work we focus on the rollout approximation for solving the intractable DP. We first prove the improving nature of rollout in tackling lookahead BO and provide a sufficient condition for the used heuristic to be rollout improving. We then provide both a theoretical and practical guideline to decide on the rolling horizon stagewise. This guideline is built on quantifying the negative effect of a mis-specified model. To illustrate our idea, we provide case studies on both single and multi-information source BO. Empirical results show the advantageous properties of our method over several myopic and non-myopic BO algorithms.
Tasks
Published	2019-11-04
URL	https://arxiv.org/abs/1911.01004v2
PDF	https://arxiv.org/pdf/1911.01004v2.pdf
PWC	https://paperswithcode.com/paper/lookahead-bayesian-optimization-via-rollout
Repo
Framework