January 30, 2020

3199 words 16 mins read

Paper Group ANR 355

Reducing Transformer Depth on Demand with Structured Dropout. White-Box Evaluation of Fingerprint Matchers: Robustness to Minutiae Perturbations. Towards Robust RGB-D Human Mesh Recovery. Recovery of binary sparse signals from compressed linear measurements via polynomial optimization. Invariance-Preserving Localized Activation Functions for Graph …

Reducing Transformer Depth on Demand with Structured Dropout


Title	Reducing Transformer Depth on Demand with Structured Dropout
Authors	Angela Fan, Edouard Grave, Armand Joulin
Abstract	Overparameterized transformer networks have obtained state of the art results in various natural language processing tasks, such as machine translation, language modeling, and question answering. These models contain hundreds of millions of parameters, necessitating a large amount of computation and making them prone to overfitting. In this work, we explore LayerDrop, a form of structured dropout, which has a regularization effect during training and allows for efficient pruning at inference time. In particular, we show that it is possible to select sub-networks of any depth from one large network without having to finetune them and with limited impact on performance. We demonstrate the effectiveness of our approach by improving the state of the art on machine translation, language modeling, summarization, question answering, and language understanding benchmarks. Moreover, we show that our approach leads to small BERT-like models of higher quality compared to training from scratch or using distillation.
Tasks	Language Modelling, Machine Translation, Question Answering
Published	2019-09-25
URL	https://arxiv.org/abs/1909.11556v1
PDF	https://arxiv.org/pdf/1909.11556v1.pdf
PWC	https://paperswithcode.com/paper/reducing-transformer-depth-on-demand-with-1
Repo
Framework

White-Box Evaluation of Fingerprint Matchers: Robustness to Minutiae Perturbations


Title	White-Box Evaluation of Fingerprint Matchers: Robustness to Minutiae Perturbations
Authors	Steven A. Grosz, Joshua J. Engelsma, Nicholas G. Paulter Jr., Anil K. Jain
Abstract	Prevailing evaluations of fingerprint recognition systems have been performed as end-to-end black-box tests of fingerprint identification or authentication accuracy. However, performance of the end-to-end system is subject to errors arising in any of its constituent modules, including: fingerprint scanning, preprocessing, feature extraction, and matching. On the other hand, white-box evaluations provide a more granular evaluation by studying the individual sub-components of a system. While a few studies have conducted stand-alone evaluations of the fingerprint reader and feature extraction modules of fingerprint recognition systems, little work has been devoted towards white-box evaluations of the fingerprint matching module. We report results of a controlled, white-box evaluation of one open-source and two commercial-off-the-shelf (COTS) minutiae-based matchers in terms of their robustness against controlled perturbations (random noise and non-linear distortions) introduced into the input minutiae feature sets. Our white-box evaluations reveal that the performance of fingerprint minutiae matchers are more susceptible to non-linear distortion and missing minutiae than spurious minutiae and small positional displacements of the minutiae locations.
Tasks
Published	2019-09-02
URL	https://arxiv.org/abs/1909.00799v3
PDF	https://arxiv.org/pdf/1909.00799v3.pdf
PWC	https://paperswithcode.com/paper/white-box-evaluation-of-fingerprint-matchers
Repo
Framework

Towards Robust RGB-D Human Mesh Recovery


Title	Towards Robust RGB-D Human Mesh Recovery
Authors	Ren Li, Changjiang Cai, Georgios Georgakis, Srikrishna Karanam, Terrence Chen, Ziyan Wu
Abstract	We consider the problem of human pose estimation. While much recent work has focused on the RGB domain, these techniques are inherently under-constrained since there can be many 3D configurations that explain the same 2D projection. To this end, we propose a new method that uses RGB-D data to estimate a parametric human mesh model. Our key innovations include (a) the design of a new dynamic data fusion module that facilitates learning with a combination of RGB-only and RGB-D datasets, (b) a new constraint generator module that provides SMPL supervisory signals when explicit SMPL annotations are not available, and (c) the design of a new depth ranking learning objective, all of which enable principled model training with RGB-D data. We conduct extensive experiments on a variety of RGB-D datasets to demonstrate efficacy.
Tasks	Pose Estimation
Published	2019-11-18
URL	https://arxiv.org/abs/1911.07383v1
PDF	https://arxiv.org/pdf/1911.07383v1.pdf
PWC	https://paperswithcode.com/paper/towards-robust-rgb-d-human-mesh-recovery
Repo
Framework

Recovery of binary sparse signals from compressed linear measurements via polynomial optimization


Title	Recovery of binary sparse signals from compressed linear measurements via polynomial optimization
Authors	Sophie M. Fosson, Mohammad Abuabiah
Abstract	The recovery of signals with finite-valued components from few linear measurements is a problem with widespread applications and interesting mathematical characteristics. In the compressed sensing framework, tailored methods have been recently proposed to deal with the case of finite-valued sparse signals. In this work, we focus on binary sparse signals and we propose a novel formulation, based on polynomial optimization. This approach is analyzed and compared to the state-of-the-art binary compressed sensing methods.
Tasks
Published	2019-05-30
URL	https://arxiv.org/abs/1905.13181v1
PDF	https://arxiv.org/pdf/1905.13181v1.pdf
PWC	https://paperswithcode.com/paper/recovery-of-binary-sparse-signals-from
Repo
Framework

Invariance-Preserving Localized Activation Functions for Graph Neural Networks


Title	Invariance-Preserving Localized Activation Functions for Graph Neural Networks
Authors	Luana Ruiz, Fernando Gama, Antonio G. Marques, Alejandro Ribeiro
Abstract	Graph signals are signals with an irregular structure that can be described by a graph. Graph neural networks (GNNs) are information processing architectures tailored to these graph signals and made of stacked layers that compose graph convolutional filters with nonlinear activation functions. Graph convolutions endow GNNs with invariance to permutations of the graph nodes’ labels. In this paper, we consider the design of trainable nonlinear activation functions that take into consideration the structure of the graph. This is accomplished by using graph median filters and graph max filters, which mimic linear graph convolutions and are shown to retain the permutation invariance of GNNs. We also discuss modifications to the backpropagation algorithm necessary to train local activation functions. The advantages of localized activation function architectures are demonstrated in four numerical experiments: source localization on synthetic graphs, authorship attribution of 19th century novels, movie recommender systems and scientific article classification. In all cases, localized activation functions are shown to improve model capacity.
Tasks	Recommendation Systems
Published	2019-03-29
URL	https://arxiv.org/abs/1903.12575v2
PDF	https://arxiv.org/pdf/1903.12575v2.pdf
PWC	https://paperswithcode.com/paper/invariance-preserving-localized-activation
Repo
Framework

Exploring Object Relation in Mean Teacher for Cross-Domain Detection


Title	Exploring Object Relation in Mean Teacher for Cross-Domain Detection
Authors	Qi Cai, Yingwei Pan, Chong-Wah Ngo, Xinmei Tian, Lingyu Duan, Ting Yao
Abstract	Rendering synthetic data (e.g., 3D CAD-rendered images) to generate annotations for learning deep models in vision tasks has attracted increasing attention in recent years. However, simply applying the models learnt on synthetic images may lead to high generalization error on real images due to domain shift. To address this issue, recent progress in cross-domain recognition has featured the Mean Teacher, which directly simulates unsupervised domain adaptation as semi-supervised learning. The domain gap is thus naturally bridged with consistency regularization in a teacher-student scheme. In this work, we advance this Mean Teacher paradigm to be applicable for cross-domain detection. Specifically, we present Mean Teacher with Object Relations (MTOR) that novelly remolds Mean Teacher under the backbone of Faster R-CNN by integrating the object relations into the measure of consistency cost between teacher and student modules. Technically, MTOR firstly learns relational graphs that capture similarities between pairs of regions for teacher and student respectively. The whole architecture is then optimized with three consistency regularizations: 1) region-level consistency to align the region-level predictions between teacher and student, 2) inter-graph consistency for matching the graph structures between teacher and student, and 3) intra-graph consistency to enhance the similarity between regions of same class within the graph of student. Extensive experiments are conducted on the transfers across Cityscapes, Foggy Cityscapes, and SIM10k, and superior results are reported when comparing to state-of-the-art approaches. More remarkably, we obtain a new record of single model: 22.8% of mAP on Syn2Real detection dataset.
Tasks	Domain Adaptation, Unsupervised Domain Adaptation
Published	2019-04-25
URL	https://arxiv.org/abs/1904.11245v2
PDF	https://arxiv.org/pdf/1904.11245v2.pdf
PWC	https://paperswithcode.com/paper/exploring-object-relation-in-mean-teacher-for
Repo
Framework

Measuring the effects of confounders in medical supervised classification problems: the Confounding Index (CI)


Title	Measuring the effects of confounders in medical supervised classification problems: the Confounding Index (CI)
Authors	Elisa Ferrari, Alessandra Retico, Davide Bacciu
Abstract	Over the years, there has been growing interest in using Machine Learning techniques for biomedical data processing. When tackling these tasks, one needs to bear in mind that biomedical data depends on a variety of characteristics, such as demographic aspects (age, gender, etc) or the acquisition technology, which might be unrelated with the target of the analysis. In supervised tasks, failing to match the ground truth targets with respect to such characteristics, called confounders, may lead to very misleading estimates of the predictive performance. Many strategies have been proposed to handle confounders, ranging from data selection, to normalization techniques, up to the use of training algorithm for learning with imbalanced data. However, all these solutions require the confounders to be known a priori. To this aim, we introduce a novel index that is able to measure the confounding effect of a data attribute in a bias-agnostic way. This index can be used to quantitatively compare the confounding effects of different variables and to inform correction methods such as normalization procedures or ad-hoc-prepared learning algorithms. The effectiveness of this index is validated on both simulated data and real-world neuroimaging data.
Tasks
Published	2019-05-21
URL	https://arxiv.org/abs/1905.08871v2
PDF	https://arxiv.org/pdf/1905.08871v2.pdf
PWC	https://paperswithcode.com/paper/measuring-the-effects-of-confounders-in
Repo
Framework

Satellite Pose Estimation Challenge: Dataset, Competition Design and Results


Title	Satellite Pose Estimation Challenge: Dataset, Competition Design and Results
Authors	Mate Kisantal, Sumant Sharma, Tae Ha Park, Dario Izzo, Marcus Märtens, Simone D’Amico
Abstract	Reliable pose estimation of uncooperative satellites is a key technology for enabling future on-orbit servicing and debris removal missions. The Kelvins Satellite Pose Estimation Challenge aims at evaluating and comparing monocular vision-based approaches and pushing the state-of-the-art on this problem. This work is based on the Satellite Pose Estimation Dataset, the first publicly available machine learning set of synthetic and real spacecraft imagery. The choice of dataset reflects one of the unique challenges associated with spaceborne computer vision tasks, namely the lack of spaceborne images to train and validate the developed algorithms. This work briefly reviews the basic properties and the collection process of the dataset which was made publicly available. The competition design, including the definition of performance metrics and the adopted testbed, is also discussed. Furthermore, the submissions of the 48 participants are analyzed to compare the performance of their approaches and uncover what factors make the satellite pose estimation problem especially challenging.
Tasks	Pose Estimation
Published	2019-11-05
URL	https://arxiv.org/abs/1911.02050v1
PDF	https://arxiv.org/pdf/1911.02050v1.pdf
PWC	https://paperswithcode.com/paper/satellite-pose-estimation-challenge-dataset
Repo
Framework

Accelerating Gradient Boosting Machine


Title	Accelerating Gradient Boosting Machine
Authors	Haihao Lu, Sai Praneeth Karimireddy, Natalia Ponomareva, Vahab Mirrokni
Abstract	Gradient Boosting Machine (GBM) is an extremely powerful supervised learning algorithm that is widely used in practice. GBM routinely features as a leading algorithm in machine learning competitions such as Kaggle and the KDDCup. In this work, we propose Accelerated Gradient Boosting Machine (AGBM) by incorporating Nesterov’s acceleration techniques into the design of GBM. The difficulty in accelerating GBM lies in the fact that weak (inexact) learners are commonly used, and therefore the errors can accumulate in the momentum term. To overcome it, we design a “corrected pseudo residual” and fit best weak learner to this corrected pseudo residual, in order to perform the z-update. Thus, we are able to derive novel computational guarantees for AGBM. This is the first GBM type of algorithm with theoretically-justified accelerated convergence rate. Finally we demonstrate with a number of numerical experiments the effectiveness of AGBM over conventional GBM in obtaining a model with good training and/or testing data fidelity.
Tasks
Published	2019-03-20
URL	https://arxiv.org/abs/1903.08708v2
PDF	https://arxiv.org/pdf/1903.08708v2.pdf
PWC	https://paperswithcode.com/paper/accelerating-gradient-boosting-machine
Repo
Framework

Towards Unsupervised Familiar Scene Recognition in Egocentric Videos


Title	Towards Unsupervised Familiar Scene Recognition in Egocentric Videos
Authors	Estefania Talavera, Nicolai Petkov, Petia Radeva
Abstract	Nowadays, there is an upsurge of interest in using lifelogging devices. Such devices generate huge amounts of image data; consequently, the need for automatic methods for analyzing and summarizing these data is drastically increasing. We present a new method for familiar scene recognition in egocentric videos, based on background pattern detection through automatically configurable COSFIRE filters. We present some experiments over egocentric data acquired with the Narrative Clip.
Tasks	Scene Recognition
Published	2019-05-10
URL	https://arxiv.org/abs/1905.04093v1
PDF	https://arxiv.org/pdf/1905.04093v1.pdf
PWC	https://paperswithcode.com/paper/towards-unsupervised-familiar-scene
Repo
Framework

3DPeople: Modeling the Geometry of Dressed Humans


Title	3DPeople: Modeling the Geometry of Dressed Humans
Authors	Albert Pumarola, Jordi Sanchez, Gary P. T. Choi, Alberto Sanfeliu, Francesc Moreno-Noguer
Abstract	Recent advances in 3D human shape estimation build upon parametric representations that model very well the shape of the naked body, but are not appropriate to represent the clothing geometry. In this paper, we present an approach to model dressed humans and predict their geometry from single images. We contribute in three fundamental aspects of the problem, namely, a new dataset, a novel shape parameterization algorithm and an end-to-end deep generative network for predicting shape. First, we present 3DPeople, a large-scale synthetic dataset with 2.5 Million photo-realistic images of 80 subjects performing 70 activities and wearing diverse outfits. Besides providing textured 3D meshes for clothes and body, we annotate the dataset with segmentation masks, skeletons, depth, normal maps and optical flow. All this together makes 3DPeople suitable for a plethora of tasks. We then represent the 3D shapes using 2D geometry images. To build these images we propose a novel spherical area-preserving parameterization algorithm based on the optimal mass transportation method. We show this approach to improve existing spherical maps which tend to shrink the elongated parts of the full body models such as the arms and legs, making the geometry images incomplete. Finally, we design a multi-resolution deep generative network that, given an input image of a dressed human, predicts his/her geometry image (and thus the clothed body shape) in an end-to-end manner. We obtain very promising results in jointly capturing body pose and clothing shape, both for synthetic validation and on the wild images.
Tasks	Optical Flow Estimation
Published	2019-04-09
URL	http://arxiv.org/abs/1904.04571v1
PDF	http://arxiv.org/pdf/1904.04571v1.pdf
PWC	https://paperswithcode.com/paper/3dpeople-modeling-the-geometry-of-dressed
Repo
Framework

A Probabilistic Bitwise Genetic Algorithm for B-Spline based Image Deformation Estimation


Title	A Probabilistic Bitwise Genetic Algorithm for B-Spline based Image Deformation Estimation
Authors	Takumi Nakane, Takuya Akashi, Xuequan Lu, Chao Zhang
Abstract	We propose a novel genetic algorithm to solve the image deformation estimation problem by preserving the genetic diversity. As a classical problem, there is always a trade-off between the complexity of deformation models and the difficulty of parameters search in image deformation. 2D cubic B-spline surface is a highly free-form deformation model and is able to handle complex deformations such as fluid image distortions. However, it is challenging to estimate an apposite global solution. To tackle this problem, we develop a genetic operation named probabilistic bitwise operation (PBO) to replace the crossover and mutation operations, which can preserve the diversity during generation iteration and achieve better coverage ratio of the solution space. Furthermore, a selection strategy named annealing selection is proposed to control the convergence. Qualitative and quantitative results on synthetic data show the effectiveness of our method.
Tasks
Published	2019-03-26
URL	http://arxiv.org/abs/1903.10657v1
PDF	http://arxiv.org/pdf/1903.10657v1.pdf
PWC	https://paperswithcode.com/paper/a-probabilistic-bitwise-genetic-algorithm-for
Repo
Framework

Consequential Ranking Algorithms and Long-term Welfare


Title	Consequential Ranking Algorithms and Long-term Welfare
Authors	Behzad Tabibian, Vicenç Gómez, Abir De, Bernhard Schölkopf, Manuel Gomez Rodriguez
Abstract	Ranking models are typically designed to provide rankings that optimize some measure of immediate utility to the users. As a result, they have been unable to anticipate an increasing number of undesirable long-term consequences of their proposed rankings, from fueling the spread of misinformation and increasing polarization to degrading social discourse. Can we design ranking models that understand the consequences of their proposed rankings and, more importantly, are able to avoid the undesirable ones? In this paper, we first introduce a joint representation of rankings and user dynamics using Markov decision processes. Then, we show that this representation greatly simplifies the construction of consequential ranking models that trade off the immediate utility and the long-term welfare. In particular, we can obtain optimal consequential rankings just by applying weighted sampling on the rankings provided by models that maximize measures of immediate utility. However, in practice, such a strategy may be inefficient and impractical, specially in high dimensional scenarios. To overcome this, we introduce an efficient gradient-based algorithm to learn parameterized consequential ranking models that effectively approximate optimal ones. We showcase our methodology using synthetic and real data gathered from Reddit and show that ranking models derived using our methodology provide ranks that may mitigate the spread of misinformation and improve the civility of online discussions.
Tasks
Published	2019-05-13
URL	https://arxiv.org/abs/1905.05305v1
PDF	https://arxiv.org/pdf/1905.05305v1.pdf
PWC	https://paperswithcode.com/paper/consequential-ranking-algorithms-and-long
Repo
Framework

Machine Translation Evaluation using Bi-directional Entailment


Title	Machine Translation Evaluation using Bi-directional Entailment
Authors	Rakesh Khobragade, Heaven Patel, Anand Namdev, Anish Mishra, Pushpak Bhattacharyya
Abstract	In this paper, we propose a new metric for Machine Translation (MT) evaluation, based on bi-directional entailment. We show that machine generated translation can be evaluated by determining paraphrasing with a reference translation provided by a human translator. We hypothesize, and show through experiments, that paraphrasing can be detected by evaluating entailment relationship in the forward and backward direction. Unlike conventional metrics, like BLEU or METEOR, our approach uses deep learning to determine the semantic similarity between candidate and reference translation for generating scores rather than relying upon simple n-gram overlap. We use BERT’s pre-trained implementation of transformer networks, fine-tuned on MNLI corpus, for natural language inferencing. We apply our evaluation metric on WMT’14 and WMT’17 dataset to evaluate systems participating in the translation task and find that our metric has a better correlation with the human annotated score compared to the other traditional metrics at system level.
Tasks	Machine Translation, Semantic Similarity, Semantic Textual Similarity
Published	2019-11-02
URL	https://arxiv.org/abs/1911.00681v1
PDF	https://arxiv.org/pdf/1911.00681v1.pdf
PWC	https://paperswithcode.com/paper/machine-translation-evaluation-using-bi
Repo
Framework

Sample Adaptive Multiple Kernel Learning for Failure Prediction of Railway Points


Title	Sample Adaptive Multiple Kernel Learning for Failure Prediction of Railway Points
Authors	Zhibin Li, Jian Zhang, Qiang Wu, Yongshun Gong, Jinfeng Yi, Christina Kirsch
Abstract	Railway points are among the key components of railway infrastructure. As a part of signal equipment, points control the routes of trains at railway junctions, having a significant impact on the reliability, capacity, and punctuality of rail transport. Traditionally, maintenance of points is based on a fixed time interval or raised after the equipment failures. Instead, it would be of great value if we could forecast points’ failures and take action beforehand, minimising any negative effect. To date, most of the existing prediction methods are either lab-based or relying on specially installed sensors which makes them infeasible for large-scale implementation. Besides, they often use data from only one source. We, therefore, explore a new way that integrates multi-source data which are ready to hand to fulfil this task. We conducted our case study based on Sydney Trains rail network which is an extensive network of passenger and freight railways. Unfortunately, the real-world data are usually incomplete due to various reasons, e.g., faults in the database, operational errors or transmission faults. Besides, railway points differ in their locations, types and some other properties, which means it is hard to use a unified model to predict their failures. Aiming at this challenging task, we firstly constructed a dataset from multiple sources and selected key features with the help of domain experts. In this paper, we formulate our prediction task as a multiple kernel learning problem with missing kernels. We present a robust multiple kernel learning algorithm for predicting points failures. Our model takes into account the missing pattern of data as well as the inherent variance on different sets of railway points. Extensive experiments demonstrate the superiority of our algorithm compared with other state-of-the-art methods.
Tasks
Published	2019-07-02
URL	https://arxiv.org/abs/1907.01162v1
PDF	https://arxiv.org/pdf/1907.01162v1.pdf
PWC	https://paperswithcode.com/paper/sample-adaptive-multiple-kernel-learning-for
Repo
Framework