October 20, 2019

3407 words 16 mins read

Paper Group ANR 3

Convergence of Gradient Descent on Separable Data. Event-based Moving Object Detection and Tracking. Neuroscientific User Models: The Source of Uncertain User Feedback and Potentials for Improving Web Personalisation. Attaining human-level performance with atlas location autocontext for anatomical landmark detection in 3D CT data. Robust Nonparamet …

Convergence of Gradient Descent on Separable Data


Title	Convergence of Gradient Descent on Separable Data
Authors	Mor Shpigel Nacson, Jason D. Lee, Suriya Gunasekar, Pedro H. P. Savarese, Nathan Srebro, Daniel Soudry
Abstract	We provide a detailed study on the implicit bias of gradient descent when optimizing loss functions with strictly monotone tails, such as the logistic loss, over separable datasets. We look at two basic questions: (a) what are the conditions on the tail of the loss function under which gradient descent converges in the direction of the $L_2$ maximum-margin separator? (b) how does the rate of margin convergence depend on the tail of the loss function and the choice of the step size? We show that for a large family of super-polynomial tailed losses, gradient descent iterates on linear networks of any depth converge in the direction of $L_2$ maximum-margin solution, while this does not hold for losses with heavier tails. Within this family, for simple linear models we show that the optimal rates with fixed step size is indeed obtained for the commonly used exponentially tailed losses such as logistic loss. However, with a fixed step size the optimal convergence rate is extremely slow as $1/\log(t)$, as also proved in Soudry et al. (2018). For linear models with exponential loss, we further prove that the convergence rate could be improved to $\log (t) /\sqrt{t}$ by using aggressive step sizes that compensates for the rapidly vanishing gradients. Numerical results suggest this method might be useful for deep networks.
Tasks
Published	2018-03-05
URL	http://arxiv.org/abs/1803.01905v3
PDF	http://arxiv.org/pdf/1803.01905v3.pdf
PWC	https://paperswithcode.com/paper/convergence-of-gradient-descent-on-separable
Repo
Framework

Event-based Moving Object Detection and Tracking


Title	Event-based Moving Object Detection and Tracking
Authors	Anton Mitrokhin, Cornelia Fermuller, Chethan Parameshwara, Yiannis Aloimonos
Abstract	Event-based vision sensors, such as the Dynamic Vision Sensor (DVS), are ideally suited for real-time motion analysis. The unique properties encompassed in the readings of such sensors provide high temporal resolution, superior sensitivity to light and low latency. These properties provide the grounds to estimate motion extremely reliably in the most sophisticated scenarios but they come at a price - modern event-based vision sensors have extremely low resolution and produce a lot of noise. Moreover, the asynchronous nature of the event stream calls for novel algorithms. This paper presents a new, efficient approach to object tracking with asynchronous cameras. We present a novel event stream representation which enables us to utilize information about the dynamic (temporal) component of the event stream, and not only the spatial component, at every moment of time. This is done by approximating the 3D geometry of the event stream with a parametric model; as a result, the algorithm is capable of producing the motion-compensated event stream (effectively approximating egomotion), and without using any form of external sensors in extremely low-light and noisy conditions without any form of feature tracking or explicit optical flow computation. We demonstrate our framework on the task of independent motion detection and tracking, where we use the temporal model inconsistencies to locate differently moving objects in challenging situations of very fast motion.
Tasks	Event-based vision, Motion Detection, Object Detection, Object Tracking, Optical Flow Estimation
Published	2018-03-12
URL	https://arxiv.org/abs/1803.04523v3
PDF	https://arxiv.org/pdf/1803.04523v3.pdf
PWC	https://paperswithcode.com/paper/event-based-moving-object-detection-and
Repo
Framework

Neuroscientific User Models: The Source of Uncertain User Feedback and Potentials for Improving Web Personalisation


Title	Neuroscientific User Models: The Source of Uncertain User Feedback and Potentials for Improving Web Personalisation
Authors	Kevin Jasberg, Sergej Sizov
Abstract	In this paper we consider the neuroscientific theory of the Bayesian brain in the light of adaptive web systems and content personalisation. In particular, we elaborate on neural mechanisms of human decision-making and the origin of lacking reliability of user feedback, often denoted as noise or human uncertainty. To this end, we first introduce an adaptive model of cognitive agency in which populations of neurons provide an estimation for states of the world. Subsequently, we present various so-called decoder functions with which neuronal activity can be translated into quantitative decisions. The interplay of the underlying cognition model and the chosen decoder function leads to different model-based properties of decision processes. The goal of this paper is to promote novel user models and exploit them to naturally associate users to different clusters on the basis of their individual neural characteristics and thinking patterns. These user models might be able to turn the variability of user behaviour into additional information for improving web personalisation and its experience.
Tasks	Decision Making
Published	2018-02-16
URL	http://arxiv.org/abs/1802.05892v1
PDF	http://arxiv.org/pdf/1802.05892v1.pdf
PWC	https://paperswithcode.com/paper/neuroscientific-user-models-the-source-of
Repo
Framework

Attaining human-level performance with atlas location autocontext for anatomical landmark detection in 3D CT data


Title	Attaining human-level performance with atlas location autocontext for anatomical landmark detection in 3D CT data
Authors	Alison Q O’Neil, Antanas Kascenas, Joseph Henry, Daniel Wyeth, Matthew Shepherd, Erin Beveridge, Lauren Clunie, Carrie Sansom, Evelina Šeduikytė, Keith Muir, Ian Poole
Abstract	We present an efficient neural network method for locating anatomical landmarks in 3D medical CT scans, using atlas location autocontext in order to learn long-range spatial context. Location predictions are made by regression to Gaussian heatmaps, one heatmap per landmark. This system allows patchwise application of a shallow network, thus enabling multiple volumetric heatmaps to be predicted concurrently without prohibitive GPU memory requirements. Further, the system allows inter-landmark spatial relationships to be exploited using a simple overdetermined affine mapping that is robust to detection failures and occlusion or partial views. Evaluation is performed for 22 landmarks defined on a range of structures in head CT scans. Models are trained and validated on 201 scans. Over the final test set of 20 scans which was independently annotated by 2 human annotators, the neural network reaches an accuracy which matches the annotator variability, with similar human and machine patterns of variability across landmark classes.
Tasks
Published	2018-05-14
URL	http://arxiv.org/abs/1805.08687v2
PDF	http://arxiv.org/pdf/1805.08687v2.pdf
PWC	https://paperswithcode.com/paper/attaining-human-level-performance-with-atlas
Repo
Framework

Robust Nonparametric Regression under Huber’s $ε$-contamination Model


Title	Robust Nonparametric Regression under Huber’s $ε$-contamination Model
Authors	Simon S. Du, Yining Wang, Sivaraman Balakrishnan, Pradeep Ravikumar, Aarti Singh
Abstract	We consider the non-parametric regression problem under Huber’s $\epsilon$-contamination model, in which an $\epsilon$ fraction of observations are subject to arbitrary adversarial noise. We first show that a simple local binning median step can effectively remove the adversary noise and this median estimator is minimax optimal up to absolute constants over the H"{o}lder function class with smoothness parameters smaller than or equal to 1. Furthermore, when the underlying function has higher smoothness, we show that using local binning median as pre-preprocessing step to remove the adversarial noise, then we can apply any non-parametric estimator on top of the medians. In particular we show local median binning followed by kernel smoothing and local polynomial regression achieve minimaxity over H"{o}lder and Sobolev classes with arbitrary smoothness parameters. Our main proof technique is a decoupled analysis of adversary noise and stochastic noise, which can be potentially applied to other robust estimation problems. We also provide numerical results to verify the effectiveness of our proposed methods.
Tasks
Published	2018-05-26
URL	http://arxiv.org/abs/1805.10406v1
PDF	http://arxiv.org/pdf/1805.10406v1.pdf
PWC	https://paperswithcode.com/paper/robust-nonparametric-regression-under-hubers
Repo
Framework

A Decision Tree Approach to Predicting Recidivism in Domestic Violence


Title	A Decision Tree Approach to Predicting Recidivism in Domestic Violence
Authors	Senuri Wijenayake, Timothy Graham, Peter Christen
Abstract	Domestic violence (DV) is a global social and public health issue that is highly gendered. Being able to accurately predict DV recidivism, i.e., re-offending of a previously convicted offender, can speed up and improve risk assessment procedures for police and front-line agencies, better protect victims of DV, and potentially prevent future re-occurrences of DV. Previous work in DV recidivism has employed different classification techniques, including decision tree (DT) induction and logistic regression, where the main focus was on achieving high prediction accuracy. As a result, even the diagrams of trained DTs were often too difficult to interpret due to their size and complexity, making decision-making challenging. Given there is often a trade-off between model accuracy and interpretability, in this work our aim is to employ DT induction to obtain both interpretable trees as well as high prediction accuracy. Specifically, we implement and evaluate different approaches to deal with class imbalance as well as feature selection. Compared to previous work in DV recidivism prediction that employed logistic regression, our approach can achieve comparable area under the ROC curve results by using only 3 of 11 available features and generating understandable decision trees that contain only 4 leaf nodes.
Tasks	Decision Making, Feature Selection
Published	2018-03-27
URL	http://arxiv.org/abs/1803.09862v1
PDF	http://arxiv.org/pdf/1803.09862v1.pdf
PWC	https://paperswithcode.com/paper/a-decision-tree-approach-to-predicting
Repo
Framework

A complete hand-drawn sketch vectorization framework


Title	A complete hand-drawn sketch vectorization framework
Authors	Luca Donati, Simone Cesano, Andrea Prati
Abstract	Vectorizing hand-drawn sketches is a challenging task, which is of paramount importance for creating CAD vectorized versions for the fashion and creative workflows. This paper proposes a complete framework that automatically transforms noisy and complex hand-drawn sketches with different stroke types in a precise, reliable and highly-simplified vectorized model. The proposed framework includes a novel line extraction algorithm based on a multi-resolution application of Pearson’s cross correlation and a new unbiased thinning algorithm that can get rid of scribbles and variable-width strokes to obtain clean 1-pixel lines. Other contributions include variants of pruning, merging and edge linking procedures to post-process the obtained paths. Finally, a modification of the original Schneider’s vectorization algorithm is designed to obtain fewer control points in the resulting Bezier splines. All the proposed steps of the framework have been extensively tested and compared with state-of-the-art algorithms, showing (both qualitatively and quantitatively) its outperformance.
Tasks
Published	2018-02-16
URL	http://arxiv.org/abs/1802.05902v1
PDF	http://arxiv.org/pdf/1802.05902v1.pdf
PWC	https://paperswithcode.com/paper/a-complete-hand-drawn-sketch-vectorization
Repo
Framework

Comparing heterogeneous entities using artificial neural networks of trainable weighted structural components and machine-learned activation functions


Title	Comparing heterogeneous entities using artificial neural networks of trainable weighted structural components and machine-learned activation functions
Authors	Artit Wangperawong, Kettip Kriangchaivech, Austin Lanari, Supui Lam, Panthong Wangperawong
Abstract	To compare entities of differing types and structural components, the artificial neural network paradigm was used to cross-compare structural components between heterogeneous documents. Trainable weighted structural components were input into machine-learned activation functions of the neurons. The model was used for matching news articles and videos, where the inputs and activation functions respectively consisted of term vectors and cosine similarity measures between the weighted structural components. The model was tested with different weights, achieving as high as 59.2% accuracy for matching videos to news articles. A mobile application user interface for recommending related videos for news articles was developed to demonstrate consumer value, including its potential usefulness for cross-selling products from unrelated categories.
Tasks
Published	2018-01-09
URL	http://arxiv.org/abs/1801.03143v1
PDF	http://arxiv.org/pdf/1801.03143v1.pdf
PWC	https://paperswithcode.com/paper/comparing-heterogeneous-entities-using
Repo
Framework

Jointly Optimize Data Augmentation and Network Training: Adversarial Data Augmentation in Human Pose Estimation


Title	Jointly Optimize Data Augmentation and Network Training: Adversarial Data Augmentation in Human Pose Estimation
Authors	Xi Peng, Zhiqiang Tang, Fei Yang, Rogerio Feris, Dimitris Metaxas
Abstract	Random data augmentation is a critical technique to avoid overfitting in training deep neural network models. However, data augmentation and network training are usually treated as two isolated processes, limiting the effectiveness of network training. Why not jointly optimize the two? We propose adversarial data augmentation to address this limitation. The main idea is to design an augmentation network (generator) that competes against a target network (discriminator) by generating `hard' augmentation operations online. The augmentation network explores the weaknesses of the target network, while the latter learns from` hard’ augmentations to achieve better performance. We also design a reward/penalty strategy for effective joint training. We demonstrate our approach on the problem of human pose estimation and carry out a comprehensive experimental analysis, showing that our method can significantly improve state-of-the-art models without additional data efforts.
Tasks	Data Augmentation, Pose Estimation
Published	2018-05-24
URL	http://arxiv.org/abs/1805.09707v1
PDF	http://arxiv.org/pdf/1805.09707v1.pdf
PWC	https://paperswithcode.com/paper/jointly-optimize-data-augmentation-and
Repo
Framework

Robot Learning in Homes: Improving Generalization and Reducing Dataset Bias


Title	Robot Learning in Homes: Improving Generalization and Reducing Dataset Bias
Authors	Abhinav Gupta, Adithyavairavan Murali, Dhiraj Gandhi, Lerrel Pinto
Abstract	Data-driven approaches to solving robotic tasks have gained a lot of traction in recent years. However, most existing policies are trained on large-scale datasets collected in curated lab settings. If we aim to deploy these models in unstructured visual environments like people’s homes, they will be unable to cope with the mismatch in data distribution. In such light, we present the first systematic effort in collecting a large dataset for robotic grasping in homes. First, to scale and parallelize data collection, we built a low cost mobile manipulator assembled for under 3K USD. Second, data collected using low cost robots suffer from noisy labels due to imperfect execution and calibration errors. To handle this, we develop a framework which factors out the noise as a latent variable. Our model is trained on 28K grasps collected in several houses under an array of different environmental conditions. We evaluate our models by physically executing grasps on a collection of novel objects in multiple unseen homes. The models trained with our home dataset showed a marked improvement of 43.7% over a baseline model trained with data collected in lab. Our architecture which explicitly models the latent noise in the dataset also performed 10% better than one that did not factor out the noise. We hope this effort inspires the robotics community to look outside the lab and embrace learning based approaches to handle inaccurate cheap robots.
Tasks	Calibration, Robotic Grasping
Published	2018-07-18
URL	http://arxiv.org/abs/1807.07049v1
PDF	http://arxiv.org/pdf/1807.07049v1.pdf
PWC	https://paperswithcode.com/paper/robot-learning-in-homes-improving
Repo
Framework

Randomized Wagering Mechanisms


Title	Randomized Wagering Mechanisms
Authors	Yiling Chen, Yang Liu, Juntao Wang
Abstract	Wagering mechanisms are one-shot betting mechanisms that elicit agents’ predictions of an event. For deterministic wagering mechanisms, an existing impossibility result has shown incompatibility of some desirable theoretical properties. In particular, Pareto optimality (no profitable side bet before allocation) can not be achieved together with weak incentive compatibility, weak budget balance and individual rationality. In this paper, we expand the design space of wagering mechanisms to allow randomization and ask whether there are randomized wagering mechanisms that can achieve all previously considered desirable properties, including Pareto optimality. We answer this question positively with two classes of randomized wagering mechanisms: i) one simple randomized lottery-type implementation of existing deterministic wagering mechanisms, and ii) another family of simple and randomized wagering mechanisms which we call surrogate wagering mechanisms, which are robust to noisy ground truth. This family of mechanisms builds on the idea of learning with noisy labels (Natarajan et al. 2013) as well as a recent extension of this idea to the information elicitation without verification setting (Liu and Chen 2018). We show that a broad family of randomized wagering mechanisms satisfy all desirable theoretical properties.
Tasks
Published	2018-09-11
URL	http://arxiv.org/abs/1809.04136v4
PDF	http://arxiv.org/pdf/1809.04136v4.pdf
PWC	https://paperswithcode.com/paper/randomized-wagering-mechanisms
Repo
Framework

Towards Neural Network Patching: Evaluating Engagement-Layers and Patch-Architectures


Title	Towards Neural Network Patching: Evaluating Engagement-Layers and Patch-Architectures
Authors	Sebastian Kauschke, David Hermann Lehmann
Abstract	In this report we investigate fundamental requirements for the application of classifier patching on neural networks. Neural network patching is an approach for adapting neural network models to handle concept drift in nonstationary environments. Instead of creating or updating the existing network to accommodate concept drift, neural network patching leverages the inner layers of the network as well as its output to learn a patch that enhances the classification and corrects errors caused by the drift. It learns (i) a predictor that estimates whether the original network will misclassify an instance, and (ii) a patching network that fixes the misclassification. Neural network patching is based on the idea that the original network can still classify a majority of instances well, and that the inner feature representations encoded in the deep network aid the classifier to cope with unseen or changed inputs. In order to apply this kind of patching, we evaluate different engagement layers and patch architectures in this report, and find a set of generally applicable heuristics, which aid in parametrizing the patching procedure.
Tasks
Published	2018-12-09
URL	http://arxiv.org/abs/1812.03468v2
PDF	http://arxiv.org/pdf/1812.03468v2.pdf
PWC	https://paperswithcode.com/paper/towards-neural-network-patching-evaluating
Repo
Framework

Hyperspectral recovery from RGB images using Gaussian Processes


Title	Hyperspectral recovery from RGB images using Gaussian Processes
Authors	Naveed Akhtar, Ajmal Mian
Abstract	We propose to recover spectral details from RGB images of known spectral quantization by modeling natural spectra under Gaussian Processes and combining them with the RGB images. Our technique exploits Process Kernels to model the relative smoothness of reflectance spectra, and encourages non-negativity in the resulting signals for better estimation of the reflectance values. The Gaussian Processes are inferred in sets using clusters of spatio-spectrally correlated hyperspectral training patches. Each set is transformed to match the spectral quantization of the test RGB image. We extract overlapping patches from the RGB image and match them to the hyperspectral training patches by spectrally transforming the latter. The RGB patches are encoded over the transformed Gaussian Processes related to those hyperspectral patches and the resulting image is constructed by combining the codes with the original Processes. Our approach infers the desired Gaussian Processes under a fully Bayesian model inspired by Beta-Bernoulli Process, for which we also present the inference procedure. A thorough evaluation using three hyperspectral datasets demonstrates the effective extraction of spectral details from RGB images by the proposed technique.
Tasks	Gaussian Processes, Quantization
Published	2018-01-15
URL	http://arxiv.org/abs/1801.04654v2
PDF	http://arxiv.org/pdf/1801.04654v2.pdf
PWC	https://paperswithcode.com/paper/hyperspectral-recovery-from-rgb-images-using
Repo
Framework

Recognizing Cuneiform Signs Using Graph Based Methods


Title	Recognizing Cuneiform Signs Using Graph Based Methods
Authors	Nils M. Kriege, Matthias Fey, Denis Fisseler, Petra Mutzel, Frank Weichert
Abstract	The cuneiform script constitutes one of the earliest systems of writing and is realized by wedge-shaped marks on clay tablets. A tremendous number of cuneiform tablets have already been discovered and are incrementally digitalized and made available to automated processing. As reading cuneiform script is still a manual task, we address the real-world application of recognizing cuneiform signs by two graph based methods with complementary runtime characteristics. We present a graph model for cuneiform signs together with a tailored distance measure based on the concept of the graph edit distance. We propose efficient heuristics for its computation and demonstrate its effectiveness in classification tasks experimentally. To this end, the distance measure is used to implement a nearest neighbor classifier leading to a high computational cost for the prediction phase with increasing training set size. In order to overcome this issue, we propose to use CNNs adapted to graphs as an alternative approach shifting the computational cost to the training phase. We demonstrate the practicability of both approaches in an extensive experimental comparison regarding runtime and prediction accuracy. Although currently available annotated real-world data is still limited, we obtain a high accuracy using CNNs, in particular, when the training set is enriched by augmented examples.
Tasks
Published	2018-02-16
URL	http://arxiv.org/abs/1802.05908v2
PDF	http://arxiv.org/pdf/1802.05908v2.pdf
PWC	https://paperswithcode.com/paper/recognizing-cuneiform-signs-using-graph-based
Repo
Framework

Learning representations for multivariate time series with missing data using Temporal Kernelized Autoencoders


Title	Learning representations for multivariate time series with missing data using Temporal Kernelized Autoencoders
Authors	Filippo Maria Bianchi, Lorenzo Livi, Karl Øyvind Mikalsen, Michael Kampffmeyer, Robert Jenssen
Abstract	Learning compressed representations of multivariate time series (MTS) facilitates data analysis in the presence of noise and redundant information, and for a large number of variates and time steps. However, classical dimensionality reduction approaches are designed for vectorial data and cannot deal explicitly with missing values. In this work, we propose a novel autoencoder architecture based on recurrent neural networks to generate compressed representations of MTS. The proposed model can process inputs characterized by variable lengths and it is specifically designed to handle missing data. Our autoencoder learns fixed-length vectorial representations, whose pairwise similarities are aligned to a kernel function that operates in input space and that handles missing values. This allows to learn good representations, even in the presence of a significant amount of missing data. To show the effectiveness of the proposed approach, we evaluate the quality of the learned representations in several classification tasks, including those involving medical data, and we compare to other methods for dimensionality reduction. Successively, we design two frameworks based on the proposed architecture: one for imputing missing data and another for one-class classification. Finally, we analyze under what circumstances an autoencoder with recurrent layers can learn better compressed representations of MTS than feed-forward architectures.
Tasks	Dimensionality Reduction, Imputation, Time Series, Time Series Classification
Published	2018-05-09
URL	https://arxiv.org/abs/1805.03473v2
PDF	https://arxiv.org/pdf/1805.03473v2.pdf
PWC	https://paperswithcode.com/paper/learning-representations-for-multivariate
Repo
Framework