October 16, 2019

2812 words 14 mins read

Paper Group ANR 1054

A Spectral View of Adversarially Robust Features. cpSGD: Communication-efficient and differentially-private distributed SGD. Driving maneuvers prediction based on cognition-driven and data-driven method. Unsupervised Pathology Image Segmentation Using Representation Learning with Spherical K-means. Tight Dimension Independent Lower Bound on the Exp …

A Spectral View of Adversarially Robust Features


Title	A Spectral View of Adversarially Robust Features
Authors	Shivam Garg, Vatsal Sharan, Brian Hu Zhang, Gregory Valiant
Abstract	Given the apparent difficulty of learning models that are robust to adversarial perturbations, we propose tackling the simpler problem of developing adversarially robust features. Specifically, given a dataset and metric of interest, the goal is to return a function (or multiple functions) that 1) is robust to adversarial perturbations, and 2) has significant variation across the datapoints. We establish strong connections between adversarially robust features and a natural spectral property of the geometry of the dataset and metric of interest. This connection can be leveraged to provide both robust features, and a lower bound on the robustness of any function that has significant variance across the dataset. Finally, we provide empirical evidence that the adversarially robust features given by this spectral approach can be fruitfully leveraged to learn a robust (and accurate) model.
Tasks
Published	2018-11-15
URL	http://arxiv.org/abs/1811.06609v1
PDF	http://arxiv.org/pdf/1811.06609v1.pdf
PWC	https://paperswithcode.com/paper/a-spectral-view-of-adversarially-robust
Repo
Framework

cpSGD: Communication-efficient and differentially-private distributed SGD


Title	cpSGD: Communication-efficient and differentially-private distributed SGD
Authors	Naman Agarwal, Ananda Theertha Suresh, Felix Yu, Sanjiv Kumar, H. Brendan Mcmahan
Abstract	Distributed stochastic gradient descent is an important subroutine in distributed learning. A setting of particular interest is when the clients are mobile devices, where two important concerns are communication efficiency and the privacy of the clients. Several recent works have focused on reducing the communication cost or introducing privacy guarantees, but none of the proposed communication efficient methods are known to be privacy preserving and none of the known privacy mechanisms are known to be communication efficient. To this end, we study algorithms that achieve both communication efficiency and differential privacy. For $d$ variables and $n \approx d$ clients, the proposed method uses $O(\log \log(nd))$ bits of communication per client per coordinate and ensures constant privacy. We also extend and improve previous analysis of the \emph{Binomial mechanism} showing that it achieves nearly the same utility as the Gaussian mechanism, while requiring fewer representation bits, which can be of independent interest.
Tasks
Published	2018-05-27
URL	http://arxiv.org/abs/1805.10559v1
PDF	http://arxiv.org/pdf/1805.10559v1.pdf
PWC	https://paperswithcode.com/paper/cpsgd-communication-efficient-and
Repo
Framework

Driving maneuvers prediction based on cognition-driven and data-driven method


Title	Driving maneuvers prediction based on cognition-driven and data-driven method
Authors	Dong Zhou, Huimin Ma, Yuhan Dong
Abstract	Advanced Driver Assistance Systems (ADAS) improve driving safety significantly. They alert drivers from unsafe traffic conditions when a dangerous maneuver appears. Traditional methods to predict driving maneuvers are mostly based on data-driven models alone. However, existing methods to understand the driver’s intention remain an ongoing challenge due to a lack of intersection of human cognition and data analysis. To overcome this challenge, we propose a novel method that combines both the cognition-driven model and the data-driven model. We introduce a model named Cognitive Fusion-RNN (CF-RNN) which fuses the data inside the vehicle and the data outside the vehicle in a cognitive way. The CF-RNN model consists of two Long Short-Term Memory (LSTM) branches regulated by human reaction time. Experiments on the Brain4Cars benchmark dataset demonstrate that the proposed method outperforms previous methods and achieves state-of-the-art performance.
Tasks
Published	2018-05-08
URL	http://arxiv.org/abs/1805.02895v1
PDF	http://arxiv.org/pdf/1805.02895v1.pdf
PWC	https://paperswithcode.com/paper/driving-maneuvers-prediction-based-on
Repo
Framework

Unsupervised Pathology Image Segmentation Using Representation Learning with Spherical K-means


Title	Unsupervised Pathology Image Segmentation Using Representation Learning with Spherical K-means
Authors	Takayasu Moriya, Holger R. Roth, Shota Nakamura, Hirohisa Oda, Kai Nagara, Masahiro Oda, Kensaku Mori
Abstract	This paper presents a novel method for unsupervised segmentation of pathology images. Staging of lung cancer is a major factor of prognosis. Measuring the maximum dimensions of the invasive component in a pathology images is an essential task. Therefore, image segmentation methods for visualizing the extent of invasive and noninvasive components on pathology images could support pathological examination. However, it is challenging for most of the recent segmentation methods that rely on supervised learning to cope with unlabeled pathology images. In this paper, we propose a unified approach to unsupervised representation learning and clustering for pathology image segmentation. Our method consists of two phases. In the first phase, we learn feature representations of training patches from a target image using the spherical k-means. The purpose of this phase is to obtain cluster centroids which could be used as filters for feature extraction. In the second phase, we apply conventional k-means to the representations extracted by the centroids and then project cluster labels to the target images. We evaluated our methods on pathology images of lung cancer specimen. Our experiments showed that the proposed method outperforms traditional k-means segmentation and the multithreshold Otsu method both quantitatively and qualitatively with an improved normalized mutual information (NMI) score of 0.626 compared to 0.168 and 0.167, respectively. Furthermore, we found that the centroids can be applied to the segmentation of other slices from the same sample.
Tasks	Representation Learning, Semantic Segmentation, Unsupervised Representation Learning
Published	2018-04-11
URL	http://arxiv.org/abs/1804.03828v1
PDF	http://arxiv.org/pdf/1804.03828v1.pdf
PWC	https://paperswithcode.com/paper/unsupervised-pathology-image-segmentation
Repo
Framework

Tight Dimension Independent Lower Bound on the Expected Convergence Rate for Diminishing Step Sizes in SGD


Title	Tight Dimension Independent Lower Bound on the Expected Convergence Rate for Diminishing Step Sizes in SGD
Authors	Phuong Ha Nguyen, Lam M. Nguyen, Marten van Dijk
Abstract	We study the convergence of Stochastic Gradient Descent (SGD) for strongly convex objective functions. We prove for all $t$ a lower bound on the expected convergence rate after the $t$-th SGD iteration; the lower bound is over all possible sequences of diminishing step sizes. It implies that recently proposed sequences of step sizes at ICML 2018 and ICML 2019 are {\em universally} close to optimal in that the expected convergence rate after {\em each} iteration is within a factor $32$ of our lower bound. This factor is independent of dimension $d$. We offer a framework for comparing with lower bounds in state-of-the-art literature and when applied to SGD for strongly convex objective functions our lower bound is a significant factor $775\cdot d$ larger compared to existing work.
Tasks
Published	2018-10-10
URL	https://arxiv.org/abs/1810.04723v4
PDF	https://arxiv.org/pdf/1810.04723v4.pdf
PWC	https://paperswithcode.com/paper/tight-dimension-independent-lower-bound-on
Repo
Framework

The Power of Linear Recurrent Neural Networks


Title	The Power of Linear Recurrent Neural Networks
Authors	Frieder Stolzenburg, Sandra Litz, Olivia Michael, Oliver Obst
Abstract	Recurrent neural networks are a powerful means to cope with time series. We show how a type of linearly activated recurrent neural networks, which we call predictive neural networks, can approximate any time-dependent function f(t) given by a number of function values. The approximation can effectively be learned by simply solving a linear equation system; no backpropagation or similar methods are needed. Furthermore, the network size can be reduced by taking only most relevant components. Thus, in contrast to others, our approach not only learns network weights but also the network architecture. The networks have interesting properties: They end up in ellipse trajectories in the long run and allow the prediction of further values and compact representations of functions. We demonstrate this by several experiments, among them multiple superimposed oscillators (MSO), robotic soccer, and predicting stock prices. Predictive neural networks outperform the previous state-of-the-art for the MSO task with a minimal number of units.
Tasks	Time Series
Published	2018-02-09
URL	https://arxiv.org/abs/1802.03308v4
PDF	https://arxiv.org/pdf/1802.03308v4.pdf
PWC	https://paperswithcode.com/paper/the-power-of-linear-recurrent-neural-networks
Repo
Framework

High-Quality Automatic Foreground Extraction Using Consensus Equilibrium


Title	High-Quality Automatic Foreground Extraction Using Consensus Equilibrium
Authors	Xiran Wang, Jason Juang, Stanley H. Chan
Abstract	Extracting accurate foreground objects from a scene is a fundamental step in creating virtual reality content. However, majority of the professional softwares today still require human interventions, e.g., providing trimaps or labeling key frames. This is not only time consuming, but is also error prone. In this paper, we present a fully automatic algorithm to extract foreground objects. Our solution is based on a newly developed concept called the Multi-Agent Consensus Equilibrium (MACE), a framework which allows us to integrate multiple sources of expertise to produce an overall superior result. Our MACE framework consists of three agents: (1) A new dual layer closed-form matting agent to estimate the foreground mask using the color image and a background image; (2) A background probability estimator using color difference and object segmentation; (3) A total variation minimization agent to control the smoothness of the foreground masks. We show how these agents are constructed, and how their interactions lead to better performance. The algorithm is evaluated by comparing to several state-of-the-art methods. Extensive experimental study shows that the proposed method has less error compared to the competing methods.
Tasks	Semantic Segmentation
Published	2018-08-24
URL	http://arxiv.org/abs/1808.08210v2
PDF	http://arxiv.org/pdf/1808.08210v2.pdf
PWC	https://paperswithcode.com/paper/automatic-foreground-extraction-using-multi
Repo
Framework

A Scalable Neural Shortlisting-Reranking Approach for Large-Scale Domain Classification in Natural Language Understanding


Title	A Scalable Neural Shortlisting-Reranking Approach for Large-Scale Domain Classification in Natural Language Understanding
Authors	Young-Bum Kim, Dongchan Kim, Joo-Kyung Kim, Ruhi Sarikaya
Abstract	Intelligent personal digital assistants (IPDAs), a popular real-life application with spoken language understanding capabilities, can cover potentially thousands of overlapping domains for natural language understanding, and the task of finding the best domain to handle an utterance becomes a challenging problem on a large scale. In this paper, we propose a set of efficient and scalable neural shortlisting-reranking models for large-scale domain classification in IPDAs. The shortlisting stage focuses on efficiently trimming all domains down to a list of k-best candidate domains, and the reranking stage performs a list-wise reranking of the initial k-best domains with additional contextual information. We show the effectiveness of our approach with extensive experiments on 1,500 IPDA domains.
Tasks	Spoken Language Understanding
Published	2018-04-22
URL	http://arxiv.org/abs/1804.08064v1
PDF	http://arxiv.org/pdf/1804.08064v1.pdf
PWC	https://paperswithcode.com/paper/a-scalable-neural-shortlisting-reranking
Repo
Framework

Reasoning about Discrete and Continuous Noisy Sensors and Effectors in Dynamical Systems


Title	Reasoning about Discrete and Continuous Noisy Sensors and Effectors in Dynamical Systems
Authors	Vaishak Belle, Hector J. Levesque
Abstract	Among the many approaches for reasoning about degrees of belief in the presence of noisy sensing and acting, the logical account proposed by Bacchus, Halpern, and Levesque is perhaps the most expressive. While their formalism is quite general, it is restricted to fluents whose values are drawn from discrete finite domains, as opposed to the continuous domains seen in many robotic applications. In this work, we show how this limitation in that approach can be lifted. By dealing seamlessly with both discrete distributions and continuous densities within a rich theory of action, we provide a very general logical specification of how belief should change after acting and sensing in complex noisy domains.
Tasks
Published	2018-09-14
URL	http://arxiv.org/abs/1809.05314v1
PDF	http://arxiv.org/pdf/1809.05314v1.pdf
PWC	https://paperswithcode.com/paper/reasoning-about-discrete-and-continuous-noisy
Repo
Framework

Discovering Features in Sr$_{14}$Cu$_{24}$O$_{41}$ Neutron Single Crystal Diffraction Data by Cluster Analysis


Title	Discovering Features in Sr$_{14}$Cu$_{24}$O$_{41}$ Neutron Single Crystal Diffraction Data by Cluster Analysis
Authors	Yawei Hui, Yaohua Liu, Byung-Hoon Park
Abstract	To address the SMC’18 data challenge, “Discovering Features in Sr$_{14}$Cu$_{24}$O$_{41}$”, we have used the clustering algorithm “DBSCAN” to separate the diffuse scattering features from the Bragg peaks, which takes into account both spatial and photometric information in the dataset during in the clustering process. We find that, in additional to highly localized Bragg peaks, there exists broad diffuse scattering patterns consisting of distinguishable geometries. Besides these two distinctive features, we also identify a third distinguishable feature submerged in the low signal-to-noise region in the reciprocal space, whose origin remains an open question.
Tasks
Published	2018-09-13
URL	http://arxiv.org/abs/1809.05039v1
PDF	http://arxiv.org/pdf/1809.05039v1.pdf
PWC	https://paperswithcode.com/paper/discovering-features-in-sr_14cu_24o_41
Repo
Framework

Bandit Online Learning with Unknown Delays


Title	Bandit Online Learning with Unknown Delays
Authors	Bingcong Li, Tianyi Chen, Georgios B. Giannakis
Abstract	This paper deals with bandit online learning problems involving feedback of unknown delay that can emerge in multi-armed bandit (MAB) and bandit convex optimization (BCO) settings. MAB and BCO require only values of the objective function involved that become available through feedback, and are used to estimate the gradient appearing in the corresponding iterative algorithms. Since the challenging case of feedback with \emph{unknown} delays prevents one from constructing the sought gradient estimates, existing MAB and BCO algorithms become intractable. For such challenging setups, delayed exploration, exploitation, and exponential (DEXP3) iterations, along with delayed bandit gradient descent (DBGD) iterations are developed for MAB and BCO, respectively. Leveraging a unified analysis framework, it is established that the regret of DEXP3 and DBGD are ${\cal O}\big( \sqrt{K\bar{d}(T+D)} \big)$ and ${\cal O}\big( \sqrt{K(T+D)} \big)$, respectively, where $\bar{d}$ is the maximum delay and $D$ denotes the delay accumulated over $T$ slots. Numerical tests using both synthetic and real data validate the performance of DEXP3 and DBGD.
Tasks
Published	2018-07-09
URL	https://arxiv.org/abs/1807.03205v3
PDF	https://arxiv.org/pdf/1807.03205v3.pdf
PWC	https://paperswithcode.com/paper/bandit-online-learning-with-unknown-delays
Repo
Framework

Leveraging Uncertainty Estimates for Predicting Segmentation Quality


Title	Leveraging Uncertainty Estimates for Predicting Segmentation Quality
Authors	Terrance DeVries, Graham W. Taylor
Abstract	The use of deep learning for medical imaging has seen tremendous growth in the research community. One reason for the slow uptake of these systems in the clinical setting is that they are complex, opaque and tend to fail silently. Outside of the medical imaging domain, the machine learning community has recently proposed several techniques for quantifying model uncertainty (i.e.~a model knowing when it has failed). This is important in practical settings, as we can refer such cases to manual inspection or correction by humans. In this paper, we aim to bring these recent results on estimating uncertainty to bear on two important outputs in deep learning-based segmentation. The first is producing spatial uncertainty maps, from which a clinician can observe where and why a system thinks it is failing. The second is quantifying an image-level prediction of failure, which is useful for isolating specific cases and removing them from automated pipelines. We also show that reasoning about spatial uncertainty, the first output, is a useful intermediate representation for generating segmentation quality predictions, the second output. We propose a two-stage architecture for producing these measures of uncertainty, which can accommodate any deep learning-based medical segmentation pipeline.
Tasks
Published	2018-07-02
URL	http://arxiv.org/abs/1807.00502v1
PDF	http://arxiv.org/pdf/1807.00502v1.pdf
PWC	https://paperswithcode.com/paper/leveraging-uncertainty-estimates-for
Repo
Framework

Joint Cell Nuclei Detection and Segmentation in Microscopy Images Using 3D Convolutional Networks


Title	Joint Cell Nuclei Detection and Segmentation in Microscopy Images Using 3D Convolutional Networks
Authors	Sundaresh Ram, Vicky T. Nguyen, Kirsten H. Limesand, Mert R. Sabuncu
Abstract	We propose a 3D convolutional neural network to simultaneously segment and detect cell nuclei in confocal microscopy images. Mirroring the co-dependency of these tasks, our proposed model consists of two serial components: the first part computes a segmentation of cell bodies, while the second module identifies the centers of these cells. Our model is trained end-to-end from scratch on a mouse parotid salivary gland stem cell nuclei dataset comprising 107 image stacks from three independent cell preparations, each containing several hundred individual cell nuclei in 3D. In our experiments, we conduct a thorough evaluation of both detection accuracy and segmentation quality, on two different datasets. The results show that the proposed method provides significantly improved detection and segmentation accuracy compared to state-of-the-art and benchmark algorithms. Finally, we use a previously described test-time drop-out strategy to obtain uncertainty estimates on our predictions and validate these estimates by demonstrating that they are strongly correlated with accuracy.
Tasks
Published	2018-05-08
URL	http://arxiv.org/abs/1805.02850v2
PDF	http://arxiv.org/pdf/1805.02850v2.pdf
PWC	https://paperswithcode.com/paper/joint-cell-nuclei-detection-and-segmentation
Repo
Framework

Learning with SGD and Random Features


Title	Learning with SGD and Random Features
Authors	Luigi Carratino, Alessandro Rudi, Lorenzo Rosasco
Abstract	Sketching and stochastic gradient methods are arguably the most common techniques to derive efficient large scale learning algorithms. In this paper, we investigate their application in the context of nonparametric statistical learning. More precisely, we study the estimator defined by stochastic gradient with mini batches and random features. The latter can be seen as form of nonlinear sketching and used to define approximate kernel methods. The considered estimator is not explicitly penalized/constrained and regularization is implicit. Indeed, our study highlights how different parameters, such as number of features, iterations, step-size and mini-batch size control the learning properties of the solutions. We do this by deriving optimal finite sample bounds, under standard assumptions. The obtained results are corroborated and illustrated by numerical experiments.
Tasks
Published	2018-07-17
URL	http://arxiv.org/abs/1807.06343v3
PDF	http://arxiv.org/pdf/1807.06343v3.pdf
PWC	https://paperswithcode.com/paper/learning-with-sgd-and-random-features
Repo
Framework

Baselines and a datasheet for the Cerema AWP dataset


Title	Baselines and a datasheet for the Cerema AWP dataset
Authors	Ismaïla Seck, Khouloud Dahmane, Pierre Duthon, Gaëlle Loosli
Abstract	This paper presents the recently published Cerema AWP (Adverse Weather Pedestrian) dataset for various machine learning tasks and its exports in machine learning friendly format. We explain why this dataset can be interesting (mainly because it is a greatly controlled and fully annotated image dataset) and present baseline results for various tasks. Moreover, we decided to follow the very recent suggestions of datasheets for dataset, trying to standardize all the available information of the dataset, with a transparency objective.
Tasks
Published	2018-06-11
URL	http://arxiv.org/abs/1806.04016v1
PDF	http://arxiv.org/pdf/1806.04016v1.pdf
PWC	https://paperswithcode.com/paper/baselines-and-a-datasheet-for-the-cerema-awp
Repo
Framework