October 20, 2019

3149 words 15 mins read

Paper Group ANR 23

The decoupled extended Kalman filter for dynamic exponential-family factorization models. Learning for Single-Shot Confidence Calibration in Deep Neural Networks through Stochastic Inferences. OCT segmentation: Integrating open parametric contour model of the retinal layers and shape constraint to the Mumford-Shah functional. Data-driven Upsampling …

The decoupled extended Kalman filter for dynamic exponential-family factorization models


Title	The decoupled extended Kalman filter for dynamic exponential-family factorization models
Authors	Carlos Alberto Gomez-Uribe, Brian Karrer
Abstract	We specialize the decoupled extended Kalman filter (DEKF) for online parameter learning in factorization models, including factorization machines, matrix and tensor factorization, and illustrate the effectiveness of the approach through simulations. Learning model parameters through the DEKF makes factorization models more broadly useful by allowing for more flexible observations through the entire exponential family, modeling parameter drift, and producing parameter uncertainty estimates that can enable explore/exploit and other applications. We use a more general dynamics of the parameters than the standard DEKF, allowing parameter drift while encouraging reasonable values. We also present an alternate derivation of the regular extended Kalman filter and DEKF that connects these methods to natural gradient methods, and suggests a similarly decoupled version of the iterated extended Kalman filter.
Tasks
Published	2018-06-26
URL	http://arxiv.org/abs/1806.09976v1
PDF	http://arxiv.org/pdf/1806.09976v1.pdf
PWC	https://paperswithcode.com/paper/the-decoupled-extended-kalman-filter-for
Repo
Framework

Learning for Single-Shot Confidence Calibration in Deep Neural Networks through Stochastic Inferences


Title	Learning for Single-Shot Confidence Calibration in Deep Neural Networks through Stochastic Inferences
Authors	Seonguk Seo, Paul Hongsuck Seo, Bohyung Han
Abstract	We propose a generic framework to calibrate accuracy and confidence of a prediction in deep neural networks through stochastic inferences. We interpret stochastic regularization using a Bayesian model, and analyze the relation between predictive uncertainty of networks and variance of the prediction scores obtained by stochastic inferences for a single example. Our empirical study shows that the accuracy and the score of a prediction are highly correlated with the variance of multiple stochastic inferences given by stochastic depth or dropout. Motivated by this observation, we design a novel variance-weighted confidence-integrated loss function that is composed of two cross-entropy loss terms with respect to ground-truth and uniform distribution, which are balanced by variance of stochastic prediction scores. The proposed loss function enables us to learn deep neural networks that predict confidence calibrated scores using a single inference. Our algorithm presents outstanding confidence calibration performance and improves classification accuracy when combined with two popular stochastic regularization techniques—stochastic depth and dropout—in multiple models and datasets; it alleviates overconfidence issue in deep neural networks significantly by training networks to achieve prediction accuracy proportional to confidence of prediction.
Tasks	Calibration
Published	2018-09-28
URL	http://arxiv.org/abs/1809.10877v5
PDF	http://arxiv.org/pdf/1809.10877v5.pdf
PWC	https://paperswithcode.com/paper/learning-for-single-shot-confidence
Repo
Framework

OCT segmentation: Integrating open parametric contour model of the retinal layers and shape constraint to the Mumford-Shah functional


Title	OCT segmentation: Integrating open parametric contour model of the retinal layers and shape constraint to the Mumford-Shah functional
Authors	Jinming Duan, Weicheng Xie, Ryan Wen Liu, Christopher Tench, Irene Gottlob, Frank Proudlock, Li Bai
Abstract	In this paper, we propose a novel retinal layer boundary model for segmentation of optical coherence tomography (OCT) images. The retinal layer boundary model consists of 9 open parametric contours representing the 9 retinal layers in OCT images. An intensity-based Mumford-Shah (MS) variational functional is first defined to evolve the retinal layer boundary model to segment the 9 layers simultaneously. By making use of the normals of open parametric contours, we construct equal sized adjacent narrowbands that are divided by each contour. Regional information in each narrowband can thus be integrated into the MS energy functional such that its optimisation is robust against different initialisations. A statistical prior is also imposed on the shape of the segmented parametric contours for the functional. As such, by minimising the MS energy functional the parametric contours can be driven towards the true boundaries of retinal layers, while the similarity of the contours with respect to training OCT shapes is preserved. Experimental results on real OCT images demonstrate that the method is accurate and robust to low quality OCT images with low contrast and high-level speckle noise, and it outperforms the recent geodesic distance based method for segmenting 9 layers of the retina in OCT images.
Tasks
Published	2018-08-08
URL	http://arxiv.org/abs/1808.02917v1
PDF	http://arxiv.org/pdf/1808.02917v1.pdf
PWC	https://paperswithcode.com/paper/oct-segmentation-integrating-open-parametric
Repo
Framework

Data-driven Upsampling of Point Clouds


Title	Data-driven Upsampling of Point Clouds
Authors	Wentai Zhang, Haoliang Jiang, Zhangsihao Yang, Soji Yamakawa, Kenji Shimada, Levent Burak Kara
Abstract	High quality upsampling of sparse 3D point clouds is critically useful for a wide range of geometric operations such as reconstruction, rendering, meshing, and analysis. In this paper, we propose a data-driven algorithm that enables an upsampling of 3D point clouds without the need for hard-coded rules. Our approach uses a deep network with Chamfer distance as the loss function, capable of learning the latent features in point clouds belonging to different object categories. We evaluate our algorithm across different amplification factors, with upsampling learned and performed on objects belonging to the same category as well as different categories. We also explore the desirable characteristics of input point clouds as a function of the distribution of the point samples. Finally, we demonstrate the performance of our algorithm in single-category training versus multi-category training scenarios. The final proposed model is compared against a baseline, optimization-based upsampling method. Results indicate that our algorithm is capable of generating more uniform and accurate upsamplings.
Tasks
Published	2018-07-08
URL	http://arxiv.org/abs/1807.02740v2
PDF	http://arxiv.org/pdf/1807.02740v2.pdf
PWC	https://paperswithcode.com/paper/data-driven-upsampling-of-point-clouds
Repo
Framework

Towards Linear Time Neural Machine Translation with Capsule Networks


Title	Towards Linear Time Neural Machine Translation with Capsule Networks
Authors	Mingxuan Wang, Jun Xie, Zhixing Tan, Jinsong Su, Deyi xiong, Lei Li
Abstract	In this study, we first investigate a novel capsule network with dynamic routing for linear time Neural Machine Translation (NMT), referred as \textsc{CapsNMT}. \textsc{CapsNMT} uses an aggregation mechanism to map the source sentence into a matrix with pre-determined size, and then applys a deep LSTM network to decode the target sequence from the source representation. Unlike the previous work \cite{sutskever2014sequence} to store the source sentence with a passive and bottom-up way, the dynamic routing policy encodes the source sentence with an iterative process to decide the credit attribution between nodes from lower and higher layers. \textsc{CapsNMT} has two core properties: it runs in time that is linear in the length of the sequences and provides a more flexible way to select, represent and aggregates the part-whole information of the source sentence. On WMT14 English-German task and a larger WMT14 English-French task, \textsc{CapsNMT} achieves comparable results with the state-of-the-art NMT systems. To the best of our knowledge, this is the first work that capsule networks have been empirically investigated for sequence to sequence problems.
Tasks	Machine Translation
Published	2018-11-01
URL	https://arxiv.org/abs/1811.00287v2
PDF	https://arxiv.org/pdf/1811.00287v2.pdf
PWC	https://paperswithcode.com/paper/towards-linear-time-neural-machine
Repo
Framework

Randomized Value Functions via Multiplicative Normalizing Flows


Title	Randomized Value Functions via Multiplicative Normalizing Flows
Authors	Ahmed Touati, Harsh Satija, Joshua Romoff, Joelle Pineau, Pascal Vincent
Abstract	Randomized value functions offer a promising approach towards the challenge of efficient exploration in complex environments with high dimensional state and action spaces. Unlike traditional point estimate methods, randomized value functions maintain a posterior distribution over action-space values. This prevents the agent’s behavior policy from prematurely exploiting early estimates and falling into local optima. In this work, we leverage recent advances in variational Bayesian neural networks and combine these with traditional Deep Q-Networks (DQN) and Deep Deterministic Policy Gradient (DDPG) to achieve randomized value functions for high-dimensional domains. In particular, we augment DQN and DDPG with multiplicative normalizing flows in order to track a rich approximate posterior distribution over the parameters of the value function. This allows the agent to perform approximate Thompson sampling in a computationally efficient manner via stochastic gradient methods. We demonstrate the benefits of our approach through an empirical comparison in high dimensional environments.
Tasks	Efficient Exploration
Published	2018-06-06
URL	https://arxiv.org/abs/1806.02315v3
PDF	https://arxiv.org/pdf/1806.02315v3.pdf
PWC	https://paperswithcode.com/paper/randomized-value-functions-via-multiplicative
Repo
Framework

Child Gender Determination with Convolutional Neural Networks on Hand Radio-Graphs


Title	Child Gender Determination with Convolutional Neural Networks on Hand Radio-Graphs
Authors	Mumtaz A. Kaloi, Kun He
Abstract	Motivation: In forensic or medico-legal investigation as well as in anthropology the gender determination of the subject (hit by a disastrous or any kind of traumatic situation) is mostly the first step. In state-of-the-art techniques the gender is determined by examining dimensions of the bones of skull and the pelvis area. In worse situations when there is only a small portion of the human remains to be investigated and the subject is a child, we need alternative techniques to determine the gender of the subject. In this work we propose a technique called GDCNN (Gender Determination with Convolutional Neural Networks), where the left hand radio-graphs of the children between a wide range of ages in 1 month to 18 years are examined to determine the gender. To our knowledge this technique is first of its kind. Further to identify the area of the attention we used Class Activation Mapping (CAM). Results: The results suggest the accuracy of the model is as high as 98%, which is very convincing by taking into account the incompletely grown skeleton of the children. The attention observed with CAM discovers that the lower part of the hand around carpals (wrist) is more important for child gender determination.
Tasks
Published	2018-11-13
URL	http://arxiv.org/abs/1811.05180v1
PDF	http://arxiv.org/pdf/1811.05180v1.pdf
PWC	https://paperswithcode.com/paper/child-gender-determination-with-convolutional
Repo
Framework

A Web-scale system for scientific knowledge exploration


Title	A Web-scale system for scientific knowledge exploration
Authors	Zhihong Shen, Hao Ma, Kuansan Wang
Abstract	To enable efficient exploration of Web-scale scientific knowledge, it is necessary to organize scientific publications into a hierarchical concept structure. In this work, we present a large-scale system to (1) identify hundreds of thousands of scientific concepts, (2) tag these identified concepts to hundreds of millions of scientific publications by leveraging both text and graph structure, and (3) build a six-level concept hierarchy with a subsumption-based model. The system builds the most comprehensive cross-domain scientific concept ontology published to date, with more than 200 thousand concepts and over one million relationships.
Tasks	Efficient Exploration
Published	2018-05-30
URL	http://arxiv.org/abs/1805.12216v1
PDF	http://arxiv.org/pdf/1805.12216v1.pdf
PWC	https://paperswithcode.com/paper/a-web-scale-system-for-scientific-knowledge
Repo
Framework

Training Convolutional Neural Networks and Compressed Sensing End-to-End for Microscopy Cell Detection


Title	Training Convolutional Neural Networks and Compressed Sensing End-to-End for Microscopy Cell Detection
Authors	Yao Xue, Gilbert Bigras, Judith Hugh, Nilanjan Ray
Abstract	Automated cell detection and localization from microscopy images are significant tasks in biomedical research and clinical practice. In this paper, we design a new cell detection and localization algorithm that combines deep convolutional neural network (CNN) and compressed sensing (CS) or sparse coding (SC) for end-to-end training. We also derive, for the first time, a backpropagation rule, which is applicable to train any algorithm that implements a sparse code recovery layer. The key observation behind our algorithm is that cell detection task is a point object detection task in computer vision, where the cell centers (i.e., point objects) occupy only a tiny fraction of the total number of pixels in an image. Thus, we can apply compressed sensing (or, equivalently sparse coding) to compactly represent a variable number of cells in a projected space. Then, CNN regresses this compressed vector from the input microscopy image. Thanks to the SC/CS recovery algorithm (L1 optimization) that can recover sparse cell locations from the output of CNN. We train this entire processing pipeline end-to-end and demonstrate that end-to-end training provides accuracy improvements over a training paradigm that treats CNN and CS-recovery layers separately. Our algorithm design also takes into account a form of ensemble average of trained models naturally to further boost accuracy of cell detection. We have validated our algorithm on benchmark datasets and achieved excellent performances.
Tasks	Object Detection
Published	2018-10-07
URL	http://arxiv.org/abs/1810.03075v1
PDF	http://arxiv.org/pdf/1810.03075v1.pdf
PWC	https://paperswithcode.com/paper/training-convolutional-neural-networks-and
Repo
Framework

When Simple Exploration is Sample Efficient: Identifying Sufficient Conditions for Random Exploration to Yield PAC RL Algorithms


Title	When Simple Exploration is Sample Efficient: Identifying Sufficient Conditions for Random Exploration to Yield PAC RL Algorithms
Authors	Yao Liu, Emma Brunskill
Abstract	Efficient exploration is one of the key challenges for reinforcement learning (RL) algorithms. Most traditional sample efficiency bounds require strategic exploration. Recently many deep RL algorithms with simple heuristic exploration strategies that have few formal guarantees, achieve surprising success in many domains. These results pose an important question about understanding these exploration strategies such as $e$-greedy, as well as understanding what characterize the difficulty of exploration in MDPs. In this work we propose problem specific sample complexity bounds of $Q$ learning with random walk exploration that rely on several structural properties. We also link our theoretical results to some empirical benchmark domains, to illustrate if our bound gives polynomial sample complexity in these domains and how that is related with the empirical performance.
Tasks	Efficient Exploration, Q-Learning
Published	2018-05-23
URL	http://arxiv.org/abs/1805.09045v4
PDF	http://arxiv.org/pdf/1805.09045v4.pdf
PWC	https://paperswithcode.com/paper/when-simple-exploration-is-sample-efficient
Repo
Framework

When Bifidelity Meets CoKriging: An Efficient Physics-Informed Multifidelity Method


Title	When Bifidelity Meets CoKriging: An Efficient Physics-Informed Multifidelity Method
Authors	Xiu Yang, Xueyu Zhu, Jing Li
Abstract	In this work, we propose a framework that combines the approximation-theory-based multifidelity method and Gaussian-process-regression-based multifidelity method to achieve data-model convergence when stochastic simulation models and sparse accurate observation data are available. Specifically, the two types of multifidelity methods we use are the bifidelity and CoKriging methods. The new approach uses the bifidelity method to efficiently estimate the empirical mean and covariance of the stochastic simulation outputs, then it uses these statistics to construct a Gaussian process (GP) representing low-fidelity in CoKriging. We also combine the bifidelity method with Kriging, where the approximated empirical statistics are used to construct the GP as well. We prove that the resulting posterior mean by the new physics-informed approach preserves linear physical constraints up to an error bound. By using this method, we can obtain an accurate construction of a state of interest based on a partially correct physical model and a few accurate observations. We present numerical examples to demonstrate performance of the method.
Tasks
Published	2018-12-07
URL	http://arxiv.org/abs/1812.02919v1
PDF	http://arxiv.org/pdf/1812.02919v1.pdf
PWC	https://paperswithcode.com/paper/when-bifidelity-meets-cokriging-an-efficient
Repo
Framework

Underwater Single Image Color Restoration Using Haze-Lines and a New Quantitative Dataset


Title	Underwater Single Image Color Restoration Using Haze-Lines and a New Quantitative Dataset
Authors	Dana Berman, Deborah Levy, Shai Avidan, Tali Treibitz
Abstract	Underwater images suffer from color distortion and low contrast, because light is attenuated while it propagates through water. Attenuation under water varies with wavelength, unlike terrestrial images where attenuation is assumed to be spectrally uniform. The attenuation depends both on the water body and the 3D structure of the scene, making color restoration difficult. Unlike existing single underwater image enhancement techniques, our method takes into account multiple spectral profiles of different water types. By estimating just two additional global parameters: the attenuation ratios of the blue-red and blue-green color channels, the problem is reduced to single image dehazing, where all color channels have the same attenuation coefficients. Since the water type is unknown, we evaluate different parameters out of an existing library of water types. Each type leads to a different restored image and the best result is automatically chosen based on color distribution. We collected a dataset of images taken in different locations with varying water properties, showing color charts in the scenes. Moreover, to obtain ground truth, the 3D structure of the scene was calculated based on stereo imaging. This dataset enables a quantitative evaluation of restoration algorithms on natural images and shows the advantage of our method.
Tasks	Image Dehazing, Image Enhancement, Single Image Dehazing
Published	2018-11-04
URL	http://arxiv.org/abs/1811.01343v3
PDF	http://arxiv.org/pdf/1811.01343v3.pdf
PWC	https://paperswithcode.com/paper/underwater-single-image-color-restoration
Repo
Framework

Partial Evaluation of Logic Programs in Vector Spaces


Title	Partial Evaluation of Logic Programs in Vector Spaces
Authors	Chiaki Sakama, Hien D. Nguyen, Taisuke Sato, Katsumi Inoue
Abstract	In this paper, we introduce methods of encoding propositional logic programs in vector spaces. Interpretations are represented by vectors and programs are represented by matrices. The least model of a definite program is computed by multiplying an interpretation vector and a program matrix. To optimize computation in vector spaces, we provide a method of partial evaluation of programs using linear algebra. Partial evaluation is done by unfolding rules in a program, and it is realized in a vector space by multiplying program matrices. We perform experiments using randomly generated programs and show that partial evaluation has potential for realizing efficient computation in huge scale of programs.
Tasks
Published	2018-11-28
URL	http://arxiv.org/abs/1811.11435v1
PDF	http://arxiv.org/pdf/1811.11435v1.pdf
PWC	https://paperswithcode.com/paper/partial-evaluation-of-logic-programs-in
Repo
Framework

Deep Semantic Instance Segmentation of Tree-like Structures Using Synthetic Data


Title	Deep Semantic Instance Segmentation of Tree-like Structures Using Synthetic Data
Authors	Kerry Halupka, Rahil Garnavi, Stephen Moore
Abstract	Tree-like structures, such as blood vessels, often express complexity at very fine scales, requiring high-resolution grids to adequately describe their shape. Such sparse morphology can alternately be represented by locations of centreline points, but learning from this type of data with deep learning is challenging due to it being unordered, and permutation invariant. In this work, we propose a deep neural network that directly consumes unordered points along the centreline of a branching structure, to identify the topology of the represented structure in a single-shot. Key to our approach is the use of a novel multi-task loss function, enabling instance segmentation of arbitrarily complex branching structures. We train the network solely using synthetically generated data, utilizing domain randomization to facilitate the transfer to real 2D and 3D data. Results show that our network can reliably extract meaningful information about branch locations, bifurcations and endpoints, and sets a new benchmark for semantic instance segmentation in branching structures.
Tasks	Instance Segmentation, Semantic Segmentation
Published	2018-11-08
URL	http://arxiv.org/abs/1811.03208v1
PDF	http://arxiv.org/pdf/1811.03208v1.pdf
PWC	https://paperswithcode.com/paper/deep-semantic-instance-segmentation-of-tree
Repo
Framework

Cost-aware Cascading Bandits


Title	Cost-aware Cascading Bandits
Authors	Ruida Zhou, Chao Gan, Jing Yan, Cong Shen
Abstract	In this paper, we propose a cost-aware cascading bandits model, a new variant of multi-armed ban- dits with cascading feedback, by considering the random cost of pulling arms. In each step, the learning agent chooses an ordered list of items and examines them sequentially, until certain stopping condition is satisfied. Our objective is then to max- imize the expected net reward in each step, i.e., the reward obtained in each step minus the total cost in- curred in examining the items, by deciding the or- dered list of items, as well as when to stop examina- tion. We study both the offline and online settings, depending on whether the state and cost statistics of the items are known beforehand. For the of- fline setting, we show that the Unit Cost Ranking with Threshold 1 (UCR-T1) policy is optimal. For the online setting, we propose a Cost-aware Cas- cading Upper Confidence Bound (CC-UCB) algo- rithm, and show that the cumulative regret scales in O(log T ). We also provide a lower bound for all {\alpha}-consistent policies, which scales in {\Omega}(log T ) and matches our upper bound. The performance of the CC-UCB algorithm is evaluated with both synthetic and real-world data.
Tasks
Published	2018-05-22
URL	http://arxiv.org/abs/1805.08638v1
PDF	http://arxiv.org/pdf/1805.08638v1.pdf
PWC	https://paperswithcode.com/paper/cost-aware-cascading-bandits
Repo
Framework