July 27, 2019

3054 words 15 mins read

Paper Group ANR 707

Performance Limits of Stochastic Sub-Gradient Learning, Part II: Multi-Agent Case. Is human face processing a feature- or pattern-based task? Evidence using a unified computational method driven by eye movements. Restricted Causal Inference Algorithm. Non-Linear Phase-Shifting of Haar Wavelets for Run-Time All-Frequency Lighting. Blood capillaries …

Performance Limits of Stochastic Sub-Gradient Learning, Part II: Multi-Agent Case


Title	Performance Limits of Stochastic Sub-Gradient Learning, Part II: Multi-Agent Case
Authors	Bicheng Ying, Ali H. Sayed
Abstract	The analysis in Part I revealed interesting properties for subgradient learning algorithms in the context of stochastic optimization when gradient noise is present. These algorithms are used when the risk functions are non-smooth and involve non-differentiable components. They have been long recognized as being slow converging methods. However, it was revealed in Part I that the rate of convergence becomes linear for stochastic optimization problems, with the error iterate converging at an exponential rate $\alpha^i$ to within an $O(\mu)-$neighborhood of the optimizer, for some $\alpha \in (0,1)$ and small step-size $\mu$. The conclusion was established under weaker assumptions than the prior literature and, moreover, several important problems (such as LASSO, SVM, and Total Variation) were shown to satisfy these weaker assumptions automatically (but not the previously used conditions from the literature). These results revealed that sub-gradient learning methods have more favorable behavior than originally thought when used to enable continuous adaptation and learning. The results of Part I were exclusive to single-agent adaptation. The purpose of the current Part II is to examine the implications of these discoveries when a collection of networked agents employs subgradient learning as their cooperative mechanism. The analysis will show that, despite the coupled dynamics that arises in a networked scenario, the agents are still able to attain linear convergence in the stochastic case; they are also able to reach agreement within $O(\mu)$ of the optimizer.
Tasks	Stochastic Optimization
Published	2017-04-20
URL	http://arxiv.org/abs/1704.06025v1
PDF	http://arxiv.org/pdf/1704.06025v1.pdf
PWC	https://paperswithcode.com/paper/performance-limits-of-stochastic-sub-gradient
Repo
Framework

Is human face processing a feature- or pattern-based task? Evidence using a unified computational method driven by eye movements


Title	Is human face processing a feature- or pattern-based task? Evidence using a unified computational method driven by eye movements
Authors	Carlos E. Thomaz, Vagner Amaral, Gilson A. Giraldi, Duncan F. Gillies, Daniel Rueckert
Abstract	Research on human face processing using eye movements has provided evidence that we recognize face images successfully focusing our visual attention on a few inner facial regions, mainly on the eyes, nose and mouth. To understand how we accomplish this process of coding high-dimensional faces so efficiently, this paper proposes and implements a multivariate extraction method that combines face images variance with human spatial attention maps modeled as feature- and pattern-based information sources. It is based on a unified multidimensional representation of the well-known face-space concept. The spatial attention maps are summary statistics of the eye-tracking fixations of a number of participants and trials to frontal and well-framed face images during separate gender and facial expression recognition tasks. Our experimental results carried out on publicly available face databases have indicated that we might emulate the human extraction system as a pattern-based computational method rather than a feature-based one to properly explain the proficiency of the human system in recognizing visual face information.
Tasks	Eye Tracking, Facial Expression Recognition
Published	2017-09-04
URL	http://arxiv.org/abs/1709.01182v1
PDF	http://arxiv.org/pdf/1709.01182v1.pdf
PWC	https://paperswithcode.com/paper/is-human-face-processing-a-feature-or-pattern
Repo
Framework

Restricted Causal Inference Algorithm


Title	Restricted Causal Inference Algorithm
Authors	Mieczysław A. Kłopotek
Abstract	This paper proposes a new algorithm for recovery of belief network structure from data handling hidden variables. It consists essentially in an extension of the CI algorithm of Spirtes et al. by restricting the number of conditional dependencies checked up to k variables and in an extension of the original CI by additional steps transforming so called partial including path graph into a belief network. Its correctness is demonstrated.
Tasks	Causal Inference
Published	2017-06-30
URL	http://arxiv.org/abs/1706.10117v1
PDF	http://arxiv.org/pdf/1706.10117v1.pdf
PWC	https://paperswithcode.com/paper/restricted-causal-inference-algorithm
Repo
Framework

Non-Linear Phase-Shifting of Haar Wavelets for Run-Time All-Frequency Lighting


Title	Non-Linear Phase-Shifting of Haar Wavelets for Run-Time All-Frequency Lighting
Authors	Mais Alnasser, Hassan Foroosh
Abstract	This paper focuses on real-time all-frequency image-based rendering using an innovative solution for run-time computation of light transport. The approach is based on new results derived for non-linear phase shifting in the Haar wavelet domain. Although image-based methods for real-time rendering of dynamic glossy objects have been proposed, they do not truly scale to all possible frequencies and high sampling rates without trading storage, glossiness, or computational time, while varying both lighting and viewpoint. This is due to the fact that current approaches are limited to precomputed radiance transfer (PRT), which is prohibitively expensive in terms of memory requirements and real-time rendering when both varying light and viewpoint changes are required together with high sampling rates for high frequency lighting of glossy material. On the other hand, current methods cannot handle object rotation, which is one of the paramount issues for all PRT methods using wavelets. This latter problem arises because the precomputed data are defined in a global coordinate system and encoded in the wavelet domain, while the object is rotated in a local coordinate system. At the root of all the above problems is the lack of efficient run-time solution to the nontrivial problem of rotating wavelets (a non-linear phase-shift), which we solve in this paper.
Tasks
Published	2017-05-20
URL	http://arxiv.org/abs/1705.07272v1
PDF	http://arxiv.org/pdf/1705.07272v1.pdf
PWC	https://paperswithcode.com/paper/non-linear-phase-shifting-of-haar-wavelets
Repo
Framework

Blood capillaries and vessels segmentation in optical coherence tomography angiogram using fuzzy C-means and Curvelet transform


Title	Blood capillaries and vessels segmentation in optical coherence tomography angiogram using fuzzy C-means and Curvelet transform
Authors	Fariborz Taherkhani
Abstract	This paper has been removed from arXiv as the submitter did not have ownership of the data presented in this work.
Tasks
Published	2017-05-31
URL	http://arxiv.org/abs/1706.00083v1
PDF	http://arxiv.org/pdf/1706.00083v1.pdf
PWC	https://paperswithcode.com/paper/blood-capillaries-and-vessels-segmentation-in
Repo
Framework

ADMM Penalty Parameter Selection by Residual Balancing


Title	ADMM Penalty Parameter Selection by Residual Balancing
Authors	Brendt Wohlberg
Abstract	Appropriate selection of the penalty parameter is crucial to obtaining good performance from the Alternating Direction Method of Multipliers (ADMM). While analytic results for optimal selection of this parameter are very limited, there is a heuristic method that appears to be relatively successful in a number of different problems. The contribution of this paper is to demonstrate that their is a potentially serious flaw in this heuristic approach, and to propose a modification that at least partially addresses it.
Tasks
Published	2017-04-20
URL	http://arxiv.org/abs/1704.06209v1
PDF	http://arxiv.org/pdf/1704.06209v1.pdf
PWC	https://paperswithcode.com/paper/admm-penalty-parameter-selection-by-residual
Repo
Framework

Collaborative-controlled LASSO for Constructing Propensity Score-based Estimators in High-Dimensional Data


Title	Collaborative-controlled LASSO for Constructing Propensity Score-based Estimators in High-Dimensional Data
Authors	Cheng Ju, Richard Wyss, Jessica M. Franklin, Sebastian Schneeweiss, Jenny Häggström, Mark J. van der Laan
Abstract	Propensity score (PS) based estimators are increasingly used for causal inference in observational studies. However, model selection for PS estimation in high-dimensional data has received little attention. In these settings, PS models have traditionally been selected based on the goodness-of-fit for the treatment mechanism itself, without consideration of the causal parameter of interest. Collaborative minimum loss-based estimation (C-TMLE) is a novel methodology for causal inference that takes into account information on the causal parameter of interest when selecting a PS model. This “collaborative learning” considers variable associations with both treatment and outcome when selecting a PS model in order to minimize a bias-variance trade off in the estimated treatment effect. In this study, we introduce a novel approach for collaborative model selection when using the LASSO estimator for PS estimation in high-dimensional covariate settings. To demonstrate the importance of selecting the PS model collaboratively, we designed quasi-experiments based on a real electronic healthcare database, where only the potential outcomes were manually generated, and the treatment and baseline covariates remained unchanged. Results showed that the C-TMLE algorithm outperformed other competing estimators for both point estimation and confidence interval coverage. In addition, the PS model selected by C-TMLE could be applied to other PS-based estimators, which also resulted in substantive improvement for both point estimation and confidence interval coverage. We illustrate the discussed concepts through an empirical example comparing the effects of non-selective nonsteroidal anti-inflammatory drugs with selective COX-2 inhibitors on gastrointestinal complications in a population of Medicare beneficiaries.
Tasks	Causal Inference, Model Selection
Published	2017-06-30
URL	http://arxiv.org/abs/1706.10029v1
PDF	http://arxiv.org/pdf/1706.10029v1.pdf
PWC	https://paperswithcode.com/paper/collaborative-controlled-lasso-for
Repo
Framework

Adaptive Inferential Method for Monotone Graph Invariants


Title	Adaptive Inferential Method for Monotone Graph Invariants
Authors	Junwei Lu, Matey Neykov, Han Liu
Abstract	We consider the problem of undirected graphical model inference. In many applications, instead of perfectly recovering the unknown graph structure, a more realistic goal is to infer some graph invariants (e.g., the maximum degree, the number of connected subgraphs, the number of isolated nodes). In this paper, we propose a new inferential framework for testing nested multiple hypotheses and constructing confidence intervals of the unknown graph invariants under undirected graphical models. Compared to perfect graph recovery, our methods require significantly weaker conditions. This paper makes two major contributions: (i) Methodologically, for testing nested multiple hypotheses, we propose a skip-down algorithm on the whole family of monotone graph invariants (The invariants which are non-decreasing under addition of edges). We further show that the same skip-down algorithm also provides valid confidence intervals for the targeted graph invariants. (ii) Theoretically, we prove that the length of the obtained confidence intervals are optimal and adaptive to the unknown signal strength. We also prove generic lower bounds for the confidence interval length for various invariants. Numerical results on both synthetic simulations and a brain imaging dataset are provided to illustrate the usefulness of the proposed method.
Tasks
Published	2017-07-28
URL	http://arxiv.org/abs/1707.09114v1
PDF	http://arxiv.org/pdf/1707.09114v1.pdf
PWC	https://paperswithcode.com/paper/adaptive-inferential-method-for-monotone
Repo
Framework

A Generic Online Parallel Learning Framework for Large Margin Models


Title	A Generic Online Parallel Learning Framework for Large Margin Models
Authors	Shuming Ma, Xu Sun
Abstract	To speed up the training process, many existing systems use parallel technology for online learning algorithms. However, most research mainly focus on stochastic gradient descent (SGD) instead of other algorithms. We propose a generic online parallel learning framework for large margin models, and also analyze our framework on popular large margin algorithms, including MIRA and Structured Perceptron. Our framework is lock-free and easy to implement on existing systems. Experiments show that systems with our framework can gain near linear speed up by increasing running threads, and with no loss in accuracy.
Tasks
Published	2017-03-02
URL	http://arxiv.org/abs/1703.00786v1
PDF	http://arxiv.org/pdf/1703.00786v1.pdf
PWC	https://paperswithcode.com/paper/a-generic-online-parallel-learning-framework
Repo
Framework

Multi-Player Bandits Revisited


Title	Multi-Player Bandits Revisited
Authors	Lilian Besson, Emilie Kaufmann
Abstract	Multi-player Multi-Armed Bandits (MAB) have been extensively studied in the literature, motivated by applications to Cognitive Radio systems. Driven by such applications as well, we motivate the introduction of several levels of feedback for multi-player MAB algorithms. Most existing work assume that sensing information is available to the algorithm. Under this assumption, we improve the state-of-the-art lower bound for the regret of any decentralized algorithms and introduce two algorithms, RandTopM and MCTopM, that are shown to empirically outperform existing algorithms. Moreover, we provide strong theoretical guarantees for these algorithms, including a notion of asymptotic optimality in terms of the number of selections of bad arms. We then introduce a promising heuristic, called Selfish, that can operate without sensing information, which is crucial for emerging applications to Internet of Things networks. We investigate the empirical performance of this algorithm and provide some first theoretical elements for the understanding of its behavior.
Tasks	Multi-Armed Bandits
Published	2017-11-07
URL	http://arxiv.org/abs/1711.02317v3
PDF	http://arxiv.org/pdf/1711.02317v3.pdf
PWC	https://paperswithcode.com/paper/multi-player-bandits-revisited
Repo
Framework

Machine learning approximation algorithms for high-dimensional fully nonlinear partial differential equations and second-order backward stochastic differential equations


Title	Machine learning approximation algorithms for high-dimensional fully nonlinear partial differential equations and second-order backward stochastic differential equations
Authors	Christian Beck, Weinan E, Arnulf Jentzen
Abstract	High-dimensional partial differential equations (PDE) appear in a number of models from the financial industry, such as in derivative pricing models, credit valuation adjustment (CVA) models, or portfolio optimization models. The PDEs in such applications are high-dimensional as the dimension corresponds to the number of financial assets in a portfolio. Moreover, such PDEs are often fully nonlinear due to the need to incorporate certain nonlinear phenomena in the model such as default risks, transaction costs, volatility uncertainty (Knightian uncertainty), or trading constraints in the model. Such high-dimensional fully nonlinear PDEs are exceedingly difficult to solve as the computational effort for standard approximation methods grows exponentially with the dimension. In this work we propose a new method for solving high-dimensional fully nonlinear second-order PDEs. Our method can in particular be used to sample from high-dimensional nonlinear expectations. The method is based on (i) a connection between fully nonlinear second-order PDEs and second-order backward stochastic differential equations (2BSDEs), (ii) a merged formulation of the PDE and the 2BSDE problem, (iii) a temporal forward discretization of the 2BSDE and a spatial approximation via deep neural nets, and (iv) a stochastic gradient descent-type optimization procedure. Numerical results obtained using ${\rm T{\small ENSOR}F{\small LOW}}$ in ${\rm P{\small YTHON}}$ illustrate the efficiency and the accuracy of the method in the cases of a $100$-dimensional Black-Scholes-Barenblatt equation, a $100$-dimensional Hamilton-Jacobi-Bellman equation, and a nonlinear expectation of a $ 100 $-dimensional $ G $-Brownian motion.
Tasks	Portfolio Optimization
Published	2017-09-18
URL	http://arxiv.org/abs/1709.05963v1
PDF	http://arxiv.org/pdf/1709.05963v1.pdf
PWC	https://paperswithcode.com/paper/machine-learning-approximation-algorithms-for
Repo
Framework

Bayesian Image Quality Transfer with CNNs: Exploring Uncertainty in dMRI Super-Resolution


Title	Bayesian Image Quality Transfer with CNNs: Exploring Uncertainty in dMRI Super-Resolution
Authors	Ryutaro Tanno, Daniel E. Worrall, Aurobrata Ghosh, Enrico Kaden, Stamatios N. Sotiropoulos, Antonio Criminisi, Daniel C. Alexander
Abstract	In this work, we investigate the value of uncertainty modeling in 3D super-resolution with convolutional neural networks (CNNs). Deep learning has shown success in a plethora of medical image transformation problems, such as super-resolution (SR) and image synthesis. However, the highly ill-posed nature of such problems results in inevitable ambiguity in the learning of networks. We propose to account for intrinsic uncertainty through a per-patch heteroscedastic noise model and for parameter uncertainty through approximate Bayesian inference in the form of variational dropout. We show that the combined benefits of both lead to the state-of-the-art performance SR of diffusion MR brain images in terms of errors compared to ground truth. We further show that the reduced error scores produce tangible benefits in downstream tractography. In addition, the probabilistic nature of the methods naturally confers a mechanism to quantify uncertainty over the super-resolved output. We demonstrate through experiments on both healthy and pathological brains the potential utility of such an uncertainty measure in the risk assessment of the super-resolved images for subsequent clinical use.
Tasks	Bayesian Inference, Image Generation, Super-Resolution
Published	2017-05-01
URL	http://arxiv.org/abs/1705.00664v2
PDF	http://arxiv.org/pdf/1705.00664v2.pdf
PWC	https://paperswithcode.com/paper/bayesian-image-quality-transfer-with-cnns
Repo
Framework

An Efficient Approach for Object Detection and Tracking of Objects in a Video with Variable Background


Title	An Efficient Approach for Object Detection and Tracking of Objects in a Video with Variable Background
Authors	Kumar S. Ray, Soma Chakraborty
Abstract	This paper proposes a novel approach to create an automated visual surveillance system which is very efficient in detecting and tracking moving objects in a video captured by moving camera without any apriori information about the captured scene. Separating foreground from the background is challenging job in videos captured by moving camera as both foreground and background information change in every consecutive frames of the image sequence; thus a pseudo-motion is perceptive in background. In the proposed algorithm, the pseudo-motion in background is estimated and compensated using phase correlation of consecutive frames based on the principle of Fourier shift theorem. Then a method is proposed to model an acting background from recent history of commonality of the current frame and the foreground is detected by the differences between the background model and the current frame. Further exploiting the recent history of dissimilarities of the current frame, actual moving objects are detected in the foreground. Next, a two-stepped morphological operation is proposed to refine the object region for an optimum object size. Each object is attributed by its centroid, dimension and three highest peaks of its gray value histogram. Finally, each object is tracked using Kalman filter based on its attributes. The major advantage of this algorithm over most of the existing object detection and tracking algorithms is that, it does not require initialization of object position in the first frame or training on sample data to perform. Performance of the algorithm is tested on benchmark videos containing variable background and very satisfiable results is achieved. The performance of the algorithm is also comparable with some of the state-of-the-art algorithms for object detection and tracking.
Tasks	Object Detection
Published	2017-05-11
URL	http://arxiv.org/abs/1706.02672v1
PDF	http://arxiv.org/pdf/1706.02672v1.pdf
PWC	https://paperswithcode.com/paper/an-efficient-approach-for-object-detection
Repo
Framework

Gaussian Process Quadrature Moment Transform


Title	Gaussian Process Quadrature Moment Transform
Authors	Jakub Prüher, Ondřej Straka
Abstract	Computation of moments of transformed random variables is a problem appearing in many engineering applications. The current methods for moment transformation are mostly based on the classical quadrature rules which cannot account for the approximation errors. Our aim is to design a method for moment transformation for Gaussian random variables which accounts for the error in the numerically computed mean. We employ an instance of Bayesian quadrature, called Gaussian process quadrature (GPQ), which allows us to treat the integral itself as a random variable, where the integral variance informs about the incurred integration error. Experiments on the coordinate transformation and nonlinear filtering examples show that the proposed GPQ moment transform performs better than the classical transforms.
Tasks
Published	2017-01-05
URL	http://arxiv.org/abs/1701.01356v1
PDF	http://arxiv.org/pdf/1701.01356v1.pdf
PWC	https://paperswithcode.com/paper/gaussian-process-quadrature-moment-transform
Repo
Framework

DenseNet for Dense Flow


Title	DenseNet for Dense Flow
Authors	Yi Zhu, Shawn Newsam
Abstract	Classical approaches for estimating optical flow have achieved rapid progress in the last decade. However, most of them are too slow to be applied in real-time video analysis. Due to the great success of deep learning, recent work has focused on using CNNs to solve such dense prediction problems. In this paper, we investigate a new deep architecture, Densely Connected Convolutional Networks (DenseNet), to learn optical flow. This specific architecture is ideal for the problem at hand as it provides shortcut connections throughout the network, which leads to implicit deep supervision. We extend current DenseNet to a fully convolutional network to learn motion estimation in an unsupervised manner. Evaluation results on three standard benchmarks demonstrate that DenseNet is a better fit than other widely adopted CNN architectures for optical flow estimation.
Tasks	Motion Estimation, Optical Flow Estimation
Published	2017-07-19
URL	http://arxiv.org/abs/1707.06316v1
PDF	http://arxiv.org/pdf/1707.06316v1.pdf
PWC	https://paperswithcode.com/paper/densenet-for-dense-flow
Repo
Framework