May 6, 2019

2958 words 14 mins read

Paper Group ANR 277

Screen Content Image Segmentation Using Sparse Decomposition and Total Variation Minimization. Secure Approximation Guarantee for Cryptographically Private Empirical Risk Minimization. A Benchmark and Comparison of Active Learning for Logistic Regression. Convex Formulation for Kernel PCA and its Use in Semi-Supervised Learning. Harmonic Networks: …

Screen Content Image Segmentation Using Sparse Decomposition and Total Variation Minimization


Title	Screen Content Image Segmentation Using Sparse Decomposition and Total Variation Minimization
Authors	Shervin Minaee, Yao Wang
Abstract	Sparse decomposition has been widely used for different applications, such as source separation, image classification, image denoising and more. This paper presents a new algorithm for segmentation of an image into background and foreground text and graphics using sparse decomposition and total variation minimization. The proposed method is designed based on the assumption that the background part of the image is smoothly varying and can be represented by a linear combination of a few smoothly varying basis functions, while the foreground text and graphics can be modeled with a sparse component overlaid on the smooth background. The background and foreground are separated using a sparse decomposition framework regularized with a few suitable regularization terms which promotes the sparsity and connectivity of foreground pixels. This algorithm has been tested on a dataset of images extracted from HEVC standard test sequences for screen content coding, and is shown to have superior performance over some prior methods, including least absolute deviation fitting, k-means clustering based segmentation in DjVu and shape primitive extraction and coding (SPEC) algorithm.
Tasks	Denoising, Image Classification, Image Denoising, Semantic Segmentation
Published	2016-02-07
URL	http://arxiv.org/abs/1602.02434v2
PDF	http://arxiv.org/pdf/1602.02434v2.pdf
PWC	https://paperswithcode.com/paper/screen-content-image-segmentation-using-1
Repo
Framework

Secure Approximation Guarantee for Cryptographically Private Empirical Risk Minimization


Title	Secure Approximation Guarantee for Cryptographically Private Empirical Risk Minimization
Authors	Toshiyuki Takada, Hiroyuki Hanada, Yoshiji Yamada, Jun Sakuma, Ichiro Takeuchi
Abstract	Privacy concern has been increasingly important in many machine learning (ML) problems. We study empirical risk minimization (ERM) problems under secure multi-party computation (MPC) frameworks. Main technical tools for MPC have been developed based on cryptography. One of limitations in current cryptographically private ML is that it is computationally intractable to evaluate non-linear functions such as logarithmic functions or exponential functions. Therefore, for a class of ERM problems such as logistic regression in which non-linear function evaluations are required, one can only obtain approximate solutions. In this paper, we introduce a novel cryptographically private tool called secure approximation guarantee (SAG) method. The key property of SAG method is that, given an arbitrary approximate solution, it can provide a non-probabilistic assumption-free bound on the approximation quality under cryptographically secure computation framework. We demonstrate the benefit of the SAG method by applying it to several problems including a practical privacy-preserving data analysis task on genomic and clinical information.
Tasks
Published	2016-02-15
URL	http://arxiv.org/abs/1602.04579v1
PDF	http://arxiv.org/pdf/1602.04579v1.pdf
PWC	https://paperswithcode.com/paper/secure-approximation-guarantee-for
Repo
Framework

A Benchmark and Comparison of Active Learning for Logistic Regression


Title	A Benchmark and Comparison of Active Learning for Logistic Regression
Authors	Yazhou Yang, Marco Loog
Abstract	Logistic regression is by far the most widely used classifier in real-world applications. In this paper, we benchmark the state-of-the-art active learning methods for logistic regression and discuss and illustrate their underlying characteristics. Experiments are carried out on three synthetic datasets and 44 real-world datasets, providing insight into the behaviors of these active learning methods with respect to the area of the learning curve (which plots classification accuracy as a function of the number of queried examples) and their computational costs. Surprisingly, one of the earliest and simplest suggested active learning methods, i.e., uncertainty sampling, performs exceptionally well overall. Another remarkable finding is that random sampling, which is the rudimentary baseline to improve upon, is not overwhelmed by individual active learning techniques in many cases.
Tasks	Active Learning
Published	2016-11-25
URL	http://arxiv.org/abs/1611.08618v2
PDF	http://arxiv.org/pdf/1611.08618v2.pdf
PWC	https://paperswithcode.com/paper/a-benchmark-and-comparison-of-active-learning
Repo
Framework

Convex Formulation for Kernel PCA and its Use in Semi-Supervised Learning


Title	Convex Formulation for Kernel PCA and its Use in Semi-Supervised Learning
Authors	Carlos M. Alaíz, Michaël Fanuel, Johan A. K. Suykens
Abstract	In this paper, Kernel PCA is reinterpreted as the solution to a convex optimization problem. Actually, there is a constrained convex problem for each principal component, so that the constraints guarantee that the principal component is indeed a solution, and not a mere saddle point. Although these insights do not imply any algorithmic improvement, they can be used to further understand the method, formulate possible extensions and properly address them. As an example, a new convex optimization problem for semi-supervised classification is proposed, which seems particularly well-suited whenever the number of known labels is small. Our formulation resembles a Least Squares SVM problem with a regularization parameter multiplied by a negative sign, combined with a variational principle for Kernel PCA. Our primal optimization principle for semi-supervised learning is solved in terms of the Lagrange multipliers. Numerical experiments in several classification tasks illustrate the performance of the proposed model in problems with only a few labeled data.
Tasks
Published	2016-10-21
URL	http://arxiv.org/abs/1610.06811v1
PDF	http://arxiv.org/pdf/1610.06811v1.pdf
PWC	https://paperswithcode.com/paper/convex-formulation-for-kernel-pca-and-its-use
Repo
Framework

Harmonic Networks: Deep Translation and Rotation Equivariance


Title	Harmonic Networks: Deep Translation and Rotation Equivariance
Authors	Daniel E. Worrall, Stephan J. Garbin, Daniyar Turmukhambetov, Gabriel J. Brostow
Abstract	Translating or rotating an input image should not affect the results of many computer vision tasks. Convolutional neural networks (CNNs) are already translation equivariant: input image translations produce proportionate feature map translations. This is not the case for rotations. Global rotation equivariance is typically sought through data augmentation, but patch-wise equivariance is more difficult. We present Harmonic Networks or H-Nets, a CNN exhibiting equivariance to patch-wise translation and 360-rotation. We achieve this by replacing regular CNN filters with circular harmonics, returning a maximal response and orientation for every receptive field patch. H-Nets use a rich, parameter-efficient and low computational complexity representation, and we show that deep feature maps within the network encode complicated rotational invariants. We demonstrate that our layers are general enough to be used in conjunction with the latest architectures and techniques, such as deep supervision and batch normalization. We also achieve state-of-the-art classification on rotated-MNIST, and competitive results on other benchmark challenges.
Tasks	Data Augmentation
Published	2016-12-14
URL	http://arxiv.org/abs/1612.04642v2
PDF	http://arxiv.org/pdf/1612.04642v2.pdf
PWC	https://paperswithcode.com/paper/harmonic-networks-deep-translation-and
Repo
Framework

Inverting The Generator Of A Generative Adversarial Network


Title	Inverting The Generator Of A Generative Adversarial Network
Authors	Antonia Creswell, Anil Anthony Bharath
Abstract	Generative adversarial networks (GANs) learn to synthesise new samples from a high-dimensional distribution by passing samples drawn from a latent space through a generative network. When the high-dimensional distribution describes images of a particular data set, the network should learn to generate visually similar image samples for latent variables that are close to each other in the latent space. For tasks such as image retrieval and image classification, it may be useful to exploit the arrangement of the latent space by projecting images into it, and using this as a representation for discriminative tasks. GANs often consist of multiple layers of non-linear computations, making them very difficult to invert. This paper introduces techniques for projecting image samples into the latent space using any pre-trained GAN, provided that the computational graph is available. We evaluate these techniques on both MNIST digits and Omniglot handwritten characters. In the case of MNIST digits, we show that projections into the latent space maintain information about the style and the identity of the digit. In the case of Omniglot characters, we show that even characters from alphabets that have not been seen during training may be projected well into the latent space; this suggests that this approach may have applications in one-shot learning.
Tasks	Image Classification, Image Retrieval, Omniglot, One-Shot Learning
Published	2016-11-17
URL	http://arxiv.org/abs/1611.05644v1
PDF	http://arxiv.org/pdf/1611.05644v1.pdf
PWC	https://paperswithcode.com/paper/inverting-the-generator-of-a-generative
Repo
Framework

Masking Strategies for Image Manifolds


Title	Masking Strategies for Image Manifolds
Authors	Hamid Dadkhahi, Marco F. Duarte
Abstract	We consider the problem of selecting an optimal mask for an image manifold, i.e., choosing a subset of the pixels of the image that preserves the manifold’s geometric structure present in the original data. Such masking implements a form of compressive sensing through emerging imaging sensor platforms for which the power expense grows with the number of pixels acquired. Our goal is for the manifold learned from masked images to resemble its full image counterpart as closely as possible. More precisely, we show that one can indeed accurately learn an image manifold without having to consider a large majority of the image pixels. In doing so, we consider two masking methods that preserve the local and global geometric structure of the manifold, respectively. In each case, the process of finding the optimal masking pattern can be cast as a binary integer program, which is computationally expensive but can be approximated by a fast greedy algorithm. Numerical experiments show that the relevant manifold structure is preserved through the data-dependent masking process, even for modest mask sizes.
Tasks	Compressive Sensing
Published	2016-06-15
URL	http://arxiv.org/abs/1606.04618v1
PDF	http://arxiv.org/pdf/1606.04618v1.pdf
PWC	https://paperswithcode.com/paper/masking-strategies-for-image-manifolds
Repo
Framework

Tensor Completion by Alternating Minimization under the Tensor Train (TT) Model


Title	Tensor Completion by Alternating Minimization under the Tensor Train (TT) Model
Authors	Wenqi Wang, Vaneet Aggarwal, Shuchin Aeron
Abstract	Using the matrix product state (MPS) representation of tensor train decompositions, in this paper we propose a tensor completion algorithm which alternates over the matrices (tensors) in the MPS representation. This development is motivated in part by the success of matrix completion algorithms which alternate over the (low-rank) factors. We comment on the computational complexity of the proposed algorithm and numerically compare it with existing methods employing low rank tensor train approximation for data completion as well as several other recently proposed methods. We show that our method is superior to existing ones for a variety of real settings.
Tasks	Matrix Completion
Published	2016-09-19
URL	http://arxiv.org/abs/1609.05587v1
PDF	http://arxiv.org/pdf/1609.05587v1.pdf
PWC	https://paperswithcode.com/paper/tensor-completion-by-alternating-minimization
Repo
Framework

Evaluating Causal Models by Comparing Interventional Distributions


Title	Evaluating Causal Models by Comparing Interventional Distributions
Authors	Dan Garant, David Jensen
Abstract	The predominant method for evaluating the quality of causal models is to measure the graphical accuracy of the learned model structure. We present an alternative method for evaluating causal models that directly measures the accuracy of estimated interventional distributions. We contrast such distributional measures with structural measures, such as structural Hamming distance and structural intervention distance, showing that structural measures often correspond poorly to the accuracy of estimated interventional distributions. We use a number of real and synthetic datasets to illustrate various scenarios in which structural measures provide misleading results with respect to algorithm selection and parameter tuning, and we recommend that distributional measures become the new standard for evaluating causal models.
Tasks
Published	2016-08-16
URL	http://arxiv.org/abs/1608.04698v1
PDF	http://arxiv.org/pdf/1608.04698v1.pdf
PWC	https://paperswithcode.com/paper/evaluating-causal-models-by-comparing
Repo
Framework

Learning Attributes Equals Multi-Source Domain Generalization


Title	Learning Attributes Equals Multi-Source Domain Generalization
Authors	Chuang Gan, Tianbao Yang, Boqing Gong
Abstract	Attributes possess appealing properties and benefit many computer vision problems, such as object recognition, learning with humans in the loop, and image retrieval. Whereas the existing work mainly pursues utilizing attributes for various computer vision problems, we contend that the most basic problem—how to accurately and robustly detect attributes from images—has been left under explored. Especially, the existing work rarely explicitly tackles the need that attribute detectors should generalize well across different categories, including those previously unseen. Noting that this is analogous to the objective of multi-source domain generalization, if we treat each category as a domain, we provide a novel perspective to attribute detection and propose to gear the techniques in multi-source domain generalization for the purpose of learning cross-category generalizable attribute detectors. We validate our understanding and approach with extensive experiments on four challenging datasets and three different problems.
Tasks	Domain Generalization, Image Retrieval, Object Recognition
Published	2016-05-03
URL	http://arxiv.org/abs/1605.00743v1
PDF	http://arxiv.org/pdf/1605.00743v1.pdf
PWC	https://paperswithcode.com/paper/learning-attributes-equals-multi-source
Repo
Framework

Theoretically-Grounded Policy Advice from Multiple Teachers in Reinforcement Learning Settings with Applications to Negative Transfer


Title	Theoretically-Grounded Policy Advice from Multiple Teachers in Reinforcement Learning Settings with Applications to Negative Transfer
Authors	Yusen Zhan, Haitham Bou Ammar, Matthew E. taylor
Abstract	Policy advice is a transfer learning method where a student agent is able to learn faster via advice from a teacher. However, both this and other reinforcement learning transfer methods have little theoretical analysis. This paper formally defines a setting where multiple teacher agents can provide advice to a student and introduces an algorithm to leverage both autonomous exploration and teacher’s advice. Our regret bounds justify the intuition that good teachers help while bad teachers hurt. Using our formalization, we are also able to quantify, for the first time, when negative transfer can occur within such a reinforcement learning setting.
Tasks	Transfer Learning
Published	2016-04-13
URL	http://arxiv.org/abs/1604.03986v1
PDF	http://arxiv.org/pdf/1604.03986v1.pdf
PWC	https://paperswithcode.com/paper/theoretically-grounded-policy-advice-from
Repo
Framework

Leveraging Mid-Level Deep Representations For Predicting Face Attributes in the Wild


Title	Leveraging Mid-Level Deep Representations For Predicting Face Attributes in the Wild
Authors	Yang Zhong, Josephine Sullivan, Haibo Li
Abstract	Predicting facial attributes from faces in the wild is very challenging due to pose and lighting variations in the real world. The key to this problem is to build proper feature representations to cope with these unfavourable conditions. Given the success of Convolutional Neural Network (CNN) in image classification, the high-level CNN feature, as an intuitive and reasonable choice, has been widely utilized for this problem. In this paper, however, we consider the mid-level CNN features as an alternative to the high-level ones for attribute prediction. This is based on the observation that face attributes are different: some of them are locally oriented while others are globally defined. Our investigations reveal that the mid-level deep representations outperform the prediction accuracy achieved by the (fine-tuned) high-level abstractions. We empirically demonstrate that the midlevel representations achieve state-of-the-art prediction performance on CelebA and LFWA datasets. Our investigations also show that by utilizing the mid-level representations one can employ a single deep network to achieve both face recognition and attribute prediction.
Tasks	Face Recognition, Image Classification
Published	2016-02-04
URL	http://arxiv.org/abs/1602.01827v3
PDF	http://arxiv.org/pdf/1602.01827v3.pdf
PWC	https://paperswithcode.com/paper/leveraging-mid-level-deep-representations-for
Repo
Framework

Shape-aware Surface Reconstruction from Sparse 3D Point-Clouds


Title	Shape-aware Surface Reconstruction from Sparse 3D Point-Clouds
Authors	Florian Bernard, Luis Salamanca, Johan Thunberg, Alexander Tack, Dennis Jentsch, Hans Lamecker, Stefan Zachow, Frank Hertel, Jorge Goncalves, Peter Gemmar
Abstract	The reconstruction of an object’s shape or surface from a set of 3D points plays an important role in medical image analysis, e.g. in anatomy reconstruction from tomographic measurements or in the process of aligning intra-operative navigation and preoperative planning data. In such scenarios, one usually has to deal with sparse data, which significantly aggravates the problem of reconstruction. However, medical applications often provide contextual information about the 3D point data that allow to incorporate prior knowledge about the shape that is to be reconstructed. To this end, we propose the use of a statistical shape model (SSM) as a prior for surface reconstruction. The SSM is represented by a point distribution model (PDM), which is associated with a surface mesh. Using the shape distribution that is modelled by the PDM, we formulate the problem of surface reconstruction from a probabilistic perspective based on a Gaussian Mixture Model (GMM). In order to do so, the given points are interpreted as samples of the GMM. By using mixture components with anisotropic covariances that are “oriented” according to the surface normals at the PDM points, a surface-based fitting is accomplished. Estimating the parameters of the GMM in a maximum a posteriori manner yields the reconstruction of the surface from the given data points. We compare our method to the extensively used Iterative Closest Points method on several different anatomical datasets/SSMs (brain, femur, tibia, hip, liver) and demonstrate superior accuracy and robustness on sparse data.
Tasks
Published	2016-02-26
URL	http://arxiv.org/abs/1602.08425v2
PDF	http://arxiv.org/pdf/1602.08425v2.pdf
PWC	https://paperswithcode.com/paper/shape-aware-surface-reconstruction-from
Repo
Framework

A Review on Algorithms for Constraint-based Causal Discovery


Title	A Review on Algorithms for Constraint-based Causal Discovery
Authors	Kui Yu, Jiuyong Li, Lin Liu
Abstract	Causal discovery studies the problem of mining causal relationships between variables from data, which is of primary interest in science. During the past decades, significant amount of progresses have been made toward this fundamental data mining paradigm. Recent years, as the availability of abundant large-sized and complex observational data, the constrain-based approaches have gradually attracted a lot of interest and have been widely applied to many diverse real-world problems due to the fast running speed and easy generalizing to the problem of causal insufficiency. In this paper, we aim to review the constraint-based causal discovery algorithms. Firstly, we discuss the learning paradigm of the constraint-based approaches. Secondly and primarily, the state-of-the-art constraint-based casual inference algorithms are surveyed with the detailed analysis. Thirdly, several related open-source software packages and benchmark data repositories are briefly summarized. As a conclusion, some open problems in constraint-based causal discovery are outlined for future research.
Tasks	Causal Discovery
Published	2016-11-12
URL	http://arxiv.org/abs/1611.03977v2
PDF	http://arxiv.org/pdf/1611.03977v2.pdf
PWC	https://paperswithcode.com/paper/a-review-on-algorithms-for-constraint-based
Repo
Framework

Improved Optimistic Mirror Descent for Sparsity and Curvature


Title	Improved Optimistic Mirror Descent for Sparsity and Curvature
Authors	Parameswaran Kamalaruban
Abstract	Online Convex Optimization plays a key role in large scale machine learning. Early approaches to this problem were conservative, in which the main focus was protection against the worst case scenario. But recently several algorithms have been developed for tightening the regret bounds in easy data instances such as sparsity, predictable sequences, and curved losses. In this work we unify some of these existing techniques to obtain new update rules for the cases when these easy instances occur together. First we analyse an adaptive and optimistic update rule which achieves tighter regret bound when the loss sequence is sparse and predictable. Then we explain an update rule that dynamically adapts to the curvature of the loss function and utilizes the predictable nature of the loss sequence as well. Finally we extend these results to composite losses.
Tasks
Published	2016-09-08
URL	http://arxiv.org/abs/1609.02383v1
PDF	http://arxiv.org/pdf/1609.02383v1.pdf
PWC	https://paperswithcode.com/paper/improved-optimistic-mirror-descent-for
Repo
Framework