January 30, 2020

3156 words 15 mins read

Paper Group ANR 215

Privacy-Preserving Classification with Secret Vector Machines. Learning Cascaded Siamese Networks for High Performance Visual Tracking. Using Deep Learning to Count Albatrosses from Space. Machine Learning for high speed channel optimization. Photofeeler-D3: A Neural Network with Voter Modeling for Dating Photo Impression Prediction. 2-Wasserstein …

Privacy-Preserving Classification with Secret Vector Machines


Title	Privacy-Preserving Classification with Secret Vector Machines
Authors	Valentin Hartmann, Konark Modi, Josep M. Pujol, Robert West
Abstract	Today, large amounts of valuable data are distributed among millions of user-held devices, such as personal computers, phones, or Internet-of-things devices. Many companies collect such data with the goal of using it for training machine learning models allowing them to improve their services. However, user-held data is often sensitive, and collecting it is problematic in terms of privacy. We address this issue by proposing a novel way of training a supervised classifier in a distributed setting akin to the recently proposed federated learning paradigm (McMahan et al. 2017), but under the stricter privacy requirement that the server that trains the model is assumed to be untrusted and potentially malicious; we thus preserve user privacy by design, rather than by trust. In particular, our framework, called secret vector machine (SecVM), provides an algorithm for training linear support vector machines (SVM) in a setting in which data-holding clients communicate with an untrusted server by exchanging messages designed to not reveal any personally identifiable information. We evaluate our model in two ways. First, in an offline evaluation, we train SecVM to predict user gender from tweets, showing that we can preserve user privacy without sacrificing classification performance. Second, we implement SecVM’s distributed framework for the Cliqz web browser and deploy it for predicting user gender in a large-scale online evaluation with thousands of clients, outperforming baselines by a large margin and thus showcasing that SecVM is practicable in production environments. Overall, this work demonstrates the feasibility of machine learning on data from thousands of users without collecting any personal data. We believe this is an innovative approach that will help reconcile machine learning with data privacy.
Tasks
Published	2019-07-08
URL	https://arxiv.org/abs/1907.03373v1
PDF	https://arxiv.org/pdf/1907.03373v1.pdf
PWC	https://paperswithcode.com/paper/privacy-preserving-classification-with-secret
Repo
Framework

Learning Cascaded Siamese Networks for High Performance Visual Tracking


Title	Learning Cascaded Siamese Networks for High Performance Visual Tracking
Authors	Peng Gao, Yipeng Ma, Ruyue Yuan, Liyi Xiao, Fei Wang
Abstract	Visual tracking is one of the most challenging computer vision problems. In order to achieve high performance visual tracking in various negative scenarios, a novel cascaded Siamese network is proposed and developed based on two different deep learning networks: a matching subnetwork and a classification subnetwork. The matching subnetwork is a fully convolutional Siamese network. According to the similarity score between the exemplar image and the candidate image, it aims to search possible object positions and crop scaled candidate patches. The classification subnetwork is designed to further evaluate the cropped candidate patches and determine the optimal tracking results based on the classification score. The matching subnetwork is trained offline and fixed online, while the classification subnetwork performs stochastic gradient descent online to learn more target-specific information. To improve the tracking performance further, an effective classification subnetwork update method based on both similarity and classification scores is utilized for updating the classification subnetwork. Extensive experimental results demonstrate that our proposed approach achieves state-of-the-art performance in recent benchmarks.
Tasks	Visual Tracking
Published	2019-05-08
URL	https://arxiv.org/abs/1905.02857v1
PDF	https://arxiv.org/pdf/1905.02857v1.pdf
PWC	https://paperswithcode.com/paper/190502857
Repo
Framework

Using Deep Learning to Count Albatrosses from Space


Title	Using Deep Learning to Count Albatrosses from Space
Authors	Ellen Bowler, Peter T. Fretwell, Geoffrey French, Michal Mackiewicz
Abstract	In this paper we test the use of a deep learning approach to automatically count Wandering Albatrosses in Very High Resolution (VHR) satellite imagery. We use a dataset of manually labelled imagery provided by the British Antarctic Survey to train and develop our methods. We employ a U-Net architecture, designed for image segmentation, to simultaneously classify and localise potential albatrosses. We aid training with the use of the Focal Loss criterion, to deal with extreme class imbalance in the dataset. Initial results achieve peak precision and recall values of approximately 80%. Finally we assess the model’s performance in relation to inter-observer variation, by comparing errors against an image labelled by multiple observers. We conclude model accuracy falls within the range of human counters. We hope that the methods will streamline the analysis of VHR satellite images, enabling more frequent monitoring of a species which is of high conservation concern.
Tasks	Semantic Segmentation
Published	2019-07-03
URL	https://arxiv.org/abs/1907.02040v1
PDF	https://arxiv.org/pdf/1907.02040v1.pdf
PWC	https://paperswithcode.com/paper/using-deep-learning-to-count-albatrosses-from
Repo
Framework

Machine Learning for high speed channel optimization


Title	Machine Learning for high speed channel optimization
Authors	Jiayi He, Aravind Sampath Kumar, Arun Chada, Bhyrav Mutnury, James Drewniak
Abstract	Design of printed circuit board (PCB) stack-up requires the consideration of characteristic impedance, insertion loss and crosstalk. As there are many parameters in a PCB stack-up design, the optimization of these parameters needs to be efficient and accurate. A less optimal stack-up would lead to expensive PCB material choices in high speed designs. In this paper, an efficient global optimization method using parallel and intelligent Bayesian optimization is proposed for the stripline design.
Tasks
Published	2019-11-01
URL	https://arxiv.org/abs/1911.04317v1
PDF	https://arxiv.org/pdf/1911.04317v1.pdf
PWC	https://paperswithcode.com/paper/machine-learning-for-high-speed-channel
Repo
Framework

Photofeeler-D3: A Neural Network with Voter Modeling for Dating Photo Impression Prediction


Title	Photofeeler-D3: A Neural Network with Voter Modeling for Dating Photo Impression Prediction
Authors	Agastya Kalra, Ben Peterson
Abstract	In just a few years, online dating has become the dominant way that young people meet to date, making the deceptively error-prone task of picking good dating profile photos vital to a generation’s ability to form romantic connections. Until now, artificial intelligence approaches to Dating Photo Impression Prediction (DPIP) have been very inaccurate, unadaptable to real-world application, and have only taken into account a subject’s physical attractiveness. To that effect, we propose Photofeeler-D3 - the first convolutional neural network as accurate as 10 human votes for how smart, trustworthy, and attractive the subject appears in highly variable dating photos. Our “attractive” output is also applicable to Facial Beauty Prediction (FBP), making Photofeeler-D3 state-of-the-art for both DPIP and FBP. We achieve this by leveraging Photofeeler’s Dating Dataset (PDD) with over 1 million images and tens of millions of votes, our novel technique of voter modeling, and cutting-edge computer vision techniques.
Tasks	Facial Beauty Prediction
Published	2019-04-16
URL	https://arxiv.org/abs/1904.07435v3
PDF	https://arxiv.org/pdf/1904.07435v3.pdf
PWC	https://paperswithcode.com/paper/photofeeler-d3-a-neural-network-with-voter
Repo
Framework

2-Wasserstein Approximation via Restricted Convex Potentials with Application to Improved Training for GANs


Title	2-Wasserstein Approximation via Restricted Convex Potentials with Application to Improved Training for GANs
Authors	Amirhossein Taghvaei, Amin Jalali
Abstract	We provide a framework to approximate the 2-Wasserstein distance and the optimal transport map, amenable to efficient training as well as statistical and geometric analysis. With the quadratic cost and considering the Kantorovich dual form of the optimal transportation problem, the Brenier theorem states that the optimal potential function is convex and the optimal transport map is the gradient of the optimal potential function. Using this geometric structure, we restrict the optimization problem to different parametrized classes of convex functions and pay special attention to the class of input-convex neural networks. We analyze the statistical generalization and the discriminative power of the resulting approximate metric, and we prove a restricted moment-matching property for the approximate optimal map. Finally, we discuss a numerical algorithm to solve the restricted optimization problem and provide numerical experiments to illustrate and compare the proposed approach with the established regularization-based approaches. We further discuss practical implications of our proposal in a modular and interpretable design for GANs which connects the generator training with discriminator computations to allow for learning an overall composite generator.
Tasks
Published	2019-02-19
URL	http://arxiv.org/abs/1902.07197v1
PDF	http://arxiv.org/pdf/1902.07197v1.pdf
PWC	https://paperswithcode.com/paper/2-wasserstein-approximation-via-restricted
Repo
Framework

Asymptotically exact data augmentation: models, properties and algorithms


Title	Asymptotically exact data augmentation: models, properties and algorithms
Authors	Maxime Vono, Nicolas Dobigeon, Pierre Chainais
Abstract	Data augmentation, by the introduction of auxiliary variables, has become an ubiquitous technique to improve convergence properties, simplify the implementation or reduce the computational time of inference methods such as Markov chain Monte Carlo ones. Nonetheless, introducing appropriate auxiliary variables while preserving the initial target probability distribution and offering a computationally efficient inference cannot be conducted in a systematic way. To deal with such issues, this paper studies a unified framework, coined asymptotically exact data augmentation (AXDA), which encompasses both well-established and more recent approximate augmented models. In a broader perspective, this paper shows that AXDA models can benefit from interesting statistical properties and yield efficient inference algorithms. The pillar of such models is a Dirac delta-converging sequence which ensures the recovery of the initial target density in a limiting case. In non-asymptotic settings, the quality of the proposed approximation is assessed with several theoretical results. The latter are illustrated on standard statistical problems. Supplementary materials including computer code for this paper are available online.
Tasks	Data Augmentation
Published	2019-02-15
URL	https://arxiv.org/abs/1902.05754v2
PDF	https://arxiv.org/pdf/1902.05754v2.pdf
PWC	https://paperswithcode.com/paper/asymptotically-exact-data-augmentation-models
Repo
Framework

Transfer Value Iteration Networks


Title	Transfer Value Iteration Networks
Authors	Junyi Shen, Hankz Hankui Zhuo, Jin Xu, Bin Zhong, Sinno Jialin Pan
Abstract	Value iteration networks (VINs) have been demonstrated to have a good generalization ability for reinforcement learning tasks across similar domains. However, based on our experiments, a policy learned by VINs still fail to generalize well on the domain whose action space and feature space are not identical to those in the domain where it is trained. In this paper, we propose a transfer learning approach on top of VINs, termed Transfer VINs (TVINs), such that a learned policy from a source domain can be generalized to a target domain with only limited training data, even if the source domain and the target domain have domain-specific actions and features. We empirically verify that our proposed TVINs outperform VINs when the source and the target domains have similar but not identical action and feature spaces. Furthermore, we show that the performance improvement is consistent across different environments, maze sizes, dataset sizes as well as different values of hyperparameters such as number of iteration and kernel size.
Tasks	Transfer Learning
Published	2019-11-11
URL	https://arxiv.org/abs/1911.05701v2
PDF	https://arxiv.org/pdf/1911.05701v2.pdf
PWC	https://paperswithcode.com/paper/transfer-value-iteration-networks
Repo
Framework

Scalable Variational Bayesian Kernel Selection for Sparse Gaussian Process Regression


Title	Scalable Variational Bayesian Kernel Selection for Sparse Gaussian Process Regression
Authors	Tong Teng, Jie Chen, Yehong Zhang, Kian Hsiang Low
Abstract	This paper presents a variational Bayesian kernel selection (VBKS) algorithm for sparse Gaussian process regression (SGPR) models. In contrast to existing GP kernel selection algorithms that aim to select only one kernel with the highest model evidence, our proposed VBKS algorithm considers the kernel as a random variable and learns its belief from data such that the uncertainty of the kernel can be interpreted and exploited to avoid overconfident GP predictions. To achieve this, we represent the probabilistic kernel as an additional variational variable in a variational inference (VI) framework for SGPR models where its posterior belief is learned together with that of the other variational variables (i.e., inducing variables and kernel hyperparameters). In particular, we transform the discrete kernel belief into a continuous parametric distribution via reparameterization in order to apply VI. Though it is computationally challenging to jointly optimize a large number of hyperparameters due to many kernels being evaluated simultaneously by our VBKS algorithm, we show that the variational lower bound of the log-marginal likelihood can be decomposed into an additive form such that each additive term depends only on a disjoint subset of the variational variables and can thus be optimized independently. Stochastic optimization is then used to maximize the variational lower bound by iteratively improving the variational approximation of the exact posterior belief via stochastic gradient ascent, which incurs constant time per iteration and hence scales to big data. We empirically evaluate the performance of our VBKS algorithm on synthetic and massive real-world datasets.
Tasks	Stochastic Optimization
Published	2019-12-05
URL	https://arxiv.org/abs/1912.02641v1
PDF	https://arxiv.org/pdf/1912.02641v1.pdf
PWC	https://paperswithcode.com/paper/scalable-variational-bayesian-kernel
Repo
Framework


Title	Methodological Blind Spots in Machine Learning Fairness: Lessons from the Philosophy of Science and Computer Science
Authors	Samuel Deng, Achille Varzi
Abstract	In the ML fairness literature, there have been few investigations through the viewpoint of philosophy, a lens that encourages the critical evaluation of basic assumptions. The purpose of this paper is to use three ideas from the philosophy of science and computer science to tease out blind spots in the assumptions that underlie ML fairness: abstraction, induction, and measurement. Through this investigation, we hope to warn of these methodological blind spots and encourage further interdisciplinary investigation in fair-ML through the framework of philosophy.
Tasks
Published	2019-10-31
URL	https://arxiv.org/abs/1910.14210v1
PDF	https://arxiv.org/pdf/1910.14210v1.pdf
PWC	https://paperswithcode.com/paper/methodological-blind-spots-in-machine
Repo
Framework

The merits of Universal Language Model Fine-tuning for Small Datasets – a case with Dutch book reviews


Title	The merits of Universal Language Model Fine-tuning for Small Datasets – a case with Dutch book reviews
Authors	Benjamin van der Burgh, Suzan Verberne
Abstract	We evaluated the effectiveness of using language models, that were pre-trained in one domain, as the basis for a classification model in another domain: Dutch book reviews. Pre-trained language models have opened up new possibilities for classification tasks with limited labelled data, because representation can be learned in an unsupervised fashion. In our experiments we have studied the effects of training set size (100-1600 items) on the prediction accuracy of a ULMFiT classifier, based on a language models that we pre-trained on the Dutch Wikipedia. We also compared ULMFiT to Support Vector Machines, which is traditionally considered suitable for small collections. We found that ULMFiT outperforms SVM for all training set sizes and that satisfactory results (~90%) can be achieved using training sets that can be manually annotated within a few hours. We deliver both our new benchmark collection of Dutch book reviews for sentiment classification as well as the pre-trained Dutch language model to the community.
Tasks	Language Modelling, Sentiment Analysis
Published	2019-10-02
URL	https://arxiv.org/abs/1910.00896v1
PDF	https://arxiv.org/pdf/1910.00896v1.pdf
PWC	https://paperswithcode.com/paper/the-merits-of-universal-language-model-fine
Repo
Framework

Practical Algorithms for Multi-Stage Voting Rules with Parallel Universes Tiebreaking


Title	Practical Algorithms for Multi-Stage Voting Rules with Parallel Universes Tiebreaking
Authors	Jun Wang, Sujoy Sikdar, Tyler Shepherd, Zhibing Zhao, Chunheng Jiang, Lirong Xia
Abstract	STV and ranked pairs (RP) are two well-studied voting rules for group decision-making. They proceed in multiple rounds, and are affected by how ties are broken in each round. However, the literature is surprisingly vague about how ties should be broken. We propose the first algorithms for computing the set of alternatives that are winners under some tiebreaking mechanism under STV and RP, which is also known as parallel-universes tiebreaking (PUT). Unfortunately, PUT-winners are NP-complete to compute under STV and RP, and standard search algorithms from AI do not apply. We propose multiple DFS-based algorithms along with pruning strategies, heuristics, sampling and machine learning to prioritize search direction to significantly improve the performance. We also propose novel ILP formulations for PUT-winners under STV and RP, respectively. Experiments on synthetic and real-world data show that our algorithms are overall faster than ILP.
Tasks	Decision Making
Published	2019-01-16
URL	http://arxiv.org/abs/1901.09791v1
PDF	http://arxiv.org/pdf/1901.09791v1.pdf
PWC	https://paperswithcode.com/paper/practical-algorithms-for-multi-stage-voting
Repo
Framework

A Combined Deep Learning-Gradient Boosting Machine Framework for Fluid Intelligence Prediction


Title	A Combined Deep Learning-Gradient Boosting Machine Framework for Fluid Intelligence Prediction
Authors	Yeeleng S. Vang, Yingxin Cao, Xiaohui Xie
Abstract	The ABCD Neurocognitive Prediction Challenge is a community driven competition asking competitors to develop algorithms to predict fluid intelligence score from T1-w MRIs. In this work, we propose a deep learning combined with gradient boosting machine framework to solve this task. We train a convolutional neural network to compress the high dimensional MRI data and learn meaningful image features by predicting the 123 continuous-valued derived data provided with each MRI. These extracted features are then used to train a gradient boosting machine that predicts the residualized fluid intelligence score. Our approach achieved mean square error (MSE) scores of 18.4374, 68.7868, and 96.1806 for the training, validation, and test set respectively.
Tasks
Published	2019-10-16
URL	https://arxiv.org/abs/1910.07640v1
PDF	https://arxiv.org/pdf/1910.07640v1.pdf
PWC	https://paperswithcode.com/paper/a-combined-deep-learning-gradient-boosting
Repo
Framework

Learning Deep Representations by Mutual Information for Person Re-identification


Title	Learning Deep Representations by Mutual Information for Person Re-identification
Authors	Peng Chen, Tong Jia, Pengfei Wu, Jianjun Wu, Dongyue Chen
Abstract	Most existing person re-identification (ReID) methods have good feature representations to distinguish pedestrians with deep convolutional neural network (CNN) and metric learning methods. However, these works concentrate on the similarity between encoder output and ground-truth, ignoring the correlation between input and encoder output, which affects the performance of identifying different pedestrians. To address this limitation, We design a Deep InfoMax (DIM) network to maximize the mutual information (MI) between the input image and encoder output, which doesn’t need any auxiliary labels. To evaluate the effectiveness of the DIM network, we propose end-to-end Global-DIM and Local-DIM models. Additionally, the DIM network provides a new solution for cross-dataset unsupervised ReID issue as it needs no extra labels. The experiments prove the superiority of MI theory on the ReID issue, which achieves the state-of-the-art results.
Tasks	Metric Learning, Person Re-Identification
Published	2019-08-16
URL	https://arxiv.org/abs/1908.05860v1
PDF	https://arxiv.org/pdf/1908.05860v1.pdf
PWC	https://paperswithcode.com/paper/learning-deep-representations-by-mutual-2
Repo
Framework

A Sparse Bayesian Deep Learning Approach for Identification of Cascaded Tanks Benchmark


Title	A Sparse Bayesian Deep Learning Approach for Identification of Cascaded Tanks Benchmark
Authors	Hongpeng Zhou, Chahine Ibrahim, Wei Pan
Abstract	Nonlinear system identification is important with a wide range of applications. The typical approaches for nonlinear system identification include Volterra series models, nonlinear autoregressive with exogenous inputs models, block-structured models, state-space models and neural network models. Among them, neural networks (NN) is an important black-box method thanks to its universal approximation capability and less dependency on prior information. However, there are several challenges associated with NN. The first one lies in the design of a proper neural network structure. A relatively simple network cannot approximate the feature of the system, while a complex model may lead to overfitting. The second lies in the availability of data for some nonlinear systems. For some systems, it is difficult to collect enough data to train a neural network. This raises the challenge that how to train a neural network for system identification with a small dataset. In addition, if the uncertainty of the NN parameter could be obtained, it would be also beneficial for further analysis. In this paper, we propose a sparse Bayesian deep learning approach to address the above problems. Specifically, the Bayesian method can reinforce the regularization on neural networks by introducing introduced sparsity-inducing priors. The Bayesian method can also compute the uncertainty of the NN parameter. An efficient iterative re-weighted algorithm is presented in this paper. We also test the capacity of our method to identify the system on various ratios of the original dataset. The one-step-ahead prediction experiment on Cascaded Tank System shows the effectiveness of our method. Furthermore, we test our algorithm with more challenging simulation experiment on this benchmark, which also outperforms other methods.
Tasks
Published	2019-11-15
URL	https://arxiv.org/abs/1911.06847v2
PDF	https://arxiv.org/pdf/1911.06847v2.pdf
PWC	https://paperswithcode.com/paper/a-sparse-bayesian-deep-learning-approach-for
Repo
Framework