January 27, 2020

2952 words 14 mins read

Paper Group ANR 1153

Triplet Based Embedding Distance and Similarity Learning for Text-independent Speaker Verification. BUT VOiCES 2019 System Description. Privacy-preserving Distributed Machine Learning via Local Randomization and ADMM Perturbation. White-Box Target Attack for EEG-Based BCI Regression Problems. Future Frame Prediction Using Convolutional VRNN for Ano …

Triplet Based Embedding Distance and Similarity Learning for Text-independent Speaker Verification


Title	Triplet Based Embedding Distance and Similarity Learning for Text-independent Speaker Verification
Authors	Zongze Ren, Zhiyong Chen, Shugong Xu
Abstract	Speaker embeddings become growing popular in the text-independent speaker verification task. In this paper, we propose two improvements during the training stage. The improvements are both based on triplet cause the training stage and the evaluation stage of the baseline x-vector system focus on different aims. Firstly, we introduce triplet loss for optimizing the Euclidean distances between embeddings while minimizing the multi-class cross entropy loss. Secondly, we design an embedding similarity measurement network for controlling the similarity between the two selected embeddings. We further jointly train the two new methods with the original network and achieve state-of-the-art. The multi-task training synergies are shown with a 9% reduction equal error rate (EER) and detected cost function (DCF) on the 2016 NIST Speaker Recognition Evaluation (SRE) Test Set.
Tasks	Speaker Recognition, Speaker Verification, Text-Independent Speaker Verification
Published	2019-08-06
URL	https://arxiv.org/abs/1908.02283v1
PDF	https://arxiv.org/pdf/1908.02283v1.pdf
PWC	https://paperswithcode.com/paper/triplet-based-embedding-distance-and
Repo
Framework

BUT VOiCES 2019 System Description


Title	BUT VOiCES 2019 System Description
Authors	Hossein Zeinali, Pavel Matějka, Ladislav Mošner, Oldřich Plchot, Anna Silnova, Ondřej Novotný, Ján Profant, Ondřej Glembek, Lukáš Burget
Abstract	This is a description of our effort in VOiCES 2019 Speaker Recognition challenge. All systems in the fixed condition are based on the x-vector paradigm with different features and DNN topologies. The single best system reaches 1.2% EER and a fusion of 3 systems yields 1.0% EER, which is 15% relative improvement. The open condition allowed us to use external data which we did for the PLDA adaptation and achieved less than ~10% relative improvement. In the submission to open condition, we used 3 x-vector systems and also one i-vector based system.
Tasks	Speaker Recognition
Published	2019-07-13
URL	https://arxiv.org/abs/1907.06112v1
PDF	https://arxiv.org/pdf/1907.06112v1.pdf
PWC	https://paperswithcode.com/paper/but-voices-2019-system-description
Repo
Framework

Privacy-preserving Distributed Machine Learning via Local Randomization and ADMM Perturbation


Title	Privacy-preserving Distributed Machine Learning via Local Randomization and ADMM Perturbation
Authors	Xin Wang, Hideaki Ishii, Linkang Du, Peng Cheng, Jiming Chen
Abstract	With the proliferation of training data, distributed machine learning (DML) is becoming more competent for large-scale learning tasks. However, privacy concerns have to be given priority in DML, since training data may contain sensitive information of users. In this paper, we propose a privacy-preserving ADMM-based DML framework with two novel features: First, we remove the assumption commonly made in the literature that the users trust the server collecting their data. Second, the framework provides heterogeneous privacy for users depending on data’s sensitive levels and servers’ trust degrees. The challenging issue is to keep the accumulation of privacy losses over ADMM iterations minimal. In the proposed framework, a local randomization approach, which is differentially private, is adopted to provide users with self-controlled privacy guarantee for the most sensitive information. Further, the ADMM algorithm is perturbed through a combined noise-adding method, which simultaneously preserves privacy for users’ less sensitive information and strengthens the privacy protection of the most sensitive information. We provide detailed analyses on the performance of the trained model according to its generalization error. Finally, we conduct extensive experiments using real-world datasets to validate the theoretical results and evaluate the classification performance of the proposed framework.
Tasks
Published	2019-07-30
URL	https://arxiv.org/abs/1908.01059v2
PDF	https://arxiv.org/pdf/1908.01059v2.pdf
PWC	https://paperswithcode.com/paper/privacy-preserving-distributed-machine
Repo
Framework

White-Box Target Attack for EEG-Based BCI Regression Problems


Title	White-Box Target Attack for EEG-Based BCI Regression Problems
Authors	Lubin Meng, Chin-Teng Lin, Tzyy-Ring Jung, Dongrui Wu
Abstract	Machine learning has achieved great success in many applications, including electroencephalogram (EEG) based brain-computer interfaces (BCIs). Unfortunately, many machine learning models are vulnerable to adversarial examples, which are crafted by adding deliberately designed perturbations to the original inputs. Many adversarial attack approaches for classification problems have been proposed, but few have considered target adversarial attacks for regression problems. This paper proposes two such approaches. More specifically, we consider white-box target attacks for regression problems, where we know all information about the regression model to be attacked, and want to design small perturbations to change the regression output by a pre-determined amount. Experiments on two BCI regression problems verified that both approaches are effective. Moreover, adversarial examples generated from both approaches are also transferable, which means that we can use adversarial examples generated from one known regression model to attack an unknown regression model, i.e., to perform black-box attacks. To our knowledge, this is the first study on adversarial attacks for EEG-based BCI regression problems, which calls for more attention on the security of BCI systems.
Tasks	Adversarial Attack, EEG
Published	2019-11-07
URL	https://arxiv.org/abs/1911.04606v1
PDF	https://arxiv.org/pdf/1911.04606v1.pdf
PWC	https://paperswithcode.com/paper/white-box-target-attack-for-eeg-based-bci
Repo
Framework

Future Frame Prediction Using Convolutional VRNN for Anomaly Detection


Title	Future Frame Prediction Using Convolutional VRNN for Anomaly Detection
Authors	Yiwei Lu, Mahesh Kumar Krishna Reddy, Seyed shahabeddin Nabavi, Yang Wang
Abstract	Anomaly detection in videos aims at reporting anything that does not conform the normal behaviour or distribution. However, due to the sparsity of abnormal video clips in real life, collecting annotated data for supervised learning is exceptionally cumbersome. Inspired by the practicability of generative models for semi-supervised learning, we propose a novel sequential generative model based on variational autoencoder (VAE) for future frame prediction with convolutional LSTM (ConvLSTM). To the best of our knowledge, this is the first work that considers temporal information in future frame prediction based anomaly detection framework from the model perspective. Our experiments demonstrate that our approach is superior to the state-of-the-art methods on three benchmark datasets.
Tasks	Anomaly Detection
Published	2019-09-05
URL	https://arxiv.org/abs/1909.02168v2
PDF	https://arxiv.org/pdf/1909.02168v2.pdf
PWC	https://paperswithcode.com/paper/future-frame-prediction-using-convolutional
Repo
Framework

Disentangling Human Dynamics for Pedestrian Locomotion Forecasting with Noisy Supervision


Title	Disentangling Human Dynamics for Pedestrian Locomotion Forecasting with Noisy Supervision
Authors	Karttikeya Mangalam, Ehsan Adeli, Kuan-Hui Lee, Adrien Gaidon, Juan Carlos Niebles
Abstract	We tackle the problem of Human Locomotion Forecasting, a task for jointly predicting the spatial positions of several keypoints on the human body in the near future under an egocentric setting. In contrast to the previous work that aims to solve either the task of pose prediction or trajectory forecasting in isolation, we propose a framework to unify the two problems and address the practically useful task of pedestrian locomotion prediction in the wild. Among the major challenges in solving this task is the scarcity of annotated egocentric video datasets with dense annotations for pose, depth, or egomotion. To surmount this difficulty, we use state-of-the-art models to generate (noisy) annotations and propose robust forecasting models that can learn from this noisy supervision. We present a method to disentangle the overall pedestrian motion into easier to learn subparts by utilizing a pose completion and a decomposition module. The completion module fills in the missing key-point annotations and the decomposition module breaks the cleaned locomotion down to global (trajectory) and local (pose keypoint movements). Further, with Quasi RNN as our backbone, we propose a novel hierarchical trajectory forecasting network that utilizes low-level vision domain specific signals like egomotion and depth to predict the global trajectory. Our method leads to state-of-the-art results for the prediction of human locomotion in the egocentric view.
Tasks	Human Dynamics, Pose Prediction
Published	2019-11-04
URL	https://arxiv.org/abs/1911.01138v1
PDF	https://arxiv.org/pdf/1911.01138v1.pdf
PWC	https://paperswithcode.com/paper/disentangling-human-dynamics-for-pedestrian
Repo
Framework

Human Pose Estimation using Motion Priors and Ensemble Models


Title	Human Pose Estimation using Motion Priors and Ensemble Models
Authors	Norimichi Ukita
Abstract	Human pose estimation in images and videos is one of key technologies for realizing a variety of human activity recognition tasks (e.g., human-computer interaction, gesture recognition, surveillance, and video summarization). This paper presents two types of human pose estimation methodologies; 1) 3D human pose tracking using motion priors and 2) 2D human pose estimation with ensemble modeling.
Tasks	Activity Recognition, Gesture Recognition, Human Activity Recognition, Pose Estimation, Pose Tracking, Video Summarization
Published	2019-01-26
URL	http://arxiv.org/abs/1901.09156v1
PDF	http://arxiv.org/pdf/1901.09156v1.pdf
PWC	https://paperswithcode.com/paper/human-pose-estimation-using-motion-priors-and
Repo
Framework

Lower Bounds for Smooth Nonconvex Finite-Sum Optimization


Title	Lower Bounds for Smooth Nonconvex Finite-Sum Optimization
Authors	Dongruo Zhou, Quanquan Gu
Abstract	Smooth finite-sum optimization has been widely studied in both convex and nonconvex settings. However, existing lower bounds for finite-sum optimization are mostly limited to the setting where each component function is (strongly) convex, while the lower bounds for nonconvex finite-sum optimization remain largely unsolved. In this paper, we study the lower bounds for smooth nonconvex finite-sum optimization, where the objective function is the average of $n$ nonconvex component functions. We prove tight lower bounds for the complexity of finding $\epsilon$-suboptimal point and $\epsilon$-approximate stationary point in different settings, for a wide regime of the smallest eigenvalue of the Hessian of the objective function (or each component function). Given our lower bounds, we can show that existing algorithms including KatyushaX (Allen-Zhu, 2018), Natasha (Allen-Zhu, 2017), RapGrad (Lan and Yang, 2018) and StagewiseKatyusha (Chen and Yang, 2018) have achieved optimal Incremental First-order Oracle (IFO) complexity (i.e., number of IFO calls) up to logarithm factors for nonconvex finite-sum optimization. We also point out potential ways to further improve these complexity results, in terms of making stronger assumptions or by a different convergence analysis.
Tasks
Published	2019-01-31
URL	http://arxiv.org/abs/1901.11224v1
PDF	http://arxiv.org/pdf/1901.11224v1.pdf
PWC	https://paperswithcode.com/paper/lower-bounds-for-smooth-nonconvex-finite-sum
Repo
Framework

Fréchet random forests


Title	Fréchet random forests
Authors	Louis Capitaine, Robin Genuer, Rodolphe Thiébaut
Abstract	Random forests are a statistical learning method widely used in many areas of scientific research essentially for its ability to learn complex relationship between input and output variables and also its capacity to handle high-dimensional data. However, data are increasingly complex with repeated measures of omics, images leading to shapes, curves… Random forests method is not specifically tailored for them. In this paper, we introduce Fr'echet trees and Fr'echet random forests, which allow to manage data for which input and output variables take values in general metric spaces (which can be unordered). To this end, a new way of splitting the nodes of trees is introduced and the prediction procedures of trees and forests are generalized. Then, random forests out-of-bag error and variable importance score are naturally adapted. Finally, the method is studied in the special case of regression on curve shapes, both within a simulation study and a real dataset from an HIV vaccine trial.
Tasks
Published	2019-06-04
URL	https://arxiv.org/abs/1906.01741v1
PDF	https://arxiv.org/pdf/1906.01741v1.pdf
PWC	https://paperswithcode.com/paper/frechet-random-forests
Repo
Framework

Adaptive Distraction Context Aware Tracking Based on Correlation Filter


Title	Adaptive Distraction Context Aware Tracking Based on Correlation Filter
Authors	Fei Feng, Xiao-Jun Wu, Tianyang Xu, Josef Kittler, Xue-Feng Zhu
Abstract	The Discriminative Correlation Filter (CF) uses a circulant convolution operation to provide several training samples for the design of a classifier that can distinguish the target from the background. The filter design may be interfered by objects close to the target during the tracking process, resulting in tracking failure. This paper proposes an adaptive distraction context aware tracking algorithm to solve this problem. In the response map obtained for the previous frame by the CF algorithm, we adaptively find the image blocks that are similar to the target and use them as negative samples. This diminishes the influence of similar image blocks on the classifier in the tracking process and its accuracy is improved. The tracking results on video sequences show that the algorithm can cope with rapid changes such as occlusion and rotation, and can adaptively use the distractive objects around the target as negative samples to improve the accuracy of target tracking.
Tasks
Published	2019-12-24
URL	https://arxiv.org/abs/1912.11325v1
PDF	https://arxiv.org/pdf/1912.11325v1.pdf
PWC	https://paperswithcode.com/paper/adaptive-distraction-context-aware-tracking
Repo
Framework

Decentralized Multi-Task Learning Based on Extreme Learning Machines


Title	Decentralized Multi-Task Learning Based on Extreme Learning Machines
Authors	Yu Ye, Ming Xiao, Mikael Skoglund
Abstract	In multi-task learning (MTL), related tasks learn jointly to improve generalization performance. To exploit the high learning speed of extreme learning machines (ELMs), we apply the ELM framework to the MTL problem, where the output weights of ELMs for all the tasks are learned collaboratively. We first present the ELM based MTL problem in the centralized setting, which is solved by the proposed MTL-ELM algorithm. Due to the fact that many data sets of different tasks are geo-distributed, decentralized machine learning is studied. We formulate the decentralized MTL problem based on ELM as majorized multi-block optimization with coupled bi-convex objective functions. To solve the problem, we propose the DMTL-ELM algorithm, which is a hybrid Jacobian and Gauss-Seidel Proximal multi-block alternating direction method of multipliers (ADMM). Further, to reduce the computation load of DMTL-ELM, DMTL-ELM with first-order approximation (FO-DMTL-ELM) is presented. Theoretical analysis shows that the convergence to the stationary point of DMTL-ELM and FO-DMTL-ELM can be guaranteed conditionally. Through simulations, we demonstrate the convergence of proposed MTL-ELM, DMTL-ELM, and FO-DMTL-ELM algorithms, and also show that they can outperform existing MTL methods. Moreover, by adjusting the dimension of hidden feature space, there exists a trade-off between communication load and learning accuracy for DMTL-ELM.
Tasks	Multi-Task Learning
Published	2019-04-25
URL	http://arxiv.org/abs/1904.11366v1
PDF	http://arxiv.org/pdf/1904.11366v1.pdf
PWC	https://paperswithcode.com/paper/decentralized-multi-task-learning-based-on
Repo
Framework

Exact Partitioning of High-order Models with a Novel Convex Tensor Cone Relaxation


Title	Exact Partitioning of High-order Models with a Novel Convex Tensor Cone Relaxation
Authors	Chuyang Ke, Jean Honorio
Abstract	In this paper we propose the first correct poly-time algorithm for exact partitioning of high-order models (a worst case NP-hard problem). We define a general class of $m$-degree Homogeneous Polynomial Models, which subsumes several examples motivated from prior literature. Exact partitioning can be formulated as a tensor optimization problem. We relax this NP-hard problem to a convex conic form problem (poly-time solvable by interior point methods). To this end, we carefully define the positive semidefinite tensor cone, and show its convexity, and the convexity of its dual cone. This allows us to construct a primal-dual certificate to show that the solution of the convex relaxation is correct (equal to the unobserved true group assignment) under some sample complexity conditions.
Tasks
Published	2019-11-06
URL	https://arxiv.org/abs/1911.02161v1
PDF	https://arxiv.org/pdf/1911.02161v1.pdf
PWC	https://paperswithcode.com/paper/exact-partitioning-of-high-order-models-with
Repo
Framework

Learning to Learn by Zeroth-Order Oracle


Title	Learning to Learn by Zeroth-Order Oracle
Authors	Yangjun Ruan, Yuanhao Xiong, Sashank Reddi, Sanjiv Kumar, Cho-Jui Hsieh
Abstract	In the learning to learn (L2L) framework, we cast the design of optimization algorithms as a machine learning problem and use deep neural networks to learn the update rules. In this paper, we extend the L2L framework to zeroth-order (ZO) optimization setting, where no explicit gradient information is available. Our learned optimizer, modeled as recurrent neural network (RNN), first approximates gradient by ZO gradient estimator and then produces parameter update utilizing the knowledge of previous iterations. To reduce high variance effect due to ZO gradient estimator, we further introduce another RNN to learn the Gaussian sampling rule and dynamically guide the query direction sampling. Our learned optimizer outperforms hand-designed algorithms in terms of convergence rate and final solution on both synthetic and practical ZO optimization tasks (in particular, the black-box adversarial attack task, which is one of the most widely used tasks of ZO optimization). We finally conduct extensive analytical experiments to demonstrate the effectiveness of our proposed optimizer.
Tasks	Adversarial Attack
Published	2019-10-21
URL	https://arxiv.org/abs/1910.09464v2
PDF	https://arxiv.org/pdf/1910.09464v2.pdf
PWC	https://paperswithcode.com/paper/learning-to-learn-by-zeroth-order-oracle
Repo
Framework

Extraction of Complex DNN Models: Real Threat or Boogeyman?


Title	Extraction of Complex DNN Models: Real Threat or Boogeyman?
Authors	Buse Gul Atli, Sebastian Szyller, Mika Juuti, Samuel Marchal, N. Asokan
Abstract	Recently, machine learning (ML) has introduced advanced solutions to many domains. Since ML models provide business advantage to model owners, protecting intellectual property of ML models has emerged as an important consideration. Confidentiality of ML models can be protected by exposing them to clients only via prediction APIs. However, model extraction attacks can steal the functionality of ML models using the information leaked to clients through the results returned via the API. In this work, we question whether model extraction is a serious threat to complex, real-life ML models. We evaluate the current state-of-the-art model extraction attack (Knockoff nets) against complex models. We reproduce and confirm the results in the original paper. But we also show that the performance of this attack can be limited by several factors, including ML model architecture and the granularity of API response. Furthermore, we introduce a defense based on distinguishing queries used for Knockoff nets from benign queries. Despite the limitations of the Knockoff nets, we show that a more realistic adversary can effectively steal complex ML models and evade known defenses.
Tasks
Published	2019-10-11
URL	https://arxiv.org/abs/1910.05429v2
PDF	https://arxiv.org/pdf/1910.05429v2.pdf
PWC	https://paperswithcode.com/paper/extraction-of-complex-dnn-models-real-threat
Repo
Framework

Lexicographic Multiarmed Bandit


Title	Lexicographic Multiarmed Bandit
Authors	Alihan Hüyük, Cem Tekin
Abstract	We consider a multiobjective multiarmed bandit problem with lexicographically ordered objectives. In this problem, the goal of the learner is to select arms that are lexicographic optimal as much as possible without knowing the arm reward distributions beforehand. We capture this goal by defining a multidimensional form of regret that measures the loss of the learner due to not selecting lexicographic optimal arms, and then, consider two settings where the learner has prior information on the expected arm rewards. In the first setting, the learner only knows for each objective the lexicographic optimal expected reward. In the second setting, it only knows for each objective near-lexicographic optimal expected rewards. For both settings we prove that the learner achieves expected regret uniformly bounded in time. The algorithm we propose for the second setting also attains bounded regret for the multiarmed bandit with satisficing objectives. In addition, we also consider the harder prior-free case, and show that the learner can still achieve sublinear in time gap-free regret. Finally, we experimentally evaluate performance of the proposed algorithms in a variety of multiobjective learning problems.
Tasks
Published	2019-07-26
URL	https://arxiv.org/abs/1907.11605v2
PDF	https://arxiv.org/pdf/1907.11605v2.pdf
PWC	https://paperswithcode.com/paper/lexicographic-multiarmed-bandit
Repo
Framework