January 28, 2020

3126 words 15 mins read

Paper Group ANR 988

Using Quantifier Elimination to Enhance the Safety Assurance of Deep Neural Networks. Privacy is What We Care About: Experimental Investigation of Federated Learning on Edge Devices. Estimation of Spectral Risk Measures. Generating an Explainable ECG Beat Space With Variational Auto-Encoders. SchrödingeRNN: Generative Modeling of Raw Audio as a Con …

Using Quantifier Elimination to Enhance the Safety Assurance of Deep Neural Networks


Title	Using Quantifier Elimination to Enhance the Safety Assurance of Deep Neural Networks
Authors	Hao Ren, Sai Krishnan Chandrasekar, Anitha Murugesan
Abstract	Advances in the field of Machine Learning and Deep Neural Networks (DNNs) has enabled rapid development of sophisticated and autonomous systems. However, the inherent complexity to rigorously assure the safe operation of such systems hinders their real-world adoption in safety-critical domains such as aerospace and medical devices. Hence, there is a surge in interest to explore the use of advanced mathematical techniques such as formal methods to address this challenge. In fact, the initial results of such efforts are promising. Along these lines, we propose the use of quantifier elimination (QE) - a formal method technique, as a complimentary technique to the state-of-the-art static analysis and verification procedures. Using an airborne collision avoidance DNN as a case example, we illustrate the use of QE to formulate the precise range forward propagation through a network as well as analyze its robustness. We discuss the initial results of this ongoing work and explore the future possibilities of extending this approach and/or integrating it with other approaches to perform advanced safety assurance of DNNs.
Tasks
Published	2019-09-18
URL	https://arxiv.org/abs/1909.09142v1
PDF	https://arxiv.org/pdf/1909.09142v1.pdf
PWC	https://paperswithcode.com/paper/using-quantifier-elimination-to-enhance-the
Repo
Framework

Privacy is What We Care About: Experimental Investigation of Federated Learning on Edge Devices


Title	Privacy is What We Care About: Experimental Investigation of Federated Learning on Edge Devices
Authors	Anirban Das, Thomas Brunschwiler
Abstract	Federated Learning enables training of a general model through edge devices without sending raw data to the cloud. Hence, this approach is attractive for digital health applications, where data is sourced through edge devices and users care about privacy. Here, we report on the feasibility to train deep neural networks on the Raspberry Pi4s as edge devices. A CNN, a LSTM and a MLP were successfully trained on the MNIST data-set. Further, federated learning is demonstrated experimentally on IID and non-IID samples in a parametric study, to benchmark the model convergence. The weight updates from the workers are shared with the cloud to train the general model through federated learning. With the CNN and the non-IID samples a test-accuracy of up to 85% could be achieved within a training time of 2 minutes, while exchanging less than $10$ MB data per device. In addition, we discuss federated learning from an use-case standpoint, elaborating on privacy risks and labeling requirements for the application of emotion detection from sound. Based on the experimental findings, we discuss possible research directions to improve model and system performance. Finally, we provide best practices for a practitioner, considering the implementation of federated learning.
Tasks
Published	2019-11-11
URL	https://arxiv.org/abs/1911.04559v1
PDF	https://arxiv.org/pdf/1911.04559v1.pdf
PWC	https://paperswithcode.com/paper/privacy-is-what-we-care-about-experimental
Repo
Framework

Estimation of Spectral Risk Measures


Title	Estimation of Spectral Risk Measures
Authors	Ajay Kumar Pandey, Prashanth L. A., Sanjay P. Bhat
Abstract	We consider the problem of estimating a spectral risk measure (SRM) from i.i.d. samples, and propose a novel method that is based on numerical integration. We show that our SRM estimate concentrates exponentially, when the underlying distribution has bounded support. Further, we also consider the case when the underlying distribution is either Gaussian or exponential, and derive a concentration bound for our estimation scheme. We validate the theoretical findings on a synthetic setup, and in a vehicular traffic routing application.
Tasks
Published	2019-12-22
URL	https://arxiv.org/abs/1912.10398v1
PDF	https://arxiv.org/pdf/1912.10398v1.pdf
PWC	https://paperswithcode.com/paper/estimation-of-spectral-risk-measures
Repo
Framework

Generating an Explainable ECG Beat Space With Variational Auto-Encoders


Title	Generating an Explainable ECG Beat Space With Variational Auto-Encoders
Authors	Tom Van Steenkiste, Dirk Deschrijver, Tom Dhaene
Abstract	Electrocardiogram signals are omnipresent in medicine. A vital aspect in the analysis of this data is the identification and classification of heart beat types which is often done through automated algorithms. Advancements in neural networks and deep learning have led to a high classification accuracy. However, the final adoption of these models into clinical practice is limited due to the black-box nature of the methods. In this work, we explore the use of variational auto-encoders based on linear dense networks to learn human interpretable beat embeddings in time-series data. We demonstrate that using this method, an interpretable and explainable ECG beat space can be generated, set up by characteristic base beats.
Tasks	Time Series
Published	2019-11-12
URL	https://arxiv.org/abs/1911.04898v1
PDF	https://arxiv.org/pdf/1911.04898v1.pdf
PWC	https://paperswithcode.com/paper/generating-an-explainable-ecg-beat-space-with
Repo
Framework

SchrödingeRNN: Generative Modeling of Raw Audio as a Continuously Observed Quantum State


Title	SchrödingeRNN: Generative Modeling of Raw Audio as a Continuously Observed Quantum State
Authors	Beñat Mencia Uranga, Austen Lamacraft
Abstract	We introduce Schr"odingeRNN, a quantum inspired generative model for raw audio. Audio data is wave-like and is sampled from a continuous signal. Although generative modelling of raw audio has made great strides lately, relational inductive biases relevant to these two characteristics are mostly absent from models explored to date. Quantum Mechanics is a natural source of probabilistic models of wave behaviour. Our model takes the form of a stochastic Schr"odinger equation describing the continuous time measurement of a quantum system, and is equivalent to the continuous Matrix Product State (cMPS) representation of wavefunctions in one dimensional many-body systems. This constitutes a deep autoregressive architecture in which the systems state is a latent representation of the past observations. We test our model on synthetic data sets of stationary and non-stationary signals. This is the first time cMPS are used in machine learning.
Tasks
Published	2019-11-26
URL	https://arxiv.org/abs/1911.11879v1
PDF	https://arxiv.org/pdf/1911.11879v1.pdf
PWC	https://paperswithcode.com/paper/schrodingernn-generative-modeling-of-raw
Repo
Framework

Matrix embedding method in match for session-based recommendation


Title	Matrix embedding method in match for session-based recommendation
Authors	Qizhi Zhang, Yi Lin, Kangle Wu, Yongliang Li, Anxiang Zeng
Abstract	Session based model is widely used in recommend system. It use the user click sequence as input of a Recurrent Neural Network (RNN), and get the output of the RNN network as the vector embedding of the session, and use the inner product of the vector embedding of session and the vector embedding of the next item as the score that is the metric of the interest to the next item. This method can be used for the “match” stage for the recommendation system whose item number is very big by using some index method like KD-Tree or Ball-Tree and etc.. But this method repudiate the variousness of the interest of user in a session. We generated the model to modify the vector embedding of session to a symmetric matrix embedding, that is equivalent to a quadratic form on the vector space of items. The score is builded as the value of the vector embedding of next item under the quadratic form. The eigenvectors of the symmetric matrix embedding corresponding to the positive eigenvalues are conjectured to represent the interests of user in the session. This method can be used for the “match” stage also. The experiments show that this method is better than the method of vector embedding.
Tasks	Session-Based Recommendations
Published	2019-08-27
URL	https://arxiv.org/abs/1908.10180v1
PDF	https://arxiv.org/pdf/1908.10180v1.pdf
PWC	https://paperswithcode.com/paper/matrix-embedding-method-in-match-for-session
Repo
Framework

Unsupervised Pre-training for Natural Language Generation: A Literature Review


Title	Unsupervised Pre-training for Natural Language Generation: A Literature Review
Authors	Yuanxin Liu, Zheng Lin
Abstract	Recently, unsupervised pre-training is gaining increasing popularity in the realm of computational linguistics, thanks to its surprising success in advancing natural language understanding (NLU) and the potential to effectively exploit large-scale unlabelled corpus. However, regardless of the success in NLU, the power of unsupervised pre-training is only partially excavated when it comes to natural language generation (NLG). The major obstacle stems from an idiosyncratic nature of NLG: Texts are usually generated based on certain context, which may vary with the target applications. As a result, it is intractable to design a universal architecture for pre-training as in NLU scenarios. Moreover, retaining the knowledge learned from pre-training when learning on the target task is also a non-trivial problem. This review summarizes the recent efforts to enhance NLG systems with unsupervised pre-training, with a special focus on the methods to catalyse the integration of pre-trained models into downstream tasks. They are classified into architecture-based methods and strategy-based methods, based on their way of handling the above obstacle. Discussions are also provided to give further insights into the relationship between these two lines of work, some informative empirical phenomenons, as well as some possible directions where future work can be devoted to.
Tasks	Text Generation
Published	2019-11-13
URL	https://arxiv.org/abs/1911.06171v1
PDF	https://arxiv.org/pdf/1911.06171v1.pdf
PWC	https://paperswithcode.com/paper/unsupervised-pre-training-for-natural
Repo
Framework

STNReID : Deep Convolutional Networks with Pairwise Spatial Transformer Networks for Partial Person Re-identification


Title	STNReID : Deep Convolutional Networks with Pairwise Spatial Transformer Networks for Partial Person Re-identification
Authors	Hao Luo, Xing Fan, Chi Zhang, Wei Jiang
Abstract	Partial person re-identification (ReID) is a challenging task because only partial information of person images is available for matching target persons. Few studies, especially on deep learning, have focused on matching partial person images with holistic person images. This study presents a novel deep partial ReID framework based on pairwise spatial transformer networks (STNReID), which can be trained on existing holistic person datasets. STNReID includes a spatial transformer network (STN) module and a ReID module. The STN module samples an affined image (a semantically corresponding patch) from the holistic image to match the partial image. The ReID module extracts the features of the holistic, partial, and affined images. Competition (or confrontation) is observed between the STN module and the ReID module, and two-stage training is applied to acquire a strong STNReID for partial ReID. Experimental results show that our STNReID obtains 66.7% and 54.6% rank-1 accuracies on partial ReID and partial iLIDS datasets, respectively. These values are at par with those obtained with state-of-the-art methods.
Tasks	Person Re-Identification
Published	2019-03-17
URL	https://arxiv.org/abs/1903.07072v2
PDF	https://arxiv.org/pdf/1903.07072v2.pdf
PWC	https://paperswithcode.com/paper/stnreid-deep-convolutional-networks-with
Repo
Framework

Local Model Poisoning Attacks to Byzantine-Robust Federated Learning


Title	Local Model Poisoning Attacks to Byzantine-Robust Federated Learning
Authors	Minghong Fang, Xiaoyu Cao, Jinyuan Jia, Neil Zhenqiang Gong
Abstract	In federated learning, multiple client devices jointly learn a machine learning model: each client device maintains a local model for its local training dataset, while a master device maintains a global model via aggregating the local models from the client devices. The machine learning community recently proposed several federated learning methods that were claimed to be robust against Byzantine failures (e.g., system failures, adversarial manipulations) of certain client devices. In this work, we perform the first systematic study on local model poisoning attacks to federated learning. We assume an attacker has compromised some client devices, and the attacker manipulates the local model parameters on the compromised client devices during the learning process such that the global model has a large testing error rate. We formulate our attacks as optimization problems and apply our attacks to four recent Byzantine-robust federated learning methods. Our empirical results on four real-world datasets show that our attacks can substantially increase the error rates of the models learnt by the federated learning methods that were claimed to be robust against Byzantine failures of some client devices. We generalize two defenses for data poisoning attacks to defend against our local model poisoning attacks. Our evaluation results show that one defense can effectively defend against our attacks in some cases, but the defenses are not effective enough in other cases, highlighting the need for new defenses against our local model poisoning attacks to federated learning.
Tasks	data poisoning
Published	2019-11-26
URL	https://arxiv.org/abs/1911.11815v1
PDF	https://arxiv.org/pdf/1911.11815v1.pdf
PWC	https://paperswithcode.com/paper/local-model-poisoning-attacks-to-byzantine
Repo
Framework

Progressively Growing Generative Adversarial Networks for High Resolution Semantic Segmentation of Satellite Images


Title	Progressively Growing Generative Adversarial Networks for High Resolution Semantic Segmentation of Satellite Images
Authors	Edward Collier, Kate Duffy, Sangram Ganguly, Geri Madanguit, Subodh Kalia, Gayaka Shreekant, Ramakrishna Nemani, Andrew Michaelis, Shuang Li, Auroop Ganguly, Supratik Mukhopadhyay
Abstract	Machine learning has proven to be useful in classification and segmentation of images. In this paper, we evaluate a training methodology for pixel-wise segmentation on high resolution satellite images using progressive growing of generative adversarial networks. We apply our model to segmenting building rooftops and compare these results to conventional methods for rooftop segmentation. We present our findings using the SpaceNet version 2 dataset. Progressive GAN training achieved a test accuracy of 93% compared to 89% for traditional GAN training.
Tasks	Image Generation, Semantic Segmentation
Published	2019-02-12
URL	http://arxiv.org/abs/1902.04604v1
PDF	http://arxiv.org/pdf/1902.04604v1.pdf
PWC	https://paperswithcode.com/paper/progressively-growing-generative-adversarial
Repo
Framework

Was ist eine Professur fuer Kuenstliche Intelligenz?


Title	Was ist eine Professur fuer Kuenstliche Intelligenz?
Authors	Kristian Kersting, Jan Peters, Constantin Rothkopf
Abstract	The Federal Government of Germany aims to boost the research in the field of Artificial Intelligence (AI). For instance, 100 new professorships are said to be established. However, the white paper of the government does not answer what an AI professorship is at all. In order to give colleagues, politicians, and citizens an idea, we present a view that is often followed when appointing professors for AI at German and international universities. We hope that it will help to establish a guideline with internationally accepted measures and thus make the public debate more informed.
Tasks
Published	2019-02-17
URL	http://arxiv.org/abs/1903.09516v1
PDF	http://arxiv.org/pdf/1903.09516v1.pdf
PWC	https://paperswithcode.com/paper/was-ist-eine-professur-fuer-kuenstliche
Repo
Framework

Bayesian machine learning for Boltzmann machine in quantum-enhanced feature spaces


Title	Bayesian machine learning for Boltzmann machine in quantum-enhanced feature spaces
Authors	Yusen Wu, Chao-hua Yu, Sujuan Qin, Qiaoyan Wen, Fei Gao
Abstract	Bayesian learning is ubiquitous for implementing classification and regression tasks, however, it is accompanied by computationally intractable limitations when the feature spaces become extremely large. Aiming to solve this problem, we develop a quantum bayesian learning framework of the restricted Boltzmann machine in the quantum-enhanced feature spaces. Our framework provides the encoding phase to map the real data and Boltzmann weight onto the quantum feature spaces and the training phase to learn an optimal inference function. Specifically, the training phase provides a physical quantity to measure the posterior distribution in quantum feature spaces, and this measure is utilized to design the quantum maximum a posterior (QMAP) algorithm and the quantum predictive distribution estimator (QPDE). It is shown that both quantum algorithms achieve exponential speed-up over their classical counterparts. Furthermore, it is interesting to note that our framework can figure out the classical bayesian learning tasks, i.e. processing the classical data and outputting corresponding classical labels. And a simulation, which is performed on an open-source software framework for quantum computing, illustrates that our algorithms show almost the same classification performance compared to their classical counterparts. Noting that the proposed quantum algorithms utilize the shallow circuit, our work is expected to be implemented on the noisy intermediate-scale quantum (NISQ) devices, and is one of the promising candidates to achieve quantum supremacy.
Tasks
Published	2019-12-20
URL	https://arxiv.org/abs/1912.10857v1
PDF	https://arxiv.org/pdf/1912.10857v1.pdf
PWC	https://paperswithcode.com/paper/bayesian-machine-learning-for-boltzmann
Repo
Framework

Probabilistic CCA with Implicit Distributions


Title	Probabilistic CCA with Implicit Distributions
Authors	Yaxin Shi, Yuangang Pan, Donna Xu, Ivor Tsang
Abstract	Canonical Correlation Analysis (CCA) is a classic technique for multi-view data analysis. To overcome the deficiency of linear correlation in practical multi-view learning tasks, various CCA variants were proposed to capture nonlinear dependency. However, it is non-trivial to have an in-principle understanding of these variants due to their inherent restrictive assumption on the data and latent code distributions. Although some works have studied probabilistic interpretation for CCA, these models still require the explicit form of the distributions to achieve a tractable solution for the inference. In this work, we study probabilistic interpretation for CCA based on implicit distributions. We present Conditional Mutual Information (CMI) as a new criterion for CCA to consider both linear and nonlinear dependency for arbitrarily distributed data. To eliminate direct estimation for CMI, in which explicit form of the distributions is still required, we derive an objective which can provide an estimation for CMI with efficient inference methods. To facilitate Bayesian inference of multi-view analysis, we propose Adversarial CCA (ACCA), which achieves consistent encoding for multi-view data with the consistent constraint imposed on the marginalization of the implicit posteriors. Such a model would achieve superiority in the alignment of the multi-view data with implicit distributions. It is interesting to note that most of the existing CCA variants can be connected with our proposed CCA model by assigning specific form for the posterior and likelihood distributions. Extensive experiments on nonlinear correlation analysis and cross-view generation on benchmark and real-world datasets demonstrate the superiority of our model.
Tasks	Bayesian Inference, MULTI-VIEW LEARNING
Published	2019-07-04
URL	https://arxiv.org/abs/1907.02345v1
PDF	https://arxiv.org/pdf/1907.02345v1.pdf
PWC	https://paperswithcode.com/paper/probabilistic-cca-with-implicit-distributions
Repo
Framework

Asymmetric Multiresolution Matrix Factorization


Title	Asymmetric Multiresolution Matrix Factorization
Authors	Pramod Kaushik Mudrakarta, Shubhendu Trivedi, Risi Kondor
Abstract	Multiresolution Matrix Factorization (MMF) was recently introduced as an alternative to the dominant low-rank paradigm in order to capture structure in matrices at multiple different scales. Using ideas from multiresolution analysis (MRA), MMF teased out hierarchical structure in symmetric matrices by constructing a sequence of wavelet bases. While effective for such matrices, there is plenty of data that is more naturally represented as nonsymmetric matrices (e.g. directed graphs), but nevertheless has similar hierarchical structure. In this paper, we explore techniques for extending MMF to any square matrix. We validate our approach on numerous matrix compression tasks, demonstrating its efficacy compared to low-rank methods. Moreover, we also show that a combined low-rank and MMF approach, which amounts to removing a small global-scale component of the matrix and then extracting hierarchical structure from the residual, is even more effective than each of the two complementary methods for matrix compression.
Tasks
Published	2019-10-10
URL	https://arxiv.org/abs/1910.05132v1
PDF	https://arxiv.org/pdf/1910.05132v1.pdf
PWC	https://paperswithcode.com/paper/asymmetric-multiresolution-matrix
Repo
Framework

Integrating Propositional and Relational Label Side Information for Hierarchical Zero-Shot Image Classification


Title	Integrating Propositional and Relational Label Side Information for Hierarchical Zero-Shot Image Classification
Authors	Colin Samplawski, Heesung Kwon, Erik Learned-Miller, Benjamin M. Marlin
Abstract	Zero-shot learning (ZSL) is one of the most extreme forms of learning from scarce labeled data. It enables predicting that images belong to classes for which no labeled training instances are available. In this paper, we present a new ZSL framework that leverages both label attribute side information and a semantic label hierarchy. We present two methods, lifted zero-shot prediction and a custom conditional random field (CRF) model, that integrate both forms of side information. We propose benchmark tasks for this framework that focus on making predictions across a range of semantic levels. We show that lifted zero-shot prediction can dramatically outperform baseline methods when making predictions within specified semantic levels, and that the probability distribution provided by the CRF model can be leveraged to yield further performance improvements when making unconstrained predictions over the hierarchy.
Tasks	Image Classification, Zero-Shot Learning
Published	2019-02-14
URL	http://arxiv.org/abs/1902.05492v1
PDF	http://arxiv.org/pdf/1902.05492v1.pdf
PWC	https://paperswithcode.com/paper/integrating-propositional-and-relational
Repo
Framework