July 27, 2019

3332 words 16 mins read

Paper Group ANR 467

Listening while Speaking: Speech Chain by Deep Learning. Private federated learning on vertically partitioned data via entity resolution and additively homomorphic encryption. Ease.ml: Towards Multi-tenant Resource Sharing for Machine Learning Workloads. Deep Tensor Encoding. Explainable Entity-based Recommendations with Knowledge Graphs. Zero-Shot …

Listening while Speaking: Speech Chain by Deep Learning


Title	Listening while Speaking: Speech Chain by Deep Learning
Authors	Andros Tjandra, Sakriani Sakti, Satoshi Nakamura
Abstract	Despite the close relationship between speech perception and production, research in automatic speech recognition (ASR) and text-to-speech synthesis (TTS) has progressed more or less independently without exerting much mutual influence on each other. In human communication, on the other hand, a closed-loop speech chain mechanism with auditory feedback from the speaker’s mouth to her ear is crucial. In this paper, we take a step further and develop a closed-loop speech chain model based on deep learning. The sequence-to-sequence model in close-loop architecture allows us to train our model on the concatenation of both labeled and unlabeled data. While ASR transcribes the unlabeled speech features, TTS attempts to reconstruct the original speech waveform based on the text from ASR. In the opposite direction, ASR also attempts to reconstruct the original text transcription given the synthesized speech. To the best of our knowledge, this is the first deep learning model that integrates human speech perception and production behaviors. Our experimental results show that the proposed approach significantly improved the performance more than separate systems that were only trained with labeled data.
Tasks	Speech Recognition, Speech Synthesis, Text-To-Speech Synthesis
Published	2017-07-16
URL	http://arxiv.org/abs/1707.04879v1
PDF	http://arxiv.org/pdf/1707.04879v1.pdf
PWC	https://paperswithcode.com/paper/listening-while-speaking-speech-chain-by-deep
Repo
Framework

Private federated learning on vertically partitioned data via entity resolution and additively homomorphic encryption


Title	Private federated learning on vertically partitioned data via entity resolution and additively homomorphic encryption
Authors	Stephen Hardy, Wilko Henecka, Hamish Ivey-Law, Richard Nock, Giorgio Patrini, Guillaume Smith, Brian Thorne
Abstract	Consider two data providers, each maintaining private records of different feature sets about common entities. They aim to learn a linear model jointly in a federated setting, namely, data is local and a shared model is trained from locally computed updates. In contrast with most work on distributed learning, in this scenario (i) data is split vertically, i.e. by features, (ii) only one data provider knows the target variable and (iii) entities are not linked across the data providers. Hence, to the challenge of private learning, we add the potentially negative consequences of mistakes in entity resolution. Our contribution is twofold. First, we describe a three-party end-to-end solution in two phases —privacy-preserving entity resolution and federated logistic regression over messages encrypted with an additively homomorphic scheme—, secure against a honest-but-curious adversary. The system allows learning without either exposing data in the clear or sharing which entities the data providers have in common. Our implementation is as accurate as a naive non-private solution that brings all data in one place, and scales to problems with millions of entities with hundreds of features. Second, we provide what is to our knowledge the first formal analysis of the impact of entity resolution’s mistakes on learning, with results on how optimal classifiers, empirical losses, margins and generalisation abilities are affected. Our results bring a clear and strong support for federated learning: under reasonable assumptions on the number and magnitude of entity resolution’s mistakes, it can be extremely beneficial to carry out federated learning in the setting where each peer’s data provides a significant uplift to the other.
Tasks	Entity Resolution
Published	2017-11-29
URL	http://arxiv.org/abs/1711.10677v1
PDF	http://arxiv.org/pdf/1711.10677v1.pdf
PWC	https://paperswithcode.com/paper/private-federated-learning-on-vertically
Repo
Framework


Title	Ease.ml: Towards Multi-tenant Resource Sharing for Machine Learning Workloads
Authors	Tian Li, Jie Zhong, Ji Liu, Wentao Wu, Ce Zhang
Abstract	We present ease.ml, a declarative machine learning service platform we built to support more than ten research groups outside the computer science departments at ETH Zurich for their machine learning needs. With ease.ml, a user defines the high-level schema of a machine learning application and submits the task via a Web interface. The system automatically deals with the rest, such as model selection and data movement. In this paper, we describe the ease.ml architecture and focus on a novel technical problem introduced by ease.ml regarding resource allocation. We ask, as a “service provider” that manages a shared cluster of machines among all our users running machine learning workloads, what is the resource allocation strategy that maximizes the global satisfaction of all our users? Resource allocation is a critical yet subtle issue in this multi-tenant scenario, as we have to balance between efficiency and fairness. We first formalize the problem that we call multi-tenant model selection, aiming for minimizing the total regret of all users running automatic model selection tasks. We then develop a novel algorithm that combines multi-armed bandits with Bayesian optimization and prove a regret bound under the multi-tenant setting. Finally, we report our evaluation of ease.ml on synthetic data and on one service we are providing to our users, namely, image classification with deep neural networks. Our experimental evaluation results show that our proposed solution can be up to 9.8x faster in achieving the same global quality for all users as the two popular heuristics used by our users before ease.ml.
Tasks	Image Classification, Model Selection, Multi-Armed Bandits
Published	2017-08-24
URL	http://arxiv.org/abs/1708.07308v1
PDF	http://arxiv.org/pdf/1708.07308v1.pdf
PWC	https://paperswithcode.com/paper/easeml-towards-multi-tenant-resource-sharing
Repo
Framework

Deep Tensor Encoding


Title	Deep Tensor Encoding
Authors	B Sengupta, E Vasquez, Y Qian
Abstract	Learning an encoding of feature vectors in terms of an over-complete dictionary or a information geometric (Fisher vectors) construct is wide-spread in statistical signal processing and computer vision. In content based information retrieval using deep-learning classifiers, such encodings are learnt on the flattened last layer, without adherence to the multi-linear structure of the underlying feature tensor. We illustrate a variety of feature encodings incl. sparse dictionary coding and Fisher vectors along with proposing that a structured tensor factorization scheme enables us to perform retrieval that can be at par, in terms of average precision, with Fisher vector encoded image signatures. In short, we illustrate how structural constraints increase retrieval fidelity.
Tasks	Information Retrieval
Published	2017-03-18
URL	http://arxiv.org/abs/1703.06324v2
PDF	http://arxiv.org/pdf/1703.06324v2.pdf
PWC	https://paperswithcode.com/paper/deep-tensor-encoding
Repo
Framework

Explainable Entity-based Recommendations with Knowledge Graphs


Title	Explainable Entity-based Recommendations with Knowledge Graphs
Authors	Rose Catherine, Kathryn Mazaitis, Maxine Eskenazi, William Cohen
Abstract	Explainable recommendation is an important task. Many methods have been proposed which generate explanations from the content and reviews written for items. When review text is unavailable, generating explanations is still a hard problem. In this paper, we illustrate how explanations can be generated in such a scenario by leveraging external knowledge in the form of knowledge graphs. Our method jointly ranks items and knowledge graph entities using a Personalized PageRank procedure to produce recommendations together with their explanations.
Tasks	Knowledge Graphs
Published	2017-07-12
URL	http://arxiv.org/abs/1707.05254v1
PDF	http://arxiv.org/pdf/1707.05254v1.pdf
PWC	https://paperswithcode.com/paper/explainable-entity-based-recommendations-with
Repo
Framework

Zero-Shot Learning with Generative Latent Prototype Model


Title	Zero-Shot Learning with Generative Latent Prototype Model
Authors	Yanan Li, Donghui Wang
Abstract	Zero-shot learning, which studies the problem of object classification for categories for which we have no training examples, is gaining increasing attention from community. Most existing ZSL methods exploit deterministic transfer learning via an in-between semantic embedding space. In this paper, we try to attack this problem from a generative probabilistic modelling perspective. We assume for any category, the observed representation, e.g. images or texts, is developed from a unique prototype in a latent space, in which the semantic relationship among prototypes is encoded via linear reconstruction. Taking advantage of this assumption, virtual instances of unseen classes can be generated from the corresponding prototype, giving rise to a novel ZSL model which can alleviate the domain shift problem existing in the way of direct transfer learning. Extensive experiments on three benchmark datasets show our proposed model can achieve state-of-the-art results.
Tasks	Object Classification, Transfer Learning, Zero-Shot Learning
Published	2017-05-26
URL	http://arxiv.org/abs/1705.09474v1
PDF	http://arxiv.org/pdf/1705.09474v1.pdf
PWC	https://paperswithcode.com/paper/zero-shot-learning-with-generative-latent
Repo
Framework

Multi-feature classifiers for burst detection in single EEG channels from preterm infants


Title	Multi-feature classifiers for burst detection in single EEG channels from preterm infants
Authors	X. Navarro, F. Porée, M. Kuchenbuch, M. Chavez, A. Beuchée, G. Carrault
Abstract	The study of electroencephalographic (EEG) bursts in preterm infants provides valuable information about maturation or prognostication after perinatal asphyxia. Over the last two decades, a number of works proposed algorithms to automatically detect EEG bursts in preterm infants, but they were designed for populations under 35 weeks of post menstrual age (PMA). However, as the brain activity evolves rapidly during postnatal life, these solutions might be under-performing with increasing PMA. In this work we focused on preterm infants reaching term ages (PMA $\geq$ 36 weeks) using multi-feature classification on a single EEG channel. Five EEG burst detectors relying on different machine learning approaches were compared: Logistic regression (LR), linear discriminant analysis (LDA), k-nearest neighbors (kNN), support vector machines (SVM) and thresholding (Th). Classifiers were trained by visually labeled EEG recordings from 14 very preterm infants (born after 28 weeks of gestation) with 36 - 41 weeks PMA. The most performing classifiers reached about 95% accuracy (kNN, SVM and LR) whereas Th obtained 84%. Compared to human-automatic agreements, LR provided the highest scores (Cohen’s kappa = 0.71) and the best computational efficiency using only three EEG features. Applying this classifier in a test database of 21 infants $\geq$ 36 weeks PMA, we show that long EEG bursts and short inter-bust periods are characteristic of infants with the highest PMA and weights. In view of these results, LR-based burst detection could be a suitable tool to study maturation in monitoring or portable devices using a single EEG channel.
Tasks	EEG
Published	2017-02-08
URL	http://arxiv.org/abs/1702.02873v1
PDF	http://arxiv.org/pdf/1702.02873v1.pdf
PWC	https://paperswithcode.com/paper/multi-feature-classifiers-for-burst-detection
Repo
Framework

Maximum Margin Principal Components


Title	Maximum Margin Principal Components
Authors	Xianghui Luo, Robert J. Durrant
Abstract	Principal Component Analysis (PCA) is a very successful dimensionality reduction technique, widely used in predictive modeling. A key factor in its widespread use in this domain is the fact that the projection of a dataset onto its first $K$ principal components minimizes the sum of squared errors between the original data and the projected data over all possible rank $K$ projections. Thus, PCA provides optimal low-rank representations of data for least-squares linear regression under standard modeling assumptions. On the other hand, when the loss function for a prediction problem is not the least-squares error, PCA is typically a heuristic choice of dimensionality reduction – in particular for classification problems under the zero-one loss. In this paper we target classification problems by proposing a straightforward alternative to PCA that aims to minimize the difference in margin distribution between the original and the projected data. Extensive experiments show that our simple approach typically outperforms PCA on any particular dataset, in terms of classification error, though this difference is not always statistically significant, and despite being a filter method is frequently competitive with Partial Least Squares (PLS) and Lasso on a wide range of datasets.
Tasks	Dimensionality Reduction
Published	2017-05-17
URL	http://arxiv.org/abs/1705.06371v1
PDF	http://arxiv.org/pdf/1705.06371v1.pdf
PWC	https://paperswithcode.com/paper/maximum-margin-principal-components
Repo
Framework

Robust Loss Functions under Label Noise for Deep Neural Networks


Title	Robust Loss Functions under Label Noise for Deep Neural Networks
Authors	Aritra Ghosh, Himanshu Kumar, P. S. Sastry
Abstract	In many applications of classifier learning, training data suffers from label noise. Deep networks are learned using huge training data where the problem of noisy labels is particularly relevant. The current techniques proposed for learning deep networks under label noise focus on modifying the network architecture and on algorithms for estimating true labels from noisy labels. An alternate approach would be to look for loss functions that are inherently noise-tolerant. For binary classification there exist theoretical results on loss functions that are robust to label noise. In this paper, we provide some sufficient conditions on a loss function so that risk minimization under that loss function would be inherently tolerant to label noise for multiclass classification problems. These results generalize the existing results on noise-tolerant loss functions for binary classification. We study some of the widely used loss functions in deep networks and show that the loss function based on mean absolute value of error is inherently robust to label noise. Thus standard back propagation is enough to learn the true classifier even under label noise. Through experiments, we illustrate the robustness of risk minimization with such loss functions for learning neural networks.
Tasks
Published	2017-12-27
URL	http://arxiv.org/abs/1712.09482v1
PDF	http://arxiv.org/pdf/1712.09482v1.pdf
PWC	https://paperswithcode.com/paper/robust-loss-functions-under-label-noise-for
Repo
Framework

Improvements to context based self-supervised learning


Title	Improvements to context based self-supervised learning
Authors	T. Nathan Mundhenk, Daniel Ho, Barry Y. Chen
Abstract	We develop a set of methods to improve on the results of self-supervised learning using context. We start with a baseline of patch based arrangement context learning and go from there. Our methods address some overt problems such as chromatic aberration as well as other potential problems such as spatial skew and mid-level feature neglect. We prevent problems with testing generalization on common self-supervised benchmark tests by using different datasets during our development. The results of our methods combined yield top scores on all standard self-supervised benchmarks, including classification and detection on PASCAL VOC 2007, segmentation on PASCAL VOC 2012, and “linear tests” on the ImageNet and CSAIL Places datasets. We obtain an improvement over our baseline method of between 4.0 to 7.1 percentage points on transfer learning classification tests. We also show results on different standard network architectures to demonstrate generalization as well as portability. All data, models and programs are available at: https://gdo-datasci.llnl.gov/selfsupervised/.
Tasks	Transfer Learning
Published	2017-11-17
URL	http://arxiv.org/abs/1711.06379v3
PDF	http://arxiv.org/pdf/1711.06379v3.pdf
PWC	https://paperswithcode.com/paper/improvements-to-context-based-self-supervised
Repo
Framework

Training of Deep Neural Networks based on Distance Measures using RMSProp


Title	Training of Deep Neural Networks based on Distance Measures using RMSProp
Authors	Thomas Kurbiel, Shahrzad Khaleghian
Abstract	The vanishing gradient problem was a major obstacle for the success of deep learning. In recent years it was gradually alleviated through multiple different techniques. However the problem was not really overcome in a fundamental way, since it is inherent to neural networks with activation functions based on dot products. In a series of papers, we are going to analyze alternative neural network structures which are not based on dot products. In this first paper, we revisit neural networks built up of layers based on distance measures and Gaussian activation functions. These kinds of networks were only sparsely used in the past since they are hard to train when using plain stochastic gradient descent methods. We show that by using Root Mean Square Propagation (RMSProp) it is possible to efficiently learn multi-layer neural networks. Furthermore we show that when appropriately initialized these kinds of neural networks suffer much less from the vanishing and exploding gradient problem than traditional neural networks even for deep networks.
Tasks
Published	2017-08-06
URL	http://arxiv.org/abs/1708.01911v1
PDF	http://arxiv.org/pdf/1708.01911v1.pdf
PWC	https://paperswithcode.com/paper/training-of-deep-neural-networks-based-on
Repo
Framework

Identification of Unmodeled Objects from Symbolic Descriptions


Title	Identification of Unmodeled Objects from Symbolic Descriptions
Authors	Andrea Baisero, Stefan Otte, Peter Englert, Marc Toussaint
Abstract	Successful human-robot cooperation hinges on each agent’s ability to process and exchange information about the shared environment and the task at hand. Human communication is primarily based on symbolic abstractions of object properties, rather than precise quantitative measures. A comprehensive robotic framework thus requires an integrated communication module which is able to establish a link and convert between perceptual and abstract information. The ability to interpret composite symbolic descriptions enables an autonomous agent to a) operate in unstructured and cluttered environments, in tasks which involve unmodeled or never seen before objects; and b) exploit the aggregation of multiple symbolic properties as an instance of ensemble learning, to improve identification performance even when the individual predicates encode generic information or are imprecisely grounded. We propose a discriminative probabilistic model which interprets symbolic descriptions to identify the referent object contextually w.r.t.\ the structure of the environment and other objects. The model is trained using a collected dataset of identifications, and its performance is evaluated by quantitative measures and a live demo developed on the PR2 robot platform, which integrates elements of perception, object extraction, object identification and grasping.
Tasks
Published	2017-01-23
URL	http://arxiv.org/abs/1701.06450v1
PDF	http://arxiv.org/pdf/1701.06450v1.pdf
PWC	https://paperswithcode.com/paper/identification-of-unmodeled-objects-from
Repo
Framework

Graph Based Recommendations: From Data Representation to Feature Extraction and Application


Title	Graph Based Recommendations: From Data Representation to Feature Extraction and Application
Authors	Amit Tiroshi, Tsvi Kuflik, Shlomo Berkovsky, Mohamed Ali Kaafar
Abstract	Modeling users for the purpose of identifying their preferences and then personalizing services on the basis of these models is a complex task, primarily due to the need to take into consideration various explicit and implicit signals, missing or uncertain information, contextual aspects, and more. In this study, a novel generic approach for uncovering latent preference patterns from user data is proposed and evaluated. The approach relies on representing the data using graphs, and then systematically extracting graph-based features and using them to enrich the original user models. The extracted features encapsulate complex relationships between users, items, and metadata. The enhanced user models can then serve as an input to any recommendation algorithm. The proposed approach is domain-independent (demonstrated on data from movies, music, and business recommender systems), and is evaluated using several state-of-the-art machine learning methods, on different recommendation tasks, and using different evaluation metrics. The results show a unanimous improvement in the recommendation accuracy across tasks and domains. In addition, the evaluation provides a deeper analysis regarding the performance of the approach in special scenarios, including high sparsity and variability of ratings.
Tasks	Recommendation Systems
Published	2017-07-05
URL	http://arxiv.org/abs/1707.01250v1
PDF	http://arxiv.org/pdf/1707.01250v1.pdf
PWC	https://paperswithcode.com/paper/graph-based-recommendations-from-data
Repo
Framework

Minimum energy path calculations with Gaussian process regression


Title	Minimum energy path calculations with Gaussian process regression
Authors	Olli-Pekka Koistinen, Emile Maras, Aki Vehtari, Hannes Jónsson
Abstract	The calculation of minimum energy paths for transitions such as atomic and/or spin re-arrangements is an important task in many contexts and can often be used to determine the mechanism and rate of transitions. An important challenge is to reduce the computational effort in such calculations, especially when ab initio or electron density functional calculations are used to evaluate the energy since they can require large computational effort. Gaussian process regression is used here to reduce significantly the number of energy evaluations needed to find minimum energy paths of atomic rearrangements. By using results of previous calculations to construct an approximate energy surface and then converge to the minimum energy path on that surface in each Gaussian process iteration, the number of energy evaluations is reduced significantly as compared with regular nudged elastic band calculations. For a test problem involving rearrangements of a heptamer island on a crystal surface, the number of energy evaluations is reduced to less than a fifth. The scaling of the computational effort with the number of degrees of freedom as well as various possible further improvements to this approach are discussed.
Tasks
Published	2017-03-30
URL	http://arxiv.org/abs/1703.10423v1
PDF	http://arxiv.org/pdf/1703.10423v1.pdf
PWC	https://paperswithcode.com/paper/minimum-energy-path-calculations-with
Repo
Framework

Learning Graphical Models from a Distributed Stream


Title	Learning Graphical Models from a Distributed Stream
Authors	Yu Zhang, Srikanta Tirthapura, Graham Cormode
Abstract	A current challenge for data management systems is to support the construction and maintenance of machine learning models over data that is large, multi-dimensional, and evolving. While systems that could support these tasks are emerging, the need to scale to distributed, streaming data requires new models and algorithms. In this setting, as well as computational scalability and model accuracy, we also need to minimize the amount of communication between distributed processors, which is the chief component of latency. We study Bayesian networks, the workhorse of graphical models, and present a communication-efficient method for continuously learning and maintaining a Bayesian network model over data that is arriving as a distributed stream partitioned across multiple processors. We show a strategy for maintaining model parameters that leads to an exponential reduction in communication when compared with baseline approaches to maintain the exact MLE (maximum likelihood estimation). Meanwhile, our strategy provides similar prediction errors for the target distribution and for classification tasks.
Tasks
Published	2017-10-05
URL	http://arxiv.org/abs/1710.02103v1
PDF	http://arxiv.org/pdf/1710.02103v1.pdf
PWC	https://paperswithcode.com/paper/learning-graphical-models-from-a-distributed
Repo
Framework