Paper Group ANR 1154
Gaussian-Constrained training for speaker verification. Coins and Logic. Reaching Data Confidentiality and Model Accountability on the CalTrain. DeepGUM: Learning Deep Robust Regression with a Gaussian-Uniform Mixture Model. Language Model Supervision for Handwriting Recognition Model Adaptation. Harmonizable mixture kernels with variational Fourie …
Gaussian-Constrained training for speaker verification
| Title | Gaussian-Constrained training for speaker verification |
| Authors | Lantian Li, Zhiyuan Tang, Ying Shi, Dong Wang |
| Abstract | Neural models, in particular the d-vector and x-vector architectures, have produced state-of-the-art performance on many speaker verification tasks. However, two potential problems of these neural models deserve more investigation. Firstly, both models suffer from information leak', which means that some parameters participating in model training will be discarded during inference, i.e, the layers that are used as the classifier. Secondly, these models do not regulate the distribution of the derived speaker vectors. This unconstrained distribution’ may degrade the performance of the subsequent scoring component, e.g., PLDA. This paper proposes a Gaussian-constrained training approach that (1) discards the parametric classifier, and (2) enforces the distribution of the derived speaker vectors to be Gaussian. Our experiments on the VoxCeleb and SITW databases demonstrated that this new training approach produced more representative and regular speaker embeddings, leading to consistent performance improvement. |
| Tasks | Speaker Verification |
| Published | 2018-11-08 |
| URL | http://arxiv.org/abs/1811.03258v2 |
| http://arxiv.org/pdf/1811.03258v2.pdf | |
| PWC | https://paperswithcode.com/paper/gaussian-constrained-training-for-speaker |
| Repo | |
| Framework | |
Coins and Logic
| Title | Coins and Logic |
| Authors | Tanya Khovanova |
| Abstract | We establish fun parallels between coin-weighing puzzles and knights-and-knaves puzzles. |
| Tasks | |
| Published | 2018-01-03 |
| URL | http://arxiv.org/abs/1801.01143v1 |
| http://arxiv.org/pdf/1801.01143v1.pdf | |
| PWC | https://paperswithcode.com/paper/coins-and-logic |
| Repo | |
| Framework | |
Reaching Data Confidentiality and Model Accountability on the CalTrain
| Title | Reaching Data Confidentiality and Model Accountability on the CalTrain |
| Authors | Zhongshu Gu, Hani Jamjoom, Dong Su, Heqing Huang, Jialong Zhang, Tengfei Ma, Dimitrios Pendarakis, Ian Molloy |
| Abstract | Distributed collaborative learning (DCL) paradigms enable building joint machine learning models from distrusting multi-party participants. Data confidentiality is guaranteed by retaining private training data on each participant’s local infrastructure. However, this approach to achieving data confidentiality makes today’s DCL designs fundamentally vulnerable to data poisoning and backdoor attacks. It also limits DCL’s model accountability, which is key to backtracking the responsible “bad” training data instances/contributors. In this paper, we introduce CALTRAIN, a Trusted Execution Environment (TEE) based centralized multi-party collaborative learning system that simultaneously achieves data confidentiality and model accountability. CALTRAIN enforces isolated computation on centrally aggregated training data to guarantee data confidentiality. To support building accountable learning models, we securely maintain the links between training instances and their corresponding contributors. Our evaluation shows that the models generated from CALTRAIN can achieve the same prediction accuracy when compared to the models trained in non-protected environments. We also demonstrate that when malicious training participants tend to implant backdoors during model training, CALTRAIN can accurately and precisely discover the poisoned and mislabeled training data that lead to the runtime mispredictions. |
| Tasks | data poisoning |
| Published | 2018-12-07 |
| URL | http://arxiv.org/abs/1812.03230v1 |
| http://arxiv.org/pdf/1812.03230v1.pdf | |
| PWC | https://paperswithcode.com/paper/reaching-data-confidentiality-and-model |
| Repo | |
| Framework | |
DeepGUM: Learning Deep Robust Regression with a Gaussian-Uniform Mixture Model
| Title | DeepGUM: Learning Deep Robust Regression with a Gaussian-Uniform Mixture Model |
| Authors | Stéphane Lathuilière, Pablo Mesejo, Xavier Alameda-Pineda, Radu Horaud |
| Abstract | In this paper, we address the problem of how to robustly train a ConvNet for regression, or deep robust regression. Traditionally, deep regression employs the L2 loss function, known to be sensitive to outliers, i.e. samples that either lie at an abnormal distance away from the majority of the training samples, or that correspond to wrongly annotated targets. This means that, during back-propagation, outliers may bias the training process due to the high magnitude of their gradient. In this paper, we propose DeepGUM: a deep regression model that is robust to outliers thanks to the use of a Gaussian-uniform mixture model. We derive an optimization algorithm that alternates between the unsupervised detection of outliers using expectation-maximization, and the supervised training with cleaned samples using stochastic gradient descent. DeepGUM is able to adapt to a continuously evolving outlier distribution, avoiding to manually impose any threshold on the proportion of outliers in the training set. Extensive experimental evaluations on four different tasks (facial and fashion landmark detection, age and head pose estimation) lead us to conclude that our novel robust technique provides reliability in the presence of various types of noise and protection against a high percentage of outliers. |
| Tasks | Head Pose Estimation, Pose Estimation |
| Published | 2018-08-28 |
| URL | http://arxiv.org/abs/1808.09211v1 |
| http://arxiv.org/pdf/1808.09211v1.pdf | |
| PWC | https://paperswithcode.com/paper/deepgum-learning-deep-robust-regression-with |
| Repo | |
| Framework | |
Language Model Supervision for Handwriting Recognition Model Adaptation
| Title | Language Model Supervision for Handwriting Recognition Model Adaptation |
| Authors | Chris Tensmeyer, Curtis Wigington, Brian Davis, Seth Stewart, Tony Martinez, William Barrett |
| Abstract | Training state-of-the-art offline handwriting recognition (HWR) models requires large labeled datasets, but unfortunately such datasets are not available in all languages and domains due to the high cost of manual labeling.We address this problem by showing how high resource languages can be leveraged to help train models for low resource languages.We propose a transfer learning methodology where we adapt HWR models trained on a source language to a target language that uses the same writing script.This methodology only requires labeled data in the source language, unlabeled data in the target language, and a language model of the target language. The language model is used in a bootstrapping fashion to refine predictions in the target language for use as ground truth in training the model.Using this approach we demonstrate improved transferability among French, English, and Spanish languages using both historical and modern handwriting datasets. In the best case, transferring with the proposed methodology results in character error rates nearly as good as full supervised training. |
| Tasks | Language Modelling, Transfer Learning |
| Published | 2018-08-04 |
| URL | http://arxiv.org/abs/1808.01423v1 |
| http://arxiv.org/pdf/1808.01423v1.pdf | |
| PWC | https://paperswithcode.com/paper/language-model-supervision-for-handwriting |
| Repo | |
| Framework | |
Harmonizable mixture kernels with variational Fourier features
| Title | Harmonizable mixture kernels with variational Fourier features |
| Authors | Zheyang Shen, Markus Heinonen, Samuel Kaski |
| Abstract | The expressive power of Gaussian processes depends heavily on the choice of kernel. In this work we propose the novel harmonizable mixture kernel (HMK), a family of expressive, interpretable, non-stationary kernels derived from mixture models on the generalized spectral representation. As a theoretically sound treatment of non-stationary kernels, HMK supports harmonizable covariances, a wide subset of kernels including all stationary and many non-stationary covariances. We also propose variational Fourier features, an inter-domain sparse GP inference framework that offers a representative set of ‘inducing frequencies’. We show that harmonizable mixture kernels interpolate between local patterns, and that variational Fourier features offers a robust kernel learning framework for the new kernel family. |
| Tasks | Gaussian Processes |
| Published | 2018-10-10 |
| URL | http://arxiv.org/abs/1810.04416v3 |
| http://arxiv.org/pdf/1810.04416v3.pdf | |
| PWC | https://paperswithcode.com/paper/harmonizable-mixture-kernels-with-variational |
| Repo | |
| Framework | |
Theory of Curriculum Learning, with Convex Loss Functions
| Title | Theory of Curriculum Learning, with Convex Loss Functions |
| Authors | Daphna Weinshall, Dan Amir |
| Abstract | Curriculum Learning - the idea of teaching by gradually exposing the learner to examples in a meaningful order, from easy to hard, has been investigated in the context of machine learning long ago. Although methods based on this concept have been empirically shown to improve performance of several learning algorithms, no theoretical analysis has been provided even for simple cases. To address this shortfall, we start by formulating an ideal definition of difficulty score - the loss of the optimal hypothesis at a given datapoint. We analyze the possible contribution of curriculum learning based on this score in two convex problems - linear regression, and binary classification by hinge loss minimization. We show that in both cases, the expected convergence rate decreases monotonically with the ideal difficulty score, in accordance with earlier empirical results. We also prove that when the ideal difficulty score is fixed, the convergence rate is monotonically increasing with respect to the loss of the current hypothesis at each point. We discuss how these results bring to term two apparently contradicting heuristics: curriculum learning on the one hand, and hard data mining on the other. |
| Tasks | |
| Published | 2018-12-09 |
| URL | http://arxiv.org/abs/1812.03472v1 |
| http://arxiv.org/pdf/1812.03472v1.pdf | |
| PWC | https://paperswithcode.com/paper/theory-of-curriculum-learning-with-convex |
| Repo | |
| Framework | |
A Face Recognition Signature Combining Patch-based Features with Soft Facial Attributes
| Title | A Face Recognition Signature Combining Patch-based Features with Soft Facial Attributes |
| Authors | Lingfeng Zhang, Pengfei Dou, Ioannis A. Kakadiaris |
| Abstract | This paper focuses on improving face recognition performance with a new signature combining implicit facial features with explicit soft facial attributes. This signature has two components: the existing patch-based features and the soft facial attributes. A deep convolutional neural network adapted from state-of-the-art networks is used to learn the soft facial attributes. Then, a signature matcher is introduced that merges the contributions of both patch-based features and the facial attributes. In this matcher, the matching scores computed from patch-based features and the facial attributes are combined to obtain a final matching score. The matcher is also extended so that different weights are assigned to different facial attributes. The proposed signature and matcher have been evaluated with the UR2D system on the UHDB31 and IJB-A datasets. The experimental results indicate that the proposed signature achieve better performance than using only patch-based features. The Rank-1 accuracy is improved significantly by 4% and 0.37% on the two datasets when compared with the UR2D system. |
| Tasks | Face Recognition |
| Published | 2018-03-25 |
| URL | http://arxiv.org/abs/1803.09359v1 |
| http://arxiv.org/pdf/1803.09359v1.pdf | |
| PWC | https://paperswithcode.com/paper/a-face-recognition-signature-combining-patch |
| Repo | |
| Framework | |
Robust Depth Estimation from Auto Bracketed Images
| Title | Robust Depth Estimation from Auto Bracketed Images |
| Authors | Sunghoon Im, Hae-Gon Jeon, In So Kweon |
| Abstract | As demand for advanced photographic applications on hand-held devices grows, these electronics require the capture of high quality depth. However, under low-light conditions, most devices still suffer from low imaging quality and inaccurate depth acquisition. To address the problem, we present a robust depth estimation method from a short burst shot with varied intensity (i.e., Auto Bracketing) or strong noise (i.e., High ISO). We introduce a geometric transformation between flow and depth tailored for burst images, enabling our learning-based multi-view stereo matching to be performed effectively. We then describe our depth estimation pipeline that incorporates the geometric transformation into our residual-flow network. It allows our framework to produce an accurate depth map even with a bracketed image sequence. We demonstrate that our method outperforms state-of-the-art methods for various datasets captured by a smartphone and a DSLR camera. Moreover, we show that the estimated depth is applicable for image quality enhancement and photographic editing. |
| Tasks | Depth Estimation, Stereo Matching, Stereo Matching Hand |
| Published | 2018-03-21 |
| URL | http://arxiv.org/abs/1803.07702v1 |
| http://arxiv.org/pdf/1803.07702v1.pdf | |
| PWC | https://paperswithcode.com/paper/robust-depth-estimation-from-auto-bracketed |
| Repo | |
| Framework | |
Spectral Network Embedding: A Fast and Scalable Method via Sparsity
| Title | Spectral Network Embedding: A Fast and Scalable Method via Sparsity |
| Authors | Jie Zhang, Yan Wang, Jie Tang, Ming Ding |
| Abstract | Network embedding aims to learn low-dimensional representations of nodes in a network, while the network structure and inherent properties are preserved. It has attracted tremendous attention recently due to significant progress in downstream network learning tasks, such as node classification, link prediction, and visualization. However, most existing network embedding methods suffer from the expensive computations due to the large volume of networks. In this paper, we propose a $10\times \sim 100\times$ faster network embedding method, called Progle, by elegantly utilizing the sparsity property of online networks and spectral analysis. In Progle, we first construct a \textit{sparse} proximity matrix and train the network embedding efficiently via sparse matrix decomposition. Then we introduce a network propagation pattern via spectral analysis to incorporate local and global structure information into the embedding. Besides, this model can be generalized to integrate network information into other insufficiently trained embeddings at speed. Benefiting from sparse spectral network embedding, our experiment on four different datasets shows that Progle outperforms or is comparable to state-of-the-art unsupervised comparison approaches—DeepWalk, LINE, node2vec, GraRep, and HOPE, regarding accuracy, while is $10\times$ faster than the fastest word2vec-based method. Finally, we validate the scalability of Progle both in real large-scale networks and multiple scales of synthetic networks. |
| Tasks | Link Prediction, Network Embedding, Node Classification |
| Published | 2018-06-07 |
| URL | http://arxiv.org/abs/1806.02623v2 |
| http://arxiv.org/pdf/1806.02623v2.pdf | |
| PWC | https://paperswithcode.com/paper/spectral-network-embedding-a-fast-and |
| Repo | |
| Framework | |
Fast deep reinforcement learning using online adjustments from the past
| Title | Fast deep reinforcement learning using online adjustments from the past |
| Authors | Steven Hansen, Pablo Sprechmann, Alexander Pritzel, André Barreto, Charles Blundell |
| Abstract | We propose Ephemeral Value Adjusments (EVA): a means of allowing deep reinforcement learning agents to rapidly adapt to experience in their replay buffer. EVA shifts the value predicted by a neural network with an estimate of the value function found by planning over experience tuples from the replay buffer near the current state. EVA combines a number of recent ideas around combining episodic memory-like structures into reinforcement learning agents: slot-based storage, content-based retrieval, and memory-based planning. We show that EVAis performant on a demonstration task and Atari games. |
| Tasks | Atari Games |
| Published | 2018-10-18 |
| URL | http://arxiv.org/abs/1810.08163v1 |
| http://arxiv.org/pdf/1810.08163v1.pdf | |
| PWC | https://paperswithcode.com/paper/fast-deep-reinforcement-learning-using-online |
| Repo | |
| Framework | |
Geometric Scattering on Manifolds
| Title | Geometric Scattering on Manifolds |
| Authors | Michael Perlmutter, Guy Wolf, Matthew Hirn |
| Abstract | The Euclidean scattering transform was introduced nearly a decade ago to improve the mathematical understanding of the success of convolutional neural networks (ConvNets) in image data analysis and other tasks. Inspired by recent interest in geometric deep learning, which aims to generalize ConvNets to manifold and graph-structured domains, we generalize the scattering transform to compact manifolds. Similar to the Euclidean scattering transform, our geometric scattering transform is based on a cascade of designed filters and pointwise nonlinearities, which enables rigorous analysis of the feature extraction provided by scattering layers. Our main focus here is on theoretical understanding of this geometric scattering network, while setting aside implementation aspects, although we remark that application of similar transforms to graph data analysis has been studied recently in related work. Our results establish conditions under which geometric scattering provides localized isometry invariant descriptions of manifold signals, which are also stable to families of diffeomorphisms formulated in intrinsic manifolds terms. These results not only generalize the deformation stability and local roto-translation invariance of Euclidean scattering, but also demonstrate the importance of linking the used filter structures (e.g., in geometric deep learning) to the underlying manifold geometry, or the data geometry it represents. |
| Tasks | |
| Published | 2018-12-15 |
| URL | https://arxiv.org/abs/1812.06968v4 |
| https://arxiv.org/pdf/1812.06968v4.pdf | |
| PWC | https://paperswithcode.com/paper/geometric-scattering-on-manifolds |
| Repo | |
| Framework | |
MergeNet: A Deep Net Architecture for Small Obstacle Discovery
| Title | MergeNet: A Deep Net Architecture for Small Obstacle Discovery |
| Authors | Krishnam Gupta, Syed Ashar Javed, Vineet Gandhi, K. Madhava Krishna |
| Abstract | We present here, a novel network architecture called MergeNet for discovering small obstacles for on-road scenes in the context of autonomous driving. The basis of the architecture rests on the central consideration of training with less amount of data since the physical setup and the annotation process for small obstacles is hard to scale. For making effective use of the limited data, we propose a multi-stage training procedure involving weight-sharing, separate learning of low and high level features from the RGBD input and a refining stage which learns to fuse the obtained complementary features. The model is trained and evaluated on the Lost and Found dataset and is able to achieve state-of-art results with just 135 images in comparison to the 1000 images used by the previous benchmark. Additionally, we also compare our results with recent methods trained on 6000 images and show that our method achieves comparable performance with only 1000 training samples. |
| Tasks | Autonomous Driving |
| Published | 2018-03-17 |
| URL | http://arxiv.org/abs/1803.06508v1 |
| http://arxiv.org/pdf/1803.06508v1.pdf | |
| PWC | https://paperswithcode.com/paper/mergenet-a-deep-net-architecture-for-small |
| Repo | |
| Framework | |
Improved Crowding Distance for NSGA-II
| Title | Improved Crowding Distance for NSGA-II |
| Authors | Xiangxiang Chu, Xinjie Yu |
| Abstract | Non-dominated sorting genetic algorithm II (NSGA-II) does well in dealing with multi-objective problems. When evaluating validity of an algorithm for multi-objective problems, two kinds of indices are often considered simultaneously, i.e. the convergence to Pareto Front and the distribution characteristic. The crowding distance in the standard NSGA-II has the property that solutions within a cubic have the same crowding distance, which has no contribution to the convergence of the algorithm. Actually the closer to the Pareto Front a solution is, the higher priority it should have. In the paper, the crowding distance is redefined while keeping almost all the advantages of the original one. Moreover, the speed of converging to the Pareto Front is faster. Finally, the improvement is proved to be effective by applying it to solve nine Benchmark problems. |
| Tasks | |
| Published | 2018-11-30 |
| URL | http://arxiv.org/abs/1811.12667v1 |
| http://arxiv.org/pdf/1811.12667v1.pdf | |
| PWC | https://paperswithcode.com/paper/improved-crowding-distance-for-nsga-ii |
| Repo | |
| Framework | |
Diffusion Maps for Textual Network Embedding
| Title | Diffusion Maps for Textual Network Embedding |
| Authors | Xinyuan Zhang, Yitong Li, Dinghan Shen, Lawrence Carin |
| Abstract | Textual network embedding leverages rich text information associated with the network to learn low-dimensional vectorial representations of vertices. Rather than using typical natural language processing (NLP) approaches, recent research exploits the relationship of texts on the same edge to graphically embed text. However, these models neglect to measure the complete level of connectivity between any two texts in the graph. We present diffusion maps for textual network embedding (DMTE), integrating global structural information of the graph to capture the semantic relatedness between texts, with a diffusion-convolution operation applied on the text inputs. In addition, a new objective function is designed to efficiently preserve the high-order proximity using the graph diffusion. Experimental results show that the proposed approach outperforms state-of-the-art methods on the vertex-classification and link-prediction tasks. |
| Tasks | Link Prediction, Network Embedding |
| Published | 2018-05-24 |
| URL | http://arxiv.org/abs/1805.09906v2 |
| http://arxiv.org/pdf/1805.09906v2.pdf | |
| PWC | https://paperswithcode.com/paper/diffusion-maps-for-textual-network-embedding |
| Repo | |
| Framework | |
