January 29, 2020

2961 words 14 mins read

Paper Group ANR 708

BCMA-ES II: revisiting Bayesian CMA-ES. A Quotient Space Formulation for Statistical Analysis of Graphical Data. Continuous Integration of Machine Learning Models with ease.ml/ci: Towards a Rigorous Yet Practical Treatment. A Study on Angular Based Embedding Learning for Text-independent Speaker Verification. Mask TextSpotter: An End-to-End Trainab …

BCMA-ES II: revisiting Bayesian CMA-ES


Title	BCMA-ES II: revisiting Bayesian CMA-ES
Authors	Eric Benhamou, David Saltiel, Beatrice Guez, Nicolas Paris
Abstract	This paper revisits the Bayesian CMA-ES and provides updates for normal Wishart. It emphasizes the difference between a normal and normal inverse Wishart prior. After some computation, we prove that the only difference relies surprisingly in the expected covariance. We prove that the expected covariance should be lower in the normal Wishart prior model because of the convexity of the inverse. We present a mixture model that generalizes both normal Wishart and normal inverse Wishart model. We finally present various numerical experiments to compare both methods as well as the generalized method.
Tasks
Published	2019-04-02
URL	http://arxiv.org/abs/1904.01466v2
PDF	http://arxiv.org/pdf/1904.01466v2.pdf
PWC	https://paperswithcode.com/paper/bcma-es-ii-revisiting-bayesian-cma-es
Repo
Framework

A Quotient Space Formulation for Statistical Analysis of Graphical Data


Title	A Quotient Space Formulation for Statistical Analysis of Graphical Data
Authors	Xiaoyang Guo, Anuj Srivastava, Sudeep Sarkar
Abstract	Complex analyses involving multiple, dependent random quantities often lead to graphical models: a set of nodes denoting variables of interest, and corresponding edges denoting statistical interactions between nodes. To develop statistical analyses for graphical data, one needs mathematical representations and metrics for matching and comparing graphs, and other geometrical tools, such as geodesics, means, and covariances, on representation spaces of graphs. This paper utilizes a quotient structure to develop efficient algorithms for computing these quantities, leading to useful statistical tools, including principal component analysis, linear dimension reduction, and analytical statistical modeling. The efficacy of this framework is demonstrated using datasets taken from several problem areas, including alphabets, video summaries, social networks, and biochemical structures.
Tasks	Dimensionality Reduction
Published	2019-09-30
URL	https://arxiv.org/abs/1909.12907v1
PDF	https://arxiv.org/pdf/1909.12907v1.pdf
PWC	https://paperswithcode.com/paper/a-quotient-space-formulation-for-statistical
Repo
Framework

Continuous Integration of Machine Learning Models with ease.ml/ci: Towards a Rigorous Yet Practical Treatment


Title	Continuous Integration of Machine Learning Models with ease.ml/ci: Towards a Rigorous Yet Practical Treatment
Authors	Cedric Renggli, Bojan Karlaš, Bolin Ding, Feng Liu, Kevin Schawinski, Wentao Wu, Ce Zhang
Abstract	Continuous integration is an indispensable step of modern software engineering practices to systematically manage the life cycles of system development. Developing a machine learning model is no difference - it is an engineering process with a life cycle, including design, implementation, tuning, testing, and deployment. However, most, if not all, existing continuous integration engines do not support machine learning as first-class citizens. In this paper, we present ease.ml/ci, to our best knowledge, the first continuous integration system for machine learning. The challenge of building ease.ml/ci is to provide rigorous guarantees, e.g., single accuracy point error tolerance with 0.999 reliability, with a practical amount of labeling effort, e.g., 2K labels per test. We design a domain specific language that allows users to specify integration conditions with reliability constraints, and develop simple novel optimizations that can lower the number of labels required by up to two orders of magnitude for test conditions popularly used in real production systems.
Tasks
Published	2019-03-01
URL	http://arxiv.org/abs/1903.00278v1
PDF	http://arxiv.org/pdf/1903.00278v1.pdf
PWC	https://paperswithcode.com/paper/continuous-integration-of-machine-learning
Repo
Framework

A Study on Angular Based Embedding Learning for Text-independent Speaker Verification


Title	A Study on Angular Based Embedding Learning for Text-independent Speaker Verification
Authors	Zhiyong Chen, Zongze Ren, Shugong Xu
Abstract	Learning a good speaker embedding is important for many automatic speaker recognition tasks, including verification, identification and diarization. The embeddings learned by softmax are not discriminative enough for open-set verification tasks. Angular based embedding learning target can achieve such discriminativeness by optimizing angular distance and adding margin penalty. We apply several different popular angular margin embedding learning strategies in this work and explicitly compare their performance on Voxceleb speaker recognition dataset. Observing the fact that encouraging inter-class separability is important when applying angular based embedding learning, we propose an exclusive inter-class regularization as a complement for angular based loss. We verify the effectiveness of these methods for learning a discriminative embedding space on ASV task with several experiments. These methods together, we manage to achieve an impressive result with 16.5% improvement on equal error rate (EER) and 18.2% improvement on minimum detection cost function comparing with baseline softmax systems.
Tasks	Speaker Recognition, Speaker Verification, Text-Independent Speaker Verification
Published	2019-08-12
URL	https://arxiv.org/abs/1908.03990v1
PDF	https://arxiv.org/pdf/1908.03990v1.pdf
PWC	https://paperswithcode.com/paper/a-study-on-angular-based-embedding-learning
Repo
Framework

Mask TextSpotter: An End-to-End Trainable Neural Network for Spotting Text with Arbitrary Shapes


Title	Mask TextSpotter: An End-to-End Trainable Neural Network for Spotting Text with Arbitrary Shapes
Authors	Minghui Liao, Pengyuan Lyu, Minghang He, Cong Yao, Wenhao Wu, Xiang Bai
Abstract	Unifying text detection and text recognition in an end-to-end training fashion has become a new trend for reading text in the wild, as these two tasks are highly relevant and complementary. In this paper, we investigate the problem of scene text spotting, which aims at simultaneous text detection and recognition in natural images. An end-to-end trainable neural network named as Mask TextSpotter is presented. Different from the previous text spotters that follow the pipeline consisting of a proposal generation network and a sequence-to-sequence recognition network, Mask TextSpotter enjoys a simple and smooth end-to-end learning procedure, in which both detection and recognition can be achieved directly from two-dimensional space via semantic segmentation. Further, a spatial attention module is proposed to enhance the performance and universality. Benefiting from the proposed two-dimensional representation on both detection and recognition, it easily handles text instances of irregular shapes, for instance, curved text. We evaluate it on four English datasets and one multi-language dataset, achieving consistently superior performance over state-of-the-art methods in both detection and end-to-end text recognition tasks. Moreover, we further investigate the recognition module of our method separately, which significantly outperforms state-of-the-art methods on both regular and irregular text datasets for scene text recognition.
Tasks	Scene Text Recognition, Semantic Segmentation, Text Spotting
Published	2019-08-22
URL	https://arxiv.org/abs/1908.08207v1
PDF	https://arxiv.org/pdf/1908.08207v1.pdf
PWC	https://paperswithcode.com/paper/mask-textspotter-an-end-to-end-trainable-2
Repo
Framework

A Note on KL-UCB+ Policy for the Stochastic Bandit


Title	A Note on KL-UCB+ Policy for the Stochastic Bandit
Authors	Junya Honda
Abstract	A classic setting of the stochastic K-armed bandit problem is considered in this note. In this problem it has been known that KL-UCB policy achieves the asymptotically optimal regret bound and KL-UCB+ policy empirically performs better than the KL-UCB policy although the regret bound for the original form of the KL-UCB+ policy has been unknown. This note demonstrates that a simple proof of the asymptotic optimality of the KL-UCB+ policy can be given by the same technique as those used for analyses of other known policies.
Tasks
Published	2019-03-19
URL	http://arxiv.org/abs/1903.07839v2
PDF	http://arxiv.org/pdf/1903.07839v2.pdf
PWC	https://paperswithcode.com/paper/a-note-on-kl-ucb-policy-for-the-stochastic
Repo
Framework

Samples Are Useful? Not Always: denoising policy gradient updates using variance explained


Title	Samples Are Useful? Not Always: denoising policy gradient updates using variance explained
Authors	Yannis Flet-Berliac, Philippe Preux
Abstract	Policy gradient algorithms in reinforcement learning optimize the policy directly and rely on efficiently sampling an environment. However, while most sampling procedures are based solely on sampling the agent’s policy, other measures directly accessible through these algorithms could be used to improve sampling before each policy update. Following this line of thoughts, we propose the use of SAUNA, a method where transitions are rejected from the gradient updates if they do not meet a particular criterion, and kept otherwise. This criterion, the fraction of variance explained $\mathcal{V}^{ex}$, is a measure of the discrepancy between a model and actual samples. In this work, $\mathcal{V}^{ex}$ is used to evaluate the impact each transition will have on learning: this criterion refines sampling and improves the policy gradient algorithm. In this paper: (a) We introduce and explore $\mathcal{V}^{ex}$, the criterion used for denoising policy gradient updates. (b) We conduct experiments across a variety of benchmark environments, including standard continuous control problems. Our results show better performance with SAUNA. (c) We investigate why $\mathcal{V}^{ex}$ provides a reliable assessment for the selection of samples that will positively impact learning. (d) We show how this criterion can work as a dynamic tool to adjust the ratio between exploration and exploitation.
Tasks	Continuous Control, Denoising
Published	2019-04-08
URL	https://arxiv.org/abs/1904.04025v3
PDF	https://arxiv.org/pdf/1904.04025v3.pdf
PWC	https://paperswithcode.com/paper/samples-are-not-all-useful-denoising-policy
Repo
Framework

An Improved Convolutional Neural Network System for Automatically Detecting Rebar in GPR Data


Title	An Improved Convolutional Neural Network System for Automatically Detecting Rebar in GPR Data
Authors	Zhongming Xiang, Abbas Rashidi, Ge, Ou
Abstract	As a mature technology, Ground Penetration Radar (GPR) is now widely employed in detecting rebar and other embedded elements in concrete structures. Manually recognizing rebar from GPR data is a time-consuming and error-prone procedure. Although there are several approaches to automatically detect rebar, it is still challenging to find a high resolution and efficient method for different rebar arrangements, especially for closely spaced rebar meshes. As an improved Convolution Neural Network (CNN), AlexNet shows superiority over traditional methods in image recognition domain. Thus, this paper introduces AlexNet as an alternative solution for automatically detecting rebar within GPR data. In order to show the efficiency of the proposed approach, a traditional CNN is built as the comparative option. Moreover, this research evaluates the impacts of different rebar arrangements and different window sizes on the accuracy of results. The results revealed that: (1) AlexNet outperforms the traditional CNN approach, and its superiority is more notable when the rebar meshes are densely distributed; (2) the detection accuracy significantly varies with changing the size of splitting window, and a proper window should contain enough information about rebar; (3) uniformly and sparsely distributed rebar meshes are more recognizable than densely or unevenly distributed items, due to lower chances of signal interferences.
Tasks
Published	2019-07-23
URL	https://arxiv.org/abs/1907.09997v1
PDF	https://arxiv.org/pdf/1907.09997v1.pdf
PWC	https://paperswithcode.com/paper/an-improved-convolutional-neural-network
Repo
Framework

Understanding ML driven HPC: Applications and Infrastructure


Title	Understanding ML driven HPC: Applications and Infrastructure
Authors	Geoffrey Fox, Shantenu Jha
Abstract	We recently outlined the vision of “Learning Everywhere” which captures the possibility and impact of how learning methods and traditional HPC methods can be coupled together. A primary driver of such coupling is the promise that Machine Learning (ML) will give major performance improvements for traditional HPC simulations. Motivated by this potential, the ML around HPC class of integration is of particular significance. In a related follow-up paper, we provided an initial taxonomy for integrating learning around HPC methods. In this paper, which is part of the Learning Everywhere series, we discuss “how” learning methods and HPC simulations are being integrated to enhance effective performance of computations. This paper identifies several modes — substitution, assimilation, and control, in which learning methods integrate with HPC simulations and provide representative applications in each mode. This paper discusses some open research questions and we hope will motivate and clear the ground for MLaroundHPC benchmarks.
Tasks
Published	2019-09-05
URL	https://arxiv.org/abs/1909.02363v1
PDF	https://arxiv.org/pdf/1909.02363v1.pdf
PWC	https://paperswithcode.com/paper/understanding-ml-driven-hpc-applications-and
Repo
Framework

Deep Convolutional Compression for Massive MIMO CSI Feedback


Title	Deep Convolutional Compression for Massive MIMO CSI Feedback
Authors	Qianqian Yang, Mahdi Boloursaz Mashhadi, Deniz Gündüz
Abstract	Coded caching provides significant gains over conventional uncoded caching by creating multicasting opportunities among distinct requests. Massive multiple-input multiple-output (MIMO) systems require downlink channel state information (CSI) at the base station (BS) to better utilize the available spatial diversity and multiplexing gains. However, in a frequency division duplex (FDD) massive MIMO system, the huge CSI feedback overhead becomes restrictive and degrades the overall spectral efficiency. In this paper, we propose a deep learning based channel state matrix compression scheme, called DeepCMC, composed of convolutional layers followed by quantization and entropy coding blocks. In comparison with previous works, the main contributions of DeepCMC are two-fold: i) DeepCMC is fully convolutional, and it can be used in a wide range of scenarios with various numbers of sub-channels and transmit antennas; ii) DeepCMC includes quantization and entropy coding blocks and minimizes a cost function that accounts for both the rate of compression and the reconstruction quality of the channel matrix at the BS. Simulation results demonstrate that DeepCMC significantly outperforms the state of the art compression schemes in terms of the reconstruction quality of the channel state matrix for the same compression rate, measured in bits per channel dimension.
Tasks	Quantization
Published	2019-07-02
URL	https://arxiv.org/abs/1907.02942v1
PDF	https://arxiv.org/pdf/1907.02942v1.pdf
PWC	https://paperswithcode.com/paper/deep-convolutional-compression-for-massive
Repo
Framework

Sampling Acquisition Functions for Batch Bayesian Optimization


Title	Sampling Acquisition Functions for Batch Bayesian Optimization
Authors	Alessandro De Palma, Celestine Mendler-Dünner, Thomas Parnell, Andreea Anghel, Haralampos Pozidis
Abstract	We present Acquisition Thompson Sampling (ATS), a novel technique for batch Bayesian Optimization (BO) based on the idea of sampling multiple acquisition functions from a stochastic process. We define this process through the dependency of the acquisition functions on a set of model hyper-parameters. ATS is conceptually simple, straightforward to implement and, unlike other batch BO methods, it can be employed to parallelize any sequential acquisition function or to make existing parallel methods scale further. We present experiments on a variety of benchmark functions and on the hyper-parameter optimization of a popular gradient boosting tree algorithm. These demonstrate the advantages of ATS with respect to classical parallel Thompson Sampling for BO, its competitiveness with two state-of-the-art batch BO methods, and its effectiveness if applied to existing parallel BO algorithms.
Tasks
Published	2019-03-22
URL	https://arxiv.org/abs/1903.09434v2
PDF	https://arxiv.org/pdf/1903.09434v2.pdf
PWC	https://paperswithcode.com/paper/sampling-acquisition-functions-for-batch
Repo
Framework

Unsupervised High-Resolution Depth Learning From Videos With Dual Networks


Title	Unsupervised High-Resolution Depth Learning From Videos With Dual Networks
Authors	Junsheng Zhou, Yuwang Wang, Kaihuai Qin, Wenjun Zeng
Abstract	Unsupervised depth learning takes the appearance difference between a target view and a view synthesized from its adjacent frame as supervisory signal. Since the supervisory signal only comes from images themselves, the resolution of training data significantly impacts the performance. High-resolution images contain more fine-grained details and provide more accurate supervisory signal. However, due to the limitation of memory and computation power, the original images are typically down-sampled during training, which suffers heavy loss of details and disparity accuracy. In order to fully explore the information contained in high-resolution data, we propose a simple yet effective dual networks architecture, which can directly take high-resolution images as input and generate high-resolution and high-accuracy depth map efficiently. We also propose a Self-assembled Attention (SA-Attention) module to handle low-texture region. The evaluation on the benchmark KITTI and Make3D datasets demonstrates that our method achieves state-of-the-art results in the monocular depth estimation task.
Tasks	Depth Estimation, Monocular Depth Estimation
Published	2019-10-20
URL	https://arxiv.org/abs/1910.08897v1
PDF	https://arxiv.org/pdf/1910.08897v1.pdf
PWC	https://paperswithcode.com/paper/unsupervised-high-resolution-depth-learning
Repo
Framework

APIR-Net: Autocalibrated Parallel Imaging Reconstruction using a Neural Network


Title	APIR-Net: Autocalibrated Parallel Imaging Reconstruction using a Neural Network
Authors	Chaoping Zhang, Florian Dubost, Marleen de Bruijne, Stefan Klein, Dirk H. J. Poot
Abstract	Deep learning has been successfully demonstrated in MRI reconstruction of accelerated acquisitions. However, its dependence on representative training data limits the application across different contrasts, anatomies, or image sizes. To address this limitation, we propose an unsupervised, auto-calibrated k-space completion method, based on a uniquely designed neural network that reconstructs the full k-space from an undersampled k-space, exploiting the redundancy among the multiple channels in the receive coil in a parallel imaging acquisition. To achieve this, contrary to common convolutional network approaches, the proposed network has a decreasing number of feature maps of constant size. In contrast to conventional parallel imaging methods such as GRAPPA that estimate the prediction kernel from the fully sampled autocalibration signals in a linear way, our method is able to learn nonlinear relations between sampled and unsampled positions in k-space. The proposed method was compared to the start-of-the-art ESPIRiT and RAKI methods in terms of noise amplification and visual image quality in both phantom and in-vivo experiments. The experiments indicate that APIR-Net provides a promising alternative to the conventional parallel imaging methods, and results in improved image quality especially for low SNR acquisitions.
Tasks
Published	2019-09-19
URL	https://arxiv.org/abs/1909.09006v1
PDF	https://arxiv.org/pdf/1909.09006v1.pdf
PWC	https://paperswithcode.com/paper/apir-net-autocalibrated-parallel-imaging
Repo
Framework

Simion Zoo: A Workbench for Distributed Experimentation with Reinforcement Learning for Continuous Control Tasks


Title	Simion Zoo: A Workbench for Distributed Experimentation with Reinforcement Learning for Continuous Control Tasks
Authors	Borja Fernandez-Gauna, Manuel Graña, Roland S. Zimmermann
Abstract	We present Simion Zoo, a Reinforcement Learning (RL) workbench that provides a complete set of tools to design, run, and analyze the results,both statistically and visually, of RL control applications. The main features that set apart Simion Zoo from similar software packages are its easy-to-use GUI, its support for distributed execution including deployment over graphics processing units (GPUs) , and the possibility to explore concurrently the RL metaparameter space, which is key to successful RL experimentation.
Tasks	Continuous Control
Published	2019-04-16
URL	http://arxiv.org/abs/1904.07817v1
PDF	http://arxiv.org/pdf/1904.07817v1.pdf
PWC	https://paperswithcode.com/paper/simion-zoo-a-workbench-for-distributed
Repo
Framework

Generalization through Memorization: Nearest Neighbor Language Models


Title	Generalization through Memorization: Nearest Neighbor Language Models
Authors	Urvashi Khandelwal, Omer Levy, Dan Jurafsky, Luke Zettlemoyer, Mike Lewis
Abstract	We introduce $k$NN-LMs, which extend a pre-trained neural language model (LM) by linearly interpolating it with a $k$-nearest neighbors ($k$NN) model. The nearest neighbors are computed according to distance in the pre-trained LM embedding space, and can be drawn from any text collection, including the original LM training data. Applying this augmentation to a strong Wikitext-103 LM, with neighbors drawn from the original training set, our $k$NN-LM achieves a new state-of-the-art perplexity of 15.79 - a 2.9 point improvement with no additional training. We also show that this approach has implications for efficiently scaling up to larger training sets and allows for effective domain adaptation, by simply varying the nearest neighbor datastore, again without further training. Qualitatively, the model is particularly helpful in predicting rare patterns, such as factual knowledge. Together, these results strongly suggest that learning similarity between sequences of text is easier than predicting the next word, and that nearest neighbor search is an effective approach for language modeling in the long tail.
Tasks	Domain Adaptation, Language Modelling
Published	2019-11-01
URL	https://arxiv.org/abs/1911.00172v2
PDF	https://arxiv.org/pdf/1911.00172v2.pdf
PWC	https://paperswithcode.com/paper/generalization-through-memorization-nearest
Repo
Framework