October 17, 2019

2910 words 14 mins read

Paper Group ANR 836

Universal Statistics of Fisher Information in Deep Neural Networks: Mean Field Approach. The parallel texts of books translations in the quality evaluation of basic models and algorithms for the similarity of symbol strings. GADAM: Genetic-Evolutionary ADAM for Deep Neural Network Optimization. Designing quantum experiments with a genetic algorithm …

Universal Statistics of Fisher Information in Deep Neural Networks: Mean Field Approach


Title	Universal Statistics of Fisher Information in Deep Neural Networks: Mean Field Approach
Authors	Ryo Karakida, Shotaro Akaho, Shun-ichi Amari
Abstract	The Fisher information matrix (FIM) is a fundamental quantity to represent the characteristics of a stochastic model, including deep neural networks (DNNs). The present study reveals novel statistics of FIM that are universal among a wide class of DNNs. To this end, we use random weights and large width limits, which enables us to utilize mean field theories. We investigate the asymptotic statistics of the FIM’s eigenvalues and reveal that most of them are close to zero while the maximum eigenvalue takes a huge value. Because the landscape of the parameter space is defined by the FIM, it is locally flat in most dimensions, but strongly distorted in others. Moreover, we demonstrate the potential usage of the derived statistics in learning strategies. First, small eigenvalues that induce flatness can be connected to a norm-based capacity measure of generalization ability. Second, the maximum eigenvalue that induces the distortion enables us to quantitatively estimate an appropriately sized learning rate for gradient methods to converge.
Tasks
Published	2018-06-04
URL	https://arxiv.org/abs/1806.01316v3
PDF	https://arxiv.org/pdf/1806.01316v3.pdf
PWC	https://paperswithcode.com/paper/universal-statistics-of-fisher-information-in
Repo
Framework

The parallel texts of books translations in the quality evaluation of basic models and algorithms for the similarity of symbol strings


Title	The parallel texts of books translations in the quality evaluation of basic models and algorithms for the similarity of symbol strings
Authors	Sergej V. Znamenskij
Abstract	This numeric evaluation of string metric accuracy is based on the following idea: taking the paragraph of text in one language sort all paragraphs of the document in other language by similarity with given paragraph string and consider place of the right translation as the value of the evaluation score. Such a search of proper translation provides an objective and reproducible quality assessment for known similarity metrics and shows the most accurate ones.
Tasks
Published	2018-05-24
URL	http://arxiv.org/abs/1805.09776v3
PDF	http://arxiv.org/pdf/1805.09776v3.pdf
PWC	https://paperswithcode.com/paper/the-parallel-texts-of-books-translations-in
Repo
Framework

GADAM: Genetic-Evolutionary ADAM for Deep Neural Network Optimization


Title	GADAM: Genetic-Evolutionary ADAM for Deep Neural Network Optimization
Authors	Jiawei Zhang, Fisher B. Gouza
Abstract	Deep neural network learning can be formulated as a non-convex optimization problem. Existing optimization algorithms, e.g., Adam, can learn the models fast, but may get stuck in local optima easily. In this paper, we introduce a novel optimization algorithm, namely GADAM (Genetic-Evolutionary Adam). GADAM learns deep neural network models based on a number of unit models generations by generations: it trains the unit models with Adam, and evolves them to the new generations with genetic algorithm. We will show that GADAM can effectively jump out of the local optima in the learning process to obtain better solutions, and prove that GADAM can also achieve a very fast convergence. Extensive experiments have been done on various benchmark datasets, and the learning results will demonstrate the effectiveness and efficiency of the GADAM algorithm.
Tasks
Published	2018-05-19
URL	http://arxiv.org/abs/1805.07500v2
PDF	http://arxiv.org/pdf/1805.07500v2.pdf
PWC	https://paperswithcode.com/paper/gadam-genetic-evolutionary-adam-for-deep
Repo
Framework

Designing quantum experiments with a genetic algorithm


Title	Designing quantum experiments with a genetic algorithm
Authors	Rosanna Nichols, Lana Mineh, Jesús Rubio, Jonathan C. F. Matthews, Paul A. Knott
Abstract	We introduce a genetic algorithm that designs quantum optics experiments for engineering quantum states with specific properties. Our algorithm is powerful and flexible, and can easily be modified to find methods of engineering states for a range of applications. Here we focus on quantum metrology. First, we consider the noise-free case, and use the algorithm to find quantum states with a large quantum Fisher information (QFI). We find methods, which only involve experimental elements that are available with current or near-future technology, for engineering quantum states with up to a 100-fold improvement over the best classical state, and a 20-fold improvement over the optimal Gaussian state. Such states are a superposition of the vacuum with a large number of photons (around $80$), and can hence be seen as Schr"odinger-cat-like states. We then apply the two most dominant noise sources in our setting – photon loss and imperfect heralding – and use the algorithm to find quantum states that still improve over the optimal Gaussian state with realistic levels of noise. This will open up experimental and technological work in using exotic non-Gaussian states for quantum-enhanced phase measurements. Finally, we use the Bayesian mean square error to look beyond the regime of validity of the QFI, finding quantum states with precision enhancements over the alternatives even when the experiment operates in the regime of limited data.
Tasks
Published	2018-12-03
URL	https://arxiv.org/abs/1812.01032v2
PDF	https://arxiv.org/pdf/1812.01032v2.pdf
PWC	https://paperswithcode.com/paper/designing-quantum-experiments-with-a-genetic
Repo
Framework

A3Net: Adversarial-and-Attention Network for Machine Reading Comprehension


Title	A3Net: Adversarial-and-Attention Network for Machine Reading Comprehension
Authors	Jiuniu Wang, Xingyu Fu, Guangluan Xu, Yirong Wu, Ziyan Chen, Yang Wei, Li Jin
Abstract	In this paper, we introduce Adversarial-and-attention Network (A3Net) for Machine Reading Comprehension. This model extends existing approaches from two perspectives. First, adversarial training is applied to several target variables within the model, rather than only to the inputs or embeddings. We control the norm of adversarial perturbations according to the norm of original target variables, so that we can jointly add perturbations to several target variables during training. As an effective regularization method, adversarial training improves robustness and generalization of our model. Second, we propose a multi-layer attention network utilizing three kinds of high-efficiency attention mechanisms. Multi-layer attention conducts interaction between question and passage within each layer, which contributes to reasonable representation and understanding of the model. Combining these two contributions, we enhance the diversity of dataset and the information extracting ability of the model at the same time. Meanwhile, we construct A3Net for the WebQA dataset. Results show that our model outperforms the state-of-the-art models (improving Fuzzy Score from 73.50% to 77.0%).
Tasks	Machine Reading Comprehension, Reading Comprehension
Published	2018-09-03
URL	http://arxiv.org/abs/1809.00676v1
PDF	http://arxiv.org/pdf/1809.00676v1.pdf
PWC	https://paperswithcode.com/paper/a3net-adversarial-and-attention-network-for
Repo
Framework

Dynamic Block Matching to assess the longitudinal component of the dense motion field of the carotid artery wall in B-mode ultrasound sequences - Association with coronary artery disease


Title	Dynamic Block Matching to assess the longitudinal component of the dense motion field of the carotid artery wall in B-mode ultrasound sequences - Association with coronary artery disease
Authors	Guillaume Zahnd, Kozue Saito, Kazuyuki Nagatsuka, Yoshito Otake, Yoshinobu Sato
Abstract	Purpose: The motion of the common carotid artery tissue layers along the vessel axis during the cardiac cycle, observed in ultrasound imaging, is associated with the presence of established cardiovascular risk factors. However, the vast majority of the methods are based on the tracking of a single point, thus failing to capture the overall motion of the entire arterial wall. The aim of this work is to introduce a motion tracking framework able to simultaneously extract the trajectory of a large collection of points spanning the entire exploitable width of the image. Method: The longitudinal motion, which is the main focus of the present work, is determined in two steps. First, a series of independent block matching operations are carried out for all the tracked points. Then, an original dynamic-programming approach is exploited to regularize the collection of similarity maps and estimate the globally optimal motion over the entire vessel wall. Sixty-two atherosclerotic participants at high cardiovascular risk were involved in this study. Results: A dense displacement field, describing the longitudinal motion of the carotid far wall over time, was extracted. For each cine-loop, the method was evaluated against manual reference tracings performed on three local points, with an average absolute error of 150+/-163 um. A strong correlation was found between motion inhomogeneity and the presence of coronary artery disease (beta-coefficient=0.586, p=0.003). Conclusions: To the best of our knowledge, this is the first time that a method is specifically proposed to assess the dense motion field of the carotid far wall. This approach has potential to evaluate the (in)homogeneity of the wall dynamics. The proposed method has promising performances to improve the analysis of arterial longitudinal motion and the understanding of the underlying patho-physiological parameters.
Tasks
Published	2018-09-06
URL	http://arxiv.org/abs/1809.01924v2
PDF	http://arxiv.org/pdf/1809.01924v2.pdf
PWC	https://paperswithcode.com/paper/dynamic-block-matching-to-assess-the
Repo
Framework

FADL:Federated-Autonomous Deep Learning for Distributed Electronic Health Record


Title	FADL:Federated-Autonomous Deep Learning for Distributed Electronic Health Record
Authors	Dianbo Liu, Timothy Miller, Raheel Sayeed, Kenneth D. Mandl
Abstract	Electronic health record (EHR) data is collected by individual institutions and often stored across locations in silos. Getting access to these data is difficult and slow due to security, privacy, regulatory, and operational issues. We show, using ICU data from 58 different hospitals, that machine learning models to predict patient mortality can be trained efficiently without moving health data out of their silos using a distributed machine learning strategy. We propose a new method, called Federated-Autonomous Deep Learning (FADL) that trains part of the model using all data sources in a distributed manner and other parts using data from specific data sources. We observed that FADL outperforms traditional federated learning strategy and conclude that balance between global and local training is an important factor to consider when design distributed machine learning methods , especially in healthcare.
Tasks
Published	2018-11-28
URL	http://arxiv.org/abs/1811.11400v2
PDF	http://arxiv.org/pdf/1811.11400v2.pdf
PWC	https://paperswithcode.com/paper/fadlfederated-autonomous-deep-learning-for
Repo
Framework

Anticipating contingengies in power grids using fast neural net screening


Title	Anticipating contingengies in power grids using fast neural net screening
Authors	Benjamin Donnot, Isabelle Guyon, Marc Schoenauer, Antoine Marot, Patrick Panciatici
Abstract	We address the problem of maintaining high voltage power transmission networks in security at all time. This requires that power flowing through all lines remain below a certain nominal thermal limit above which lines might melt, break or cause other damages. Current practices include enforcing the deterministic “N-1” reliability criterion, namely anticipating exceeding of thermal limit for any eventual single line disconnection (whatever its cause may be) by running a slow, but accurate, physical grid simulator. New conceptual frameworks are calling for a probabilistic risk based security criterion and are in need of new methods to assess the risk. To tackle this difficult assessment, we address in this paper the problem of rapidly ranking higher order contingencies including all pairs of line disconnections, to better prioritize simulations. We present a novel method based on neural networks, which ranks “N-1” and “N-2” contingencies in decreasing order of presumed severity. We demonstrate on a classical benchmark problem that the residual risk of contingencies decreases dramatically compared to considering solely all “N-1” cases, at no additional computational cost. We evaluate that our method scales up to power grids of the size of the French high voltage power grid (over 1000 power lines).
Tasks
Published	2018-05-03
URL	http://arxiv.org/abs/1805.02608v1
PDF	http://arxiv.org/pdf/1805.02608v1.pdf
PWC	https://paperswithcode.com/paper/anticipating-contingengies-in-power-grids
Repo
Framework

The Profiling Machine: Active Generalization over Knowledge


Title	The Profiling Machine: Active Generalization over Knowledge
Authors	Filip Ilievski, Eduard Hovy, Qizhe Xie, Piek Vossen
Abstract	The human mind is a powerful multifunctional knowledge storage and management system that performs generalization, type inference, anomaly detection, stereotyping, and other tasks. A dynamic KR system that appropriately profiles over sparse inputs to provide complete expectations for unknown facets can help with all these tasks. In this paper, we introduce the task of profiling, inspired by theories and findings in social psychology about the potential of profiles for reasoning and information processing. We describe two generic state-of-the-art neural architectures that can be easily instantiated as profiling machines to generate expectations and applied to any kind of knowledge to fill gaps. We evaluate these methods against Wikidata and crowd expectations, and compare the results to gain insight in the nature of knowledge captured by various profiling methods. We make all code and data available to facilitate future research.
Tasks	Anomaly Detection
Published	2018-10-01
URL	http://arxiv.org/abs/1810.00782v1
PDF	http://arxiv.org/pdf/1810.00782v1.pdf
PWC	https://paperswithcode.com/paper/the-profiling-machine-active-generalization
Repo
Framework

Empirical bounds for functions with weak interactions


Title	Empirical bounds for functions with weak interactions
Authors	Andreas Maurer, Massimiliano Pontil
Abstract	We provide sharp empirical estimates of expectation, variance and normal approximation for a class of statistics whose variation in any argument does not change too much when another argument is modified. Examples of such weak interactions are furnished by U- and V-statistics, Lipschitz L-statistics and various error functionals of L2-regularized algorithms and Gibbs algorithms.
Tasks
Published	2018-03-11
URL	http://arxiv.org/abs/1803.03934v1
PDF	http://arxiv.org/pdf/1803.03934v1.pdf
PWC	https://paperswithcode.com/paper/empirical-bounds-for-functions-with-weak
Repo
Framework

Extendable Neural Matrix Completion


Title	Extendable Neural Matrix Completion
Authors	Duc Minh Nguyen, Evaggelia Tsiligianni, Nikos Deligiannis
Abstract	Matrix completion is one of the key problems in signal processing and machine learning, with applications ranging from image pro- cessing and data gathering to classification and recommender sys- tems. Recently, deep neural networks have been proposed as la- tent factor models for matrix completion and have achieved state- of-the-art performance. Nevertheless, a major problem with existing neural-network-based models is their limited capabilities to extend to samples unavailable at the training stage. In this paper, we propose a deep two-branch neural network model for matrix completion. The proposed model not only inherits the predictive power of neural net- works, but is also capable of extending to partially observed samples outside the training set, without the need of retraining or fine-tuning. Experimental studies on popular movie rating datasets prove the ef- fectiveness of our model compared to the state of the art, in terms of both accuracy and extendability.
Tasks	Matrix Completion
Published	2018-05-13
URL	http://arxiv.org/abs/1805.04912v1
PDF	http://arxiv.org/pdf/1805.04912v1.pdf
PWC	https://paperswithcode.com/paper/extendable-neural-matrix-completion
Repo
Framework

Analysis of Atomistic Representations Using Weighted Skip-Connections


Title	Analysis of Atomistic Representations Using Weighted Skip-Connections
Authors	Kim A. Nicoli, Pan Kessel, Michael Gastegger, Kristof T. Schütt
Abstract	In this work, we extend the SchNet architecture by using weighted skip connections to assemble the final representation. This enables us to study the relative importance of each interaction block for property prediction. We demonstrate on both the QM9 and MD17 dataset that their relative weighting depends strongly on the chemical composition and configurational degrees of freedom of the molecules which opens the path towards a more detailed understanding of machine learning models for molecules.
Tasks	MD17 dataset
Published	2018-10-23
URL	http://arxiv.org/abs/1810.09751v2
PDF	http://arxiv.org/pdf/1810.09751v2.pdf
PWC	https://paperswithcode.com/paper/analysis-of-atomistic-representations-using
Repo
Framework

Efficient No-Reference Quality Assessment and Classification Model for Contrast Distorted Images


Title	Efficient No-Reference Quality Assessment and Classification Model for Contrast Distorted Images
Authors	Hossein Ziaei Nafchi, Mohamed Cheriet
Abstract	In this paper, an efficient Minkowski Distance based Metric (MDM) for no-reference (NR) quality assessment of contrast distorted images is proposed. It is shown that higher orders of Minkowski distance and entropy provide accurate quality prediction for the contrast distorted images. The proposed metric performs predictions by extracting only three features from the distorted images followed by a regression analysis. Furthermore, the proposed features are able to classify type of the contrast distorted images with a high accuracy. Experimental results on four datasets CSIQ, TID2013, CCID2014, and SIQAD show that the proposed metric with a very low complexity provides better quality predictions than the state-of-the-art NR metrics. The MATLAB source code of the proposed metric is available to public at http://www.synchromedia.ca/system/files/MDM.zip.
Tasks
Published	2018-04-07
URL	http://arxiv.org/abs/1804.02554v1
PDF	http://arxiv.org/pdf/1804.02554v1.pdf
PWC	https://paperswithcode.com/paper/efficient-no-reference-quality-assessment-and
Repo
Framework

Multi-robot Dubins Coverage with Autonomous Surface Vehicles


Title	Multi-robot Dubins Coverage with Autonomous Surface Vehicles
Authors	Nare Karapetyan, Jason Moulton, Jeremy S. Lewis, Alberto Quattrini Li, Jason M. O’Kane, Ioannis Rekleitis
Abstract	In large scale coverage operations, such as marine exploration or aerial monitoring, single robot approaches are not ideal, as they may take too long to cover a large area. In such scenarios, multi-robot approaches are preferable. Furthermore, several real world vehicles are non-holonomic, but can be modeled using Dubins vehicle kinematics. This paper focuses on environmental monitoring of aquatic environments using Autonomous Surface Vehicles (ASVs). In particular, we propose a novel approach for solving the problem of complete coverage of a known environment by a multi-robot team consisting of Dubins vehicles. It is worth noting that both multi-robot coverage and Dubins vehicle coverage are NP-complete problems. As such, we present two heuristics methods based on a variant of the traveling salesman problem – k-TSP – formulation and clustering algorithms that efficiently solve the problem. The proposed methods are tested both in simulations to assess their scalability and with a team of ASVs operating on a lake to ensure their applicability in real world.
Tasks
Published	2018-08-07
URL	http://arxiv.org/abs/1808.02552v1
PDF	http://arxiv.org/pdf/1808.02552v1.pdf
PWC	https://paperswithcode.com/paper/multi-robot-dubins-coverage-with-autonomous
Repo
Framework

Median Binary-Connect Method and a Binary Convolutional Neural Nework for Word Recognition


Title	Median Binary-Connect Method and a Binary Convolutional Neural Nework for Word Recognition
Authors	Spencer Sheen, Jiancheng Lyu
Abstract	We propose and study a new projection formula for training binary weight convolutional neural networks. The projection formula measures the error in approximating a full precision (32 bit) vector by a 1-bit vector in the l_1 norm instead of the standard l_2 norm. The l_1 projector is in closed analytical form and involves a median computation instead of an arithmatic average in the l_2 projector. Experiments on 10 keywords classification show that the l_1 (median) BinaryConnect (BC) method outperforms the regular BC, regardless of cold or warm start. The binary network trained by median BC and a recent blending technique reaches test accuracy 92.4%, which is 1.1% lower than the full-precision network accuracy 93.5%. On Android phone app, the trained binary network doubles the speed of full-precision network in spoken keywords recognition.
Tasks
Published	2018-11-07
URL	http://arxiv.org/abs/1811.02784v1
PDF	http://arxiv.org/pdf/1811.02784v1.pdf
PWC	https://paperswithcode.com/paper/median-binary-connect-method-and-a-binary
Repo
Framework