January 30, 2020

3173 words 15 mins read

Paper Group ANR 289

Online Collection and Forecasting of Resource Utilization in Large-Scale Distributed Systems. Improvement of the Izhikevich model based on the rat basolateral amygdala and hippocampus neurons, and recognition of their possible firing patterns. Implementing Binarized Neural Networks with Magnetoresistive RAM without Error Correction. Foundations of …

Online Collection and Forecasting of Resource Utilization in Large-Scale Distributed Systems


Title	Online Collection and Forecasting of Resource Utilization in Large-Scale Distributed Systems
Authors	Tiffany Tuor, Shiqiang Wang, Kin K. Leung, Bong Jun Ko
Abstract	Large-scale distributed computing systems often contain thousands of distributed nodes (machines). Monitoring the conditions of these nodes is important for system management purposes, which, however, can be extremely resource demanding as this requires collecting local measurements of each individual node and constantly sending those measurements to a central controller. Meanwhile, it is often useful to forecast the future system conditions for various purposes such as resource planning/allocation and anomaly detection, but it is usually too resource-consuming to have one forecasting model running for each node, which may also neglect correlations in observed metrics across different nodes. In this paper, we propose a mechanism for collecting and forecasting the resource utilization of machines in a distributed computing system in a scalable manner. We present an algorithm that allows each local node to decide when to transmit its most recent measurement to the central node, so that the transmission frequency is kept below a given constraint value. Based on the measurements received from local nodes, the central node summarizes the received data into a small number of clusters. Since the cluster partitioning can change over time, we also present a method to capture the evolution of clusters and their centroids. As an effective way to reduce the amount of computation, time-series forecasting models are trained on the time-varying centroids of each cluster, to forecast the future resource utilizations of a group of local nodes. The effectiveness of our proposed approach is confirmed by extensive experiments using multiple real-world datasets.
Tasks	Anomaly Detection, Time Series, Time Series Forecasting
Published	2019-05-22
URL	https://arxiv.org/abs/1905.09219v1
PDF	https://arxiv.org/pdf/1905.09219v1.pdf
PWC	https://paperswithcode.com/paper/online-collection-and-forecasting-of-resource
Repo
Framework

Improvement of the Izhikevich model based on the rat basolateral amygdala and hippocampus neurons, and recognition of their possible firing patterns


Title	Improvement of the Izhikevich model based on the rat basolateral amygdala and hippocampus neurons, and recognition of their possible firing patterns
Authors	Sahar Hojjatinia, Mahdi Aliyari Shoorehdeli, Zahra Fatahi, Zeinab Hojjatinia, Abbas Haghparast
Abstract	Introduction: Identifying the potential firing patterns following by different brain regions under normal and abnormal conditions increases our understanding of what is happening in the level of neural interactions in the brain. On the other hand, it is important to be capable of modeling the potential neural activities, in order to build precise artificial neural networks. The Izhikevich model is one of the simple biologically plausible models that is capable of capturing the most known firing patterns of neurons. This property makes the model efficient in simulating large-scale networks of neurons. Improving the Izhikevich model for adapting with the neuronal activity of rat brain with great accuracy would make the model effective for future neural network implementations. Methods: Data sampling from two brain regions, the HIP and BLA, is performed by extracellular recordings of male Wistar rats and spike sorting is done using Plexon offline sorter. Further data analyses are done through NeuroExplorer and MATLAB software. In order to optimize the Izhikevich model parameters, the genetic algorithm is used. Results: In the present study, the possible firing patterns of the real single neurons of the HIP and BLA are identified. Additionally, improvement of the Izhikevich model is achieved. As a result, the real neuronal spiking pattern of these regions neurons, and the corresponding cases of the Izhikevich neuron spiking pattern are adjusted with great accuracy. Conclusion: This study is conducted to elevate our knowledge of neural interactions in different structures of the brain and accelerate the quality of future large scale neural networks simulations, as well as reducing the modeling complexity. This aim is achievable by performing the improved Izhikevich model, and inserting only the plausible firing patterns and eliminating unrealistic ones, as the results of this study.
Tasks
Published	2019-10-24
URL	https://arxiv.org/abs/1910.11380v2
PDF	https://arxiv.org/pdf/1910.11380v2.pdf
PWC	https://paperswithcode.com/paper/improvement-of-the-izhikevich-model-based-on
Repo
Framework

Implementing Binarized Neural Networks with Magnetoresistive RAM without Error Correction


Title	Implementing Binarized Neural Networks with Magnetoresistive RAM without Error Correction
Authors	Tifenn Hirtzlin, Bogdan Penkovsky, Jacques-Olivier Klein, Nicolas Locatelli, Adrien F. Vincent, Marc Bocquet, Jean-Michel Portal, Damien Querlioz
Abstract	One of the most exciting applications of Spin Torque Magnetoresistive Random Access Memory (ST-MRAM) is the in-memory implementation of deep neural networks, which could allow improving the energy efficiency of Artificial Intelligence by orders of magnitude with regards to its implementation on computers and graphics cards. In particular, ST-MRAM could be ideal for implementing Binarized Neural Networks (BNNs), a type of deep neural networks discovered in 2016, which can achieve state-of-the-art performance with a highly reduced memory footprint with regards to conventional artificial intelligence approaches. The challenge of ST-MRAM, however, is that it is prone to write errors and usually requires the use of error correction. In this work, we show that these bit errors can be tolerated by BNNs to an outstanding level, based on examples of image recognition tasks (MNIST, CIFAR-10 and ImageNet): bit error rates of ST-MRAM up to 0.1% have little impact on recognition accuracy. The requirements for ST-MRAM are therefore considerably relaxed for BNNs with regards to traditional applications. By consequence, we show that for BNNs, ST-MRAMs can be programmed with weak (low-energy) programming conditions, without error correcting codes. We show that this result can allow the use of low energy and low area ST-MRAM cells, and show that the energy savings at the system level can reach a factor two.
Tasks
Published	2019-08-12
URL	https://arxiv.org/abs/1908.04085v1
PDF	https://arxiv.org/pdf/1908.04085v1.pdf
PWC	https://paperswithcode.com/paper/implementing-binarized-neural-networks-with
Repo
Framework

Foundations of Structural Statistics: Topological Statistical Theory


Title	Foundations of Structural Statistics: Topological Statistical Theory
Authors	Patrick Michl
Abstract	Topological Statistical Theory, provides the foundation for a new understanding of classical Statistics: Structural Statistics, which emphasizes intrinsically structured model spaces and structure preserving transformations as the central objects and morphisms of respective categories. The resulting language not only turns out to be highly compatible with classical statistical theory, but indeed outperforms it in simplicity and elegance for complicated model spaces. Maybe the most important present showcases for this frameworks are machine-learning and in particular deep-learning. There above it concerns topological-, geometric- as well as algebraic- data analysis, which respectively derive statistical estimations, by the assumption of simplicial complexes, Riemannian manifolds and algebraic varieties.
Tasks
Published	2019-12-21
URL	https://arxiv.org/abs/1912.10266v1
PDF	https://arxiv.org/pdf/1912.10266v1.pdf
PWC	https://paperswithcode.com/paper/foundations-of-structural-statistics
Repo
Framework

Every Local Minimum Value is the Global Minimum Value of Induced Model in Non-convex Machine Learning


Title	Every Local Minimum Value is the Global Minimum Value of Induced Model in Non-convex Machine Learning
Authors	Kenji Kawaguchi, Jiaoyang Huang, Leslie Pack Kaelbling
Abstract	For nonconvex optimization in machine learning, this article proves that every local minimum achieves the globally optimal value of the perturbable gradient basis model at any differentiable point. As a result, nonconvex machine learning is theoretically as supported as convex machine learning with a handcrafted basis in terms of the loss at differentiable local minima, except in the case when a preference is given to the handcrafted basis over the perturbable gradient basis. The proofs of these results are derived under mild assumptions. Accordingly, the proven results are directly applicable to many machine learning models, including practical deep neural networks, without any modification of practical methods. Furthermore, as special cases of our general results, this article improves or complements several state-of-the-art theoretical results on deep neural networks, deep residual networks, and overparameterized deep neural networks with a unified proof technique and novel geometric insights. A special case of our results also contributes to the theoretical foundation of representation learning.
Tasks	Representation Learning
Published	2019-04-07
URL	https://arxiv.org/abs/1904.03673v3
PDF	https://arxiv.org/pdf/1904.03673v3.pdf
PWC	https://paperswithcode.com/paper/every-local-minimum-is-a-global-minimum-of-an
Repo
Framework

Negatively Correlated Search as a Parallel Exploration Search Strategy


Title	Negatively Correlated Search as a Parallel Exploration Search Strategy
Authors	Peng Yang, Ke Tang, Xin Yao
Abstract	Parallel exploration is a key to a successful search. The recently proposed Negatively Correlated Search (NCS) achieved this ability by constructing a set of negatively correlated search processes and has been applied to many real-world problems. In NCS, the key technique is to explicitly model and maximize the diversity among search processes in parallel. However, the original diversity model was mostly devised by intuition, which introduced several drawbacks to NCS. In this paper, a mathematically principled diversity model is proposed to solve the existing drawbacks of NCS, resulting a new NCS framework. A new instantiation of NCS is also derived and its effectiveness is verified on a set of multi-modal continuous optimization problems.
Tasks
Published	2019-10-16
URL	https://arxiv.org/abs/1910.07151v1
PDF	https://arxiv.org/pdf/1910.07151v1.pdf
PWC	https://paperswithcode.com/paper/negatively-correlated-search-as-a-parallel
Repo
Framework

Learning an Efficient Network for Large-Scale Hierarchical Object Detection with Data Imbalance: 3rd Place Solution to Open Images Challenge 2019


Title	Learning an Efficient Network for Large-Scale Hierarchical Object Detection with Data Imbalance: 3rd Place Solution to Open Images Challenge 2019
Authors	Xingyuan Bu, Junran Peng, Changbao Wang, Cunjun Yu, Guoliang Cao
Abstract	This report details our solution to the Google AI Open Images Challenge 2019 Object Detection Track. Based on our detailed analysis on the Open Images dataset, it is found that there are four typical features: large-scale, hierarchical tag system, severe annotation incompleteness and data imbalance. Considering these characteristics, many strategies are employed, including larger backbone, distributed softmax loss, class-aware sampling, expert model, and heavier classifier. In virtue of these effective strategies, our best single model could achieve a mAP of 61.90. After ensemble, the final mAP is boosted to 67.17 in the public leaderboard and 64.21 in the private leaderboard, which earns 3rd place in the Open Images Challenge 2019.
Tasks	Object Detection
Published	2019-10-26
URL	https://arxiv.org/abs/1910.12044v1
PDF	https://arxiv.org/pdf/1910.12044v1.pdf
PWC	https://paperswithcode.com/paper/learning-an-efficient-network-for-large-scale
Repo
Framework

Profiling Players with Engagement Predictions


Title	Profiling Players with Engagement Predictions
Authors	Ana Fernández del Río, Pei Pei Chen, África Periáñez
Abstract	The possibility of using player engagement predictions to profile high spending video game users is explored. In particular, individual-player survival curves in terms of days after first login, game level reached and accumulated playtime are used to classify players into different groups. Lifetime value predictions for each player—generated using a deep learning method based on long short-term memory—are also included in the analysis, and the relations between all these variables are thoroughly investigated. Our results suggest this constitutes a promising approach to user profiling.
Tasks
Published	2019-07-09
URL	https://arxiv.org/abs/1907.03870v1
PDF	https://arxiv.org/pdf/1907.03870v1.pdf
PWC	https://paperswithcode.com/paper/profiling-players-with-engagement-predictions
Repo
Framework

VCWE: Visual Character-Enhanced Word Embeddings


Title	VCWE: Visual Character-Enhanced Word Embeddings
Authors	Chi Sun, Xipeng Qiu, Xuanjing Huang
Abstract	Chinese is a logographic writing system, and the shape of Chinese characters contain rich syntactic and semantic information. In this paper, we propose a model to learn Chinese word embeddings via three-level composition: (1) a convolutional neural network to extract the intra-character compositionality from the visual shape of a character; (2) a recurrent neural network with self-attention to compose character representation into word embeddings; (3) the Skip-Gram framework to capture non-compositionality directly from the contextual information. Evaluations demonstrate the superior performance of our model on four tasks: word similarity, sentiment analysis, named entity recognition and part-of-speech tagging.
Tasks	Named Entity Recognition, Part-Of-Speech Tagging, Sentiment Analysis, Word Embeddings
Published	2019-02-23
URL	http://arxiv.org/abs/1902.08795v2
PDF	http://arxiv.org/pdf/1902.08795v2.pdf
PWC	https://paperswithcode.com/paper/vcwe-visual-character-enhanced-word
Repo
Framework

ThisIsCompetition at SemEval-2019 Task 9: BERT is unstable for out-of-domain samples


Title	ThisIsCompetition at SemEval-2019 Task 9: BERT is unstable for out-of-domain samples
Authors	Cheoneum Park, Juae Kim, Hyeon-gu Lee, Reinald Kim Amplayo, Harksoo Kim, Jungyun Seo, Changki Lee
Abstract	This paper describes our system, Joint Encoders for Stable Suggestion Inference (JESSI), for the SemEval 2019 Task 9: Suggestion Mining from Online Reviews and Forums. JESSI is a combination of two sentence encoders: (a) one using multiple pre-trained word embeddings learned from log-bilinear regression (GloVe) and translation (CoVe) models, and (b) one on top of word encodings from a pre-trained deep bidirectional transformer (BERT). We include a domain adversarial training module when training for out-of-domain samples. Our experiments show that while BERT performs exceptionally well for in-domain samples, several runs of the model show that it is unstable for out-of-domain samples. The problem is mitigated tremendously by (1) combining BERT with a non-BERT encoder, and (2) using an RNN-based classifier on top of BERT. Our final models obtained second place with 77.78% F-Score on Subtask A (i.e. in-domain) and achieved an F-Score of 79.59% on Subtask B (i.e. out-of-domain), even without using any additional external data.
Tasks	Word Embeddings
Published	2019-04-06
URL	http://arxiv.org/abs/1904.03339v1
PDF	http://arxiv.org/pdf/1904.03339v1.pdf
PWC	https://paperswithcode.com/paper/thisiscompetition-at-semeval-2019-task-9-bert
Repo
Framework

Quantum Generalized Linear Models


Title	Quantum Generalized Linear Models
Authors	Colleen M. Farrelly, Srikanth Namuduri, Uchenna Chukwu
Abstract	Generalized linear models (GLM) are link function based statistical models. Many supervised learning algorithms are extensions of GLMs and have link functions built into the algorithm to model different outcome distributions. There are two major drawbacks when using this approach in applications using real world datasets. One is that none of the link functions available in the popular packages is a good fit for the data. Second, it is computationally inefficient and impractical to test all the possible distributions to find the optimum one. In addition, many GLMs and their machine learning extensions struggle on problems of overdispersion in Tweedie distributions. In this paper we propose a quantum extension to GLM that overcomes these drawbacks. A quantum gate with non-Gaussian transformation can be used to continuously deform the outcome distribution from known results. In doing so, we eliminate the need for a link function. Further, by using an algorithm that superposes all possible distributions to collapse to fit a dataset, we optimize the model in a computationally efficient way. We provide an initial proof-of-concept by testing this approach on both a simulation of overdispersed data and then on a benchmark dataset, which is quite overdispersed, and achieved state of the art results. This is a game changer in several applied fields, such as part failure modeling, medical research, actuarial science, finance and many other fields where Tweedie regression and overdispersion are ubiquitous.
Tasks
Published	2019-05-01
URL	http://arxiv.org/abs/1905.00365v1
PDF	http://arxiv.org/pdf/1905.00365v1.pdf
PWC	https://paperswithcode.com/paper/quantum-generalized-linear-models
Repo
Framework

Assessing the Difficulty of Classifying ConceptNet Relations in a Multi-Label Classification Setting


Title	Assessing the Difficulty of Classifying ConceptNet Relations in a Multi-Label Classification Setting
Authors	Maria Becker, Michael Staniek, Vivi Nastase, Anette Frank
Abstract	Commonsense knowledge relations are crucial for advanced NLU tasks. We examine the learnability of such relations as represented in CONCEPTNET, taking into account their specific properties, which can make relation classification difficult: a given concept pair can be linked by multiple relation types, and relations can have multi-word arguments of diverse semantic types. We explore a neural open world multi-label classification approach that focuses on the evaluation of classification accuracy for individual relations. Based on an in-depth study of the specific properties of the CONCEPTNET resource, we investigate the impact of different relation representations and model variations. Our analysis reveals that the complexity of argument types and relation ambiguity are the most important challenges to address. We design a customized evaluation method to address the incompleteness of the resource that can be expanded in future work.
Tasks	Multi-Label Classification, Relation Classification
Published	2019-05-14
URL	https://arxiv.org/abs/1905.05538v1
PDF	https://arxiv.org/pdf/1905.05538v1.pdf
PWC	https://paperswithcode.com/paper/assessing-the-difficulty-of-classifying
Repo
Framework

Optimising Clifford Circuits with Quantomatic


Title	Optimising Clifford Circuits with Quantomatic
Authors	Andrew Fagan, Ross Duncan
Abstract	We present a system of equations between Clifford circuits, all derivable in the ZX-calculus, and formalised as rewrite rules in the Quantomatic proof assistant. By combining these rules with some non-trivial simplification procedures defined in the Quantomatic tactic language, we demonstrate the use of Quantomatic as a circuit optimisation tool. We prove that the system always reduces Clifford circuits of one or two qubits to their minimal form, and give numerical results demonstrating its performance on larger Clifford circuits.
Tasks
Published	2019-01-29
URL	http://arxiv.org/abs/1901.10114v1
PDF	http://arxiv.org/pdf/1901.10114v1.pdf
PWC	https://paperswithcode.com/paper/optimising-clifford-circuits-with-quantomatic
Repo
Framework

Deep Learning Based Motion Planning For Autonomous Vehicle Using Spatiotemporal LSTM Network


Title	Deep Learning Based Motion Planning For Autonomous Vehicle Using Spatiotemporal LSTM Network
Authors	Zhengwei Bai, Baigen Cai, Wei Shangguan, Linguo Chai
Abstract	Motion Planning, as a fundamental technology of automatic navigation for the autonomous vehicle, is still an open challenging issue in the real-life traffic situation and is mostly applied by the model-based approaches. However, due to the complexity of the traffic situations and the uncertainty of the edge cases, it is hard to devise a general motion planning system for the autonomous vehicle. In this paper, we proposed a motion planning model based on deep learning (named as spatiotemporal LSTM network), which is able to generate a real-time reflection based on spatiotemporal information extraction. To be specific, the model based on spatiotemporal LSTM network has three main structure. Firstly, the Convolutional Long-short Term Memory (Conv-LSTM) is used to extract hidden features through sequential image data. Then, the 3D Convolutional Neural Network(3D-CNN) is applied to extract the spatiotemporal information from the multi-frame feature information. Finally, the fully connected neural networks are used to construct a control model for autonomous vehicle steering angle. The experiments demonstrated that the proposed method can generate a robust and accurate visual motion planning results for the autonomous vehicle.
Tasks	Motion Planning
Published	2019-03-05
URL	http://arxiv.org/abs/1903.01712v1
PDF	http://arxiv.org/pdf/1903.01712v1.pdf
PWC	https://paperswithcode.com/paper/deep-learning-based-motion-planning-for
Repo
Framework

Accelerating PDE-constrained Inverse Solutions with Deep Learning and Reduced Order Models


Title	Accelerating PDE-constrained Inverse Solutions with Deep Learning and Reduced Order Models
Authors	Sheroze Sheriffdeen, Jean C. Ragusa, Jim E. Morel, Marvin L. Adams, Tan Bui-Thanh
Abstract	Inverse problems are pervasive mathematical methods in inferring knowledge from observational and experimental data by leveraging simulations and models. Unlike direct inference methods, inverse problem approaches typically require many forward model solves usually governed by Partial Differential Equations (PDEs). This a crucial bottleneck in determining the feasibility of such methods. While machine learning (ML) methods, such as deep neural networks (DNNs), can be employed to learn nonlinear forward models, designing a network architecture that preserves accuracy while generalizing to new parameter regimes is a daunting task. Furthermore, due to the computation-expensive nature of forward models, state-of-the-art black-box ML methods would require an unrealistic amount of work in order to obtain an accurate surrogate model. On the other hand, standard Reduced-Order Models (ROMs) accurately capture supposedly important physics of the forward model in the reduced subspaces, but otherwise could be inaccurate elsewhere. In this paper, we propose to enlarge the validity of ROMs and hence improve the accuracy outside the reduced subspaces by incorporating a data-driven ML technique. In particular, we focus on a goal-oriented approach that substantially improves the accuracy of reduced models by learning the error between the forward model and the ROM outputs. Once an ML-enhanced ROM is constructed it can accelerate the performance of solving many-query problems in parametrized forward and inverse problems. Numerical results for inverse problems governed by elliptic PDEs and parametrized neutron transport equations will be presented to support our approach.
Tasks
Published	2019-12-17
URL	https://arxiv.org/abs/1912.08864v1
PDF	https://arxiv.org/pdf/1912.08864v1.pdf
PWC	https://paperswithcode.com/paper/accelerating-pde-constrained-inverse
Repo
Framework