April 1, 2020

3180 words 15 mins read

Paper Group ANR 507

LUNAR: Cellular Automata for Drifting Data Streams. Deep Representation Learning for Dynamical Systems Modeling. A spatio-temporalisation of ALC(D) and its translation into alternating automata augmented with spatial constraints. A Markerless Deep Learning-based 6 Degrees of Freedom PoseEstimation for with Mobile Robots using RGB Data. Stochastic I …

LUNAR: Cellular Automata for Drifting Data Streams


Title	LUNAR: Cellular Automata for Drifting Data Streams
Authors	Jesus L. Lobo, Javier Del Ser, Francisco Herrera
Abstract	With the advent of huges volumes of data produced in the form of fast streams, real-time machine learning has become a challenge of relevance emerging in a plethora of real-world applications. Processing such fast streams often demands high memory and processing resources. In addition, they can be affected by non-stationary phenomena (concept drift), by which learning methods have to detect changes in the distribution of streaming data, and adapt to these evolving conditions. A lack of efficient and scalable solutions is particularly noted in real-time scenarios where computing resources are severely constrained, as it occurs in networks of small, numerous, interconnected processing units (such as the so-called Smart Dust, Utility Fog, or Swarm Robotics paradigms). In this work we propose LUNAR, a streamified version of cellular automata devised to successfully meet the aforementioned requirements. It is able to act as a real incremental learner while adapting to drifting conditions. Extensive simulations with synthetic and real data will provide evidence of its competitive behavior in terms of classification performance when compared to long-established and successful online learning methods.
Tasks
Published	2020-02-06
URL	https://arxiv.org/abs/2002.02164v1
PDF	https://arxiv.org/pdf/2002.02164v1.pdf
PWC	https://paperswithcode.com/paper/lunar-cellular-automata-for-drifting-data
Repo
Framework

Deep Representation Learning for Dynamical Systems Modeling


Title	Deep Representation Learning for Dynamical Systems Modeling
Authors	Anna Shalova, Ivan Oseledets
Abstract	Proper states’ representations are the key to the successful dynamics modeling of chaotic systems. Inspired by recent advances of deep representations in various areas such as natural language processing and computer vision, we propose the adaptation of the state-of-art Transformer model in application to the dynamical systems modeling. The model demonstrates promising results in trajectories generation as well as in the general attractors’ characteristics approximation, including states’ distribution and Lyapunov exponent.
Tasks	Representation Learning
Published	2020-02-10
URL	https://arxiv.org/abs/2002.05111v1
PDF	https://arxiv.org/pdf/2002.05111v1.pdf
PWC	https://paperswithcode.com/paper/deep-representation-learning-for-dynamical
Repo
Framework

A spatio-temporalisation of ALC(D) and its translation into alternating automata augmented with spatial constraints


Title	A spatio-temporalisation of ALC(D) and its translation into alternating automata augmented with spatial constraints
Authors	Amar Isli
Abstract	The aim of this work is to provide a family of qualitative theories for spatial change in general, and for motion of spatial scenes in particular. To achieve this, we consider a spatio-temporalisation MTALC(Dx), of the well-known ALC(D) family of Description Logics (DLs) with a concrete domain: the MTALC(Dx) concepts are interpreted over infinite k-ary Sigma-trees, with the nodes standing for time points, and Sigma including, additionally to its uses in classical k-ary Sigma-trees, the description of the snapshot of an n-object spatial scene of interest; the roles split into m+n immediate-successor (accessibility) relations, which are serial, irreflexive and antisymmetric, and of which m are general, not necessarily functional, the other n functional; the concrete domain Dx is generated by an RCC8-like spatial Relation Algebra (RA) x, and is used to guide the change by imposing spatial constraints on objects of the “followed” spatial scene, eventually at different time points of the input trees. In order to capture the expressiveness of most modal temporal logics encountered in the literature, we introduce weakly cyclic Terminological Boxes (TBoxes) of MTALC(Dx), whose axioms capture the decreasing property of modal temporal operators. We show the important result that satisfiability of an MTALC(Dx) concept with respect to a weakly cyclic TBox can be reduced to the emptiness problem of a Buchi weak alternating automaton augmented with spatial constraints. In another work, complementary to this one, also submitted to this conference, we thoroughly investigate Buchi automata augmented with spatial constraints, and provide, in particular, a translation of an alternating into a nondeterministic, and an effective decision procedure for the emptiness problem of the latter.
Tasks
Published	2020-02-22
URL	https://arxiv.org/abs/2002.12760v1
PDF	https://arxiv.org/pdf/2002.12760v1.pdf
PWC	https://paperswithcode.com/paper/a-spatio-temporalisation-of-alcd-and-its
Repo
Framework

A Markerless Deep Learning-based 6 Degrees of Freedom PoseEstimation for with Mobile Robots using RGB Data


Title	A Markerless Deep Learning-based 6 Degrees of Freedom PoseEstimation for with Mobile Robots using RGB Data
Authors	Linh Kästner, Daniel Dimitrov, Jens Lambrecht
Abstract	Augmented Reality has been subject to various integration efforts within industries due to its ability to enhance human machine interaction and understanding. Neural networks have achieved remarkable results in areas of computer vision, which bear great potential to assist and facilitate an enhanced Augmented Reality experience. However, most neural networks are computationally intensive and demand huge processing power thus, are not suitable for deployment on Augmented Reality devices. In this work we propose a method to deploy state of the art neural networks for real time 3D object localization on augmented reality devices. As a result, we provide a more automated method of calibrating the AR devices with mobile robotic systems. To accelerate the calibration process and enhance user experience, we focus on fast 2D detection approaches which are extracting the 3D pose of the object fast and accurately by using only 2D input. The results are implemented into an Augmented Reality application for intuitive robot control and sensor data visualization. For the 6D annotation of 2D images, we developed an annotation tool, which is, to our knowledge, the first open source tool to be available. We achieve feasible results which are generally applicable to any AR device thus making this work promising for further research in combining high demanding neural networks with Internet of Things devices.
Tasks	Calibration, Object Localization
Published	2020-01-16
URL	https://arxiv.org/abs/2001.05703v1
PDF	https://arxiv.org/pdf/2001.05703v1.pdf
PWC	https://paperswithcode.com/paper/a-markerless-deep-learning-based-6-degrees-of
Repo
Framework

Stochastic Item Descent Method for Large Scale Equal Circle Packing Problem


Title	Stochastic Item Descent Method for Large Scale Equal Circle Packing Problem
Authors	Kun He, Min Zhang, Jianrong Zhou, Yan Jin, Chu-min Li
Abstract	Stochastic gradient descent (SGD) is a powerful method for large-scale optimization problems in the area of machine learning, especially for a finite-sum formulation with numerous variables. In recent years, mini-batch SGD gains great success and has become a standard technique for training deep neural networks fed with big amount of data. Inspired by its success in deep learning, we apply the idea of SGD with batch selection of samples to a classic optimization problem in decision version. Given $n$ unit circles, the equal circle packing problem (ECPP) asks whether there exist a feasible packing that could put all the circles inside a circular container without overlapping. Specifically, we propose a stochastic item descent method (SIDM) for ECPP in large scale, which randomly divides the unit circles into batches and runs Broyden-Fletcher-Goldfarb-Shanno (BFGS) algorithm on the corresponding batch function iteratively to speedup the calculation. We also increase the batch size during the batch iterations to gain higher quality solution. Comparing to the current best packing algorithms, SIDM greatly speeds up the calculation of optimization process and guarantees the solution quality for large scale instances with up to 1500 circle items, while the baseline algorithms usually handle about 300 circle items. The results indicate the highly efficiency of SIDM for this classic optimization problem in large scale, and show potential for other large scale classic optimization problems in which gradient descent is used for optimization.
Tasks
Published	2020-01-22
URL	https://arxiv.org/abs/2001.08540v1
PDF	https://arxiv.org/pdf/2001.08540v1.pdf
PWC	https://paperswithcode.com/paper/stochastic-item-descent-method-for-large
Repo
Framework

Few-Shot Learning via Learning the Representation, Provably


Title	Few-Shot Learning via Learning the Representation, Provably
Authors	Simon S. Du, Wei Hu, Sham M. Kakade, Jason D. Lee, Qi Lei
Abstract	This paper studies few-shot learning via representation learning, where one uses $T$ source tasks with $n_1$ data per task to learn a representation in order to reduce the sample complexity of a target task for which there is only $n_2 (\ll n_1)$ data. Specifically, we focus on the setting where there exists a good \emph{common representation} between source and target, and our goal is to understand how much of a sample size reduction is possible. First, we study the setting where this common representation is low-dimensional and provide a fast rate of $O\left(\frac{\mathcal{C}\left(\Phi\right)}{n_1T} + \frac{k}{n_2}\right)$; here, $\Phi$ is the representation function class, $\mathcal{C}\left(\Phi\right)$ is its complexity measure, and $k$ is the dimension of the representation. When specialized to linear representation functions, this rate becomes $O\left(\frac{dk}{n_1T} + \frac{k}{n_2}\right)$ where $d (\gg k)$ is the ambient input dimension, which is a substantial improvement over the rate without using representation learning, i.e. over the rate of $O\left(\frac{d}{n_2}\right)$. Second, we consider the setting where the common representation may be high-dimensional but is capacity-constrained (say in norm); here, we again demonstrate the advantage of representation learning in both high-dimensional linear regression and neural network learning. Our results demonstrate representation learning can fully utilize all $n_1T$ samples from source tasks.
Tasks	Few-Shot Learning, Representation Learning
Published	2020-02-21
URL	https://arxiv.org/abs/2002.09434v1
PDF	https://arxiv.org/pdf/2002.09434v1.pdf
PWC	https://paperswithcode.com/paper/few-shot-learning-via-learning-the
Repo
Framework

Few-shot acoustic event detection via meta-learning


Title	Few-shot acoustic event detection via meta-learning
Authors	Bowen Shi, Ming Sun, Krishna C. Puvvada, Chieh-Chi Kao, Spyros Matsoukas, Chao Wang
Abstract	We study few-shot acoustic event detection (AED) in this paper. Few-shot learning enables detection of new events with very limited labeled data. Compared to other research areas like computer vision, few-shot learning for audio recognition has been under-studied. We formulate few-shot AED problem and explore different ways of utilizing traditional supervised methods for this setting as well as a variety of meta-learning approaches, which are conventionally used to solve few-shot classification problem. Compared to supervised baselines, meta-learning models achieve superior performance, thus showing its effectiveness on generalization to new audio events. Our analysis including impact of initialization and domain discrepancy further validate the advantage of meta-learning approaches in few-shot AED.
Tasks	Few-Shot Learning, Meta-Learning
Published	2020-02-21
URL	https://arxiv.org/abs/2002.09143v1
PDF	https://arxiv.org/pdf/2002.09143v1.pdf
PWC	https://paperswithcode.com/paper/few-shot-acoustic-event-detection-via-meta
Repo
Framework

Reflected Schrödinger Bridge: Density Control with Path Constraints


Title	Reflected Schrödinger Bridge: Density Control with Path Constraints
Authors	Kenneth F. Caluya, Abhishek Halder
Abstract	How to steer a given joint state probability density function to another over finite horizon subject to a controlled stochastic dynamics with hard state (sample path) constraints? In applications, state constraints may encode safety requirements such as obstacle avoidance. In this paper, we perform the feedback synthesis for minimum control effort density steering (a.k.a. Schr"{o}dinger bridge) problem subject to state constraints. We extend the theory of Schr"{o}dinger bridges to account the reflecting boundary conditions for the sample paths, and provide a computational framework building on our previous work on proximal recursions, to solve the same.
Tasks
Published	2020-03-31
URL	https://arxiv.org/abs/2003.13895v1
PDF	https://arxiv.org/pdf/2003.13895v1.pdf
PWC	https://paperswithcode.com/paper/reflected-schrodinger-bridge-density-control
Repo
Framework

Vertex-reinforced Random Walk for Network Embedding


Title	Vertex-reinforced Random Walk for Network Embedding
Authors	Wenyi Xiao, Huan Zhao, Vincent W. Zheng, Yangqiu Song
Abstract	In this paper, we study the fundamental problem of random walk for network embedding. We propose to use non-Markovian random walk, variants of vertex-reinforced random walk (VRRW), to fully use the history of a random walk path. To solve the getting stuck problem of VRRW, we introduce an exploitation-exploration mechanism to help the random walk jump out of the stuck set. The new random walk algorithms share the same convergence property of VRRW and thus can be used to learn stable network embeddings. Experimental results on two link prediction benchmark datasets and three node classification benchmark datasets show that our proposed approach reinforce2vec can outperform state-of-the-art random walk based embedding methods by a large margin.
Tasks	Link Prediction, Network Embedding, Node Classification
Published	2020-02-11
URL	https://arxiv.org/abs/2002.04497v1
PDF	https://arxiv.org/pdf/2002.04497v1.pdf
PWC	https://paperswithcode.com/paper/vertex-reinforced-random-walk-for-network
Repo
Framework

Investigating Extensions to Random Walk Based Graph Embedding


Title	Investigating Extensions to Random Walk Based Graph Embedding
Authors	Joerg Schloetterer, Martin Wehking, Fatemeh Salehi Rizi, Michael Granitzer
Abstract	Graph embedding has recently gained momentum in the research community, in particular after the introduction of random walk and neural network based approaches. However, most of the embedding approaches focus on representing the local neighborhood of nodes and fail to capture the global graph structure, i.e. to retain the relations to distant nodes. To counter that problem, we propose a novel extension to random walk based graph embedding, which removes a percentage of least frequent nodes from the walks at different levels. By this removal, we simulate farther distant nodes to reside in the close neighborhood of a node and hence explicitly represent their connection. Besides the common evaluation tasks for graph embeddings, such as node classification and link prediction, we evaluate and compare our approach against related methods on shortest path approximation. The results indicate, that extensions to random walk based methods (including our own) improve the predictive performance only slightly - if at all.
Tasks	Graph Embedding, Link Prediction, Node Classification
Published	2020-02-17
URL	https://arxiv.org/abs/2002.07252v1
PDF	https://arxiv.org/pdf/2002.07252v1.pdf
PWC	https://paperswithcode.com/paper/investigating-extensions-to-random-walk-based
Repo
Framework

TArC: Incrementally and Semi-Automatically Collecting a Tunisian Arabish Corpus


Title	TArC: Incrementally and Semi-Automatically Collecting a Tunisian Arabish Corpus
Authors	Elisa Gugliotta, Marco Dinarelli
Abstract	This article describes the constitution process of the first morpho-syntactically annotated Tunisian Arabish Corpus (TArC). Arabish, also known as Arabizi, is a spontaneous coding of Arabic dialects in Latin characters and arithmographs (numbers used as letters). This code-system was developed by Arabic-speaking users of social media in order to facilitate the writing in the Computer-Mediated Communication (CMC) and text messaging informal frameworks. There is variety in the realization of Arabish amongst dialects, and each Arabish code-system is under-resourced, in the same way as most of the Arabic dialects. In the last few years, the focus on Arabic dialects in the NLP field has considerably increased. Taking this into consideration, TArC will be a useful support for different types of analyses, computational and linguistic, as well as for NLP tools training. In this article we will describe preliminary work on the TArC semi-automatic construction process and some of the first analyses we developed on TArC. In addition, in order to provide a complete overview of the challenges faced during the building process, we will present the main Tunisian dialect characteristics and their encoding in Tunisian Arabish.
Tasks
Published	2020-03-20
URL	https://arxiv.org/abs/2003.09520v2
PDF	https://arxiv.org/pdf/2003.09520v2.pdf
PWC	https://paperswithcode.com/paper/tarc-incrementally-and-semi-automatically
Repo
Framework

Bilinear Graph Neural Network with Node Interactions


Title	Bilinear Graph Neural Network with Node Interactions
Authors	Hongmin Zhu, Fuli Feng, Xiangnan He, Xiang Wang, Yan Li, Kai Zheng, Yongdong Zhang
Abstract	Graph Neural Network (GNN) is a powerful model to learn representations and make predictions on graph data. Existing efforts on GNN have largely defined the graph convolution as a weighted sum of the features of the connected nodes to form the representation of the target node. Nevertheless, the operation of weighted sum assumes the neighbor nodes are independent of each other, and ignores the possible interactions between them. When such interactions exist, such as the co-occurrence of two neighbor nodes is a strong signal of the target node’s characteristics, existing GNN models may fail to capture the signal. In this work, we argue the importance of modeling the interactions between neighbor nodes in GNN. We propose a new graph convolution operator, which augments the weighted sum with pairwise interactions of the representations of neighbor nodes. We term this framework as Bilinear Graph Neural Network (BGNN), which improves GNN representation ability with bilinear interactions between neighbor nodes. In particular, we specify two BGNN models named BGCN and BGAT, based on the well-known GCN and GAT, respectively. Empirical results on three public benchmarks of semi-supervised node classification verify the effectiveness of BGNN — BGCN (BGAT) outperforms GCN (GAT) by 1.6% (1.5%) in classification accuracy.
Tasks	Node Classification
Published	2020-02-10
URL	https://arxiv.org/abs/2002.03575v2
PDF	https://arxiv.org/pdf/2002.03575v2.pdf
PWC	https://paperswithcode.com/paper/bilinear-graph-neural-network-with-node
Repo
Framework

AutoEmb: Automated Embedding Dimensionality Search in Streaming Recommendations


Title	AutoEmb: Automated Embedding Dimensionality Search in Streaming Recommendations
Authors	Xiangyu Zhao, Chong Wang, Ming Chen, Xudong Zheng, Xiaobing Liu, Jiliang Tang
Abstract	Deep learning based recommender systems (DLRSs) often have embedding layers, which are utilized to lessen the dimensionality of categorical variables (e.g. user/item identifiers) and meaningfully transform them in the low-dimensional space. The majority of existing DLRSs empirically pre-define a fixed and unified dimension for all user/item embeddings. It is evident from recent researches that different embedding sizes are highly desired for different users/items according to their popularity. However, manually selecting embedding sizes in recommender systems can be very challenging due to the large number of users/items and the dynamic nature of their popularity. Thus, in this paper, we propose an AutoML based end-to-end framework (AutoEmb), which can enable various embedding dimensions according to the popularity in an automated and dynamic manner. To be specific, we first enhance a typical DLRS to allow various embedding dimensions; then we propose an end-to-end differentiable framework that can automatically select different embedding dimensions according to user/item popularity; finally we propose an AutoML based optimization algorithm in a streaming recommendation setting. The experimental results based on widely used benchmark datasets demonstrate the effectiveness of the AutoEmb framework.
Tasks	AutoML, Recommendation Systems
Published	2020-02-26
URL	https://arxiv.org/abs/2002.11252v2
PDF	https://arxiv.org/pdf/2002.11252v2.pdf
PWC	https://paperswithcode.com/paper/autoemb-automated-embedding-dimensionality
Repo
Framework

Improving generalisation of AutoML systems with dynamic fitness evaluations


Title	Improving generalisation of AutoML systems with dynamic fitness evaluations
Authors	Benjamin Patrick Evans, Bing Xue, Mengjie Zhang
Abstract	A common problem machine learning developers are faced with is overfitting, that is, fitting a pipeline too closely to the training data that the performance degrades for unseen data. Automated machine learning aims to free (or at least ease) the developer from the burden of pipeline creation, but this overfitting problem can persist. In fact, this can become more of a problem as we look to iteratively optimise the performance of an internal cross-validation (most often \textit{k}-fold). While this internal cross-validation hopes to reduce this overfitting, we show we can still risk overfitting to the particular folds used. In this work, we aim to remedy this problem by introducing dynamic fitness evaluations which approximate repeated \textit{k}-fold cross-validation, at little extra cost over single \textit{k}-fold, and far lower cost than typical repeated \textit{k}-fold. The results show that when time equated, the proposed fitness function results in significant improvement over the current state-of-the-art baseline method which uses an internal single \textit{k}-fold. Furthermore, the proposed extension is very simple to implement on top of existing evolutionary computation methods, and can provide essentially a free boost in generalisation/testing performance.
Tasks	AutoML
Published	2020-01-23
URL	https://arxiv.org/abs/2001.08842v1
PDF	https://arxiv.org/pdf/2001.08842v1.pdf
PWC	https://paperswithcode.com/paper/improving-generalisation-of-automl-systems
Repo
Framework

Out-of-Distribution Detection for Skin Lesion Images with Deep Isolation Forest


Title	Out-of-Distribution Detection for Skin Lesion Images with Deep Isolation Forest
Authors	Xuan Li, Yuchen Lu, Christian Desrosiers, Xue Liu
Abstract	In this paper, we study the problem of out-of-distribution detection in skin disease images. Publicly available medical datasets normally have a limited number of lesion classes (e.g. HAM10000 has 8 lesion classes). However, there exists a few thousands of clinically identified diseases. Hence, it is important if lesions not in the training data can be differentiated. Toward this goal, we propose DeepIF, a non-parametric Isolation Forest based approach combined with deep convolutional networks. We conduct comprehensive experiments to compare our DeepIF with three baseline models. Results demonstrate state-of-the-art performance of our proposed approach on the task of detecting abnormal skin lesions.
Tasks	Out-of-Distribution Detection
Published	2020-03-20
URL	https://arxiv.org/abs/2003.09365v1
PDF	https://arxiv.org/pdf/2003.09365v1.pdf
PWC	https://paperswithcode.com/paper/out-of-distribution-detection-for-skin-lesion
Repo
Framework