October 16, 2019

3302 words 16 mins read

Paper Group ANR 1020

State-Space Abstractions for Probabilistic Inference: A Systematic Review. Stochastic Variational Inference with Gradient Linearization. Efficient Metropolitan Traffic Prediction Based on Graph Recurrent Neural Network. MMLSpark: Unifying Machine Learning Ecosystems at Massive Scales. Replica Symmetry Breaking in Bipartite Spin Glasses and Neural N …

State-Space Abstractions for Probabilistic Inference: A Systematic Review


Title	State-Space Abstractions for Probabilistic Inference: A Systematic Review
Authors	Stefan Lüdtke, Max Schröder, Frank Krüger, Sebastian Bader, Thomas Kirste
Abstract	Tasks such as social network analysis, human behavior recognition, or modeling biochemical reactions, can be solved elegantly by using the probabilistic inference framework. However, standard probabilistic inference algorithms work at a propositional level, and thus cannot capture the symmetries and redundancies that are present in these tasks. Algorithms that exploit those symmetries have been devised in different research fields, for example by the lifted inference-, multiple object tracking-, and modeling and simulation-communities. The common idea, that we call state space abstraction, is to perform inference over compact representations of sets of symmetric states. Although they are concerned with a similar topic, the relationship between these approaches has not been investigated systematically. This survey provides the following contributions. We perform a systematic literature review to outline the state of the art in probabilistic inference methods exploiting symmetries. From an initial set of more than 4,000 papers, we identify 116 relevant papers. Furthermore, we provide new high-level categories that classify the approaches, based on common properties of the approaches. The research areas underlying each of the categories are introduced concisely. Researchers from different fields that are confronted with a state space explosion problem in a probabilistic system can use this classification to identify possible solutions. Finally, based on this conceptualization, we identify potentials for future research, as some relevant application domains are not addressed by current approaches.
Tasks	Multiple Object Tracking, Object Tracking
Published	2018-04-18
URL	http://arxiv.org/abs/1804.06748v3
PDF	http://arxiv.org/pdf/1804.06748v3.pdf
PWC	https://paperswithcode.com/paper/state-space-abstractions-for-probabilistic
Repo
Framework

Stochastic Variational Inference with Gradient Linearization


Title	Stochastic Variational Inference with Gradient Linearization
Authors	Tobias Plötz, Anne S. Wannenwetsch, Stefan Roth
Abstract	Variational inference has experienced a recent surge in popularity owing to stochastic approaches, which have yielded practical tools for a wide range of model classes. A key benefit is that stochastic variational inference obviates the tedious process of deriving analytical expressions for closed-form variable updates. Instead, one simply needs to derive the gradient of the log-posterior, which is often much easier. Yet for certain model classes, the log-posterior itself is difficult to optimize using standard gradient techniques. One such example are random field models, where optimization based on gradient linearization has proven popular, since it speeds up convergence significantly and can avoid poor local optima. In this paper we propose stochastic variational inference with gradient linearization (SVIGL). It is similarly convenient as standard stochastic variational inference - all that is required is a local linearization of the energy gradient. Its benefit over stochastic variational inference with conventional gradient methods is a clear improvement in convergence speed, while yielding comparable or even better variational approximations in terms of KL divergence. We demonstrate the benefits of SVIGL in three applications: Optical flow estimation, Poisson-Gaussian denoising, and 3D surface reconstruction.
Tasks	Denoising, Optical Flow Estimation
Published	2018-03-28
URL	http://arxiv.org/abs/1803.10586v1
PDF	http://arxiv.org/pdf/1803.10586v1.pdf
PWC	https://paperswithcode.com/paper/stochastic-variational-inference-with
Repo
Framework

Efficient Metropolitan Traffic Prediction Based on Graph Recurrent Neural Network


Title	Efficient Metropolitan Traffic Prediction Based on Graph Recurrent Neural Network
Authors	Xiaoyu Wang, Cailian Chen, Yang Min, Jianping He, Bo Yang, Yang Zhang
Abstract	Traffic prediction is a fundamental and vital task in Intelligence Transportation System (ITS), but it is very challenging to get high accuracy while containing low computational complexity due to the spatiotemporal characteristics of traffic flow, especially under the metropolitan circumstances. In this work, a new topological framework, called Linkage Network, is proposed to model the road networks and present the propagation patterns of traffic flow. Based on the Linkage Network model, a novel online predictor, named Graph Recurrent Neural Network (GRNN), is designed to learn the propagation patterns in the graph. It could simultaneously predict traffic flow for all road segments based on the information gathered from the whole graph, which thus reduces the computational complexity significantly from O(nm) to O(n+m), while keeping the high accuracy. Moreover, it can also predict the variations of traffic trends. Experiments based on real-world data demonstrate that the proposed method outperforms the existing prediction methods.
Tasks	Traffic Prediction
Published	2018-11-02
URL	http://arxiv.org/abs/1811.00740v1
PDF	http://arxiv.org/pdf/1811.00740v1.pdf
PWC	https://paperswithcode.com/paper/efficient-metropolitan-traffic-prediction
Repo
Framework

MMLSpark: Unifying Machine Learning Ecosystems at Massive Scales


Title	MMLSpark: Unifying Machine Learning Ecosystems at Massive Scales
Authors	Mark Hamilton, Sudarshan Raghunathan, Ilya Matiach, Andrew Schonhoffer, Anand Raman, Eli Barzilay, Karthik Rajendran, Dalitso Banda, Casey Jisoo Hong, Manon Knoertzer, Ben Brodsky, Minsoo Thigpen, Janhavi Suresh Mahajan, Courtney Cochrane, Abhiram Eswaran, Ari Green
Abstract	We introduce Microsoft Machine Learning for Apache Spark (MMLSpark), an ecosystem of enhancements that expand the Apache Spark distributed computing library to tackle problems in Deep Learning, Micro-Service Orchestration, Gradient Boosting, Model Interpretability, and other areas of modern computation. Furthermore, we present a novel system called Spark Serving that allows users to run any Apache Spark program as a distributed, sub-millisecond latency web service backed by their existing Spark Cluster. All MMLSpark contributions have the same API to enable simple composition across frameworks and usage across batch, streaming, and RESTful web serving scenarios on static, elastic, or serverless clusters. We showcase MMLSpark by creating a method for deep object detection capable of learning without human labeled data and demonstrate its effectiveness for Snow Leopard conservation.
Tasks	Object Detection
Published	2018-10-20
URL	https://arxiv.org/abs/1810.08744v2
PDF	https://arxiv.org/pdf/1810.08744v2.pdf
PWC	https://paperswithcode.com/paper/mmlspark-unifying-machine-learning-ecosystems
Repo
Framework

Replica Symmetry Breaking in Bipartite Spin Glasses and Neural Networks


Title	Replica Symmetry Breaking in Bipartite Spin Glasses and Neural Networks
Authors	Gavin Hartnett, Edward Parker, Edward Geist
Abstract	Some interesting recent advances in the theoretical understanding of neural networks have been informed by results from the physics of disordered many-body systems. Motivated by these findings, this work uses the replica technique to study the mathematically tractable bipartite Sherrington-Kirkpatrick (SK) spin glass model, which is formally similar to a Restricted Boltzmann Machine (RBM) neural network. The bipartite SK model has been previously studied assuming replica symmetry; here this assumption is relaxed and a replica symmetry breaking analysis is performed. The bipartite SK model is found to have many features in common with Parisi’s solution of the original, unipartite SK model, including the existence of a multitude of pure states which are related in a hierarchical, ultrametric fashion. As an application of this analysis, the optimal cost for a graph partitioning problem is shown to be simply related to the ground state energy of the bipartite SK model. As a second application, empirical investigations reveal that the Gibbs sampled outputs of an RBM trained on the MNIST data set are more ultrametrically distributed than the input data itself.
Tasks	graph partitioning
Published	2018-03-17
URL	http://arxiv.org/abs/1803.06442v4
PDF	http://arxiv.org/pdf/1803.06442v4.pdf
PWC	https://paperswithcode.com/paper/replica-symmetry-breaking-in-bipartite-spin
Repo
Framework

Ontology-Based Reasoning about the Trustworthiness of Cyber-Physical Systems


Title	Ontology-Based Reasoning about the Trustworthiness of Cyber-Physical Systems
Authors	Marcello Balduccini, Edward Griffor, Michael Huth, Claire Vishik, Martin Burns, David Wollman
Abstract	It has been challenging for the technical and regulatory communities to formulate requirements for trustworthiness of the cyber-physical systems (CPS) due to the complexity of the issues associated with their design, deployment, and operations. The US National Institute of Standards and Technology (NIST), through a public working group, has released a CPS Framework that adopts a broad and integrated view of CPS and positions trustworthiness among other aspects of CPS. This paper takes the model created by the CPS Framework and its further developments one step further, by applying ontological approaches and reasoning techniques in order to achieve greater understanding of CPS. The example analyzed in the paper demonstrates the enrichment of the original CPS model obtained through ontology and reasoning and its ability to deliver additional insights to the developers and operators of CPS.
Tasks
Published	2018-03-20
URL	http://arxiv.org/abs/1803.07438v1
PDF	http://arxiv.org/pdf/1803.07438v1.pdf
PWC	https://paperswithcode.com/paper/ontology-based-reasoning-about-the
Repo
Framework

Loop Closure Detection with RGB-D Feature Pyramid Siamese Networks


Title	Loop Closure Detection with RGB-D Feature Pyramid Siamese Networks
Authors	Zhang Qianhao, Alexander Mai, Joseph Menke, Allen Yang
Abstract	In visual Simultaneous Localization And Mapping (SLAM), detecting loop closures has been an important but difficult task. Currently, most solutions are based on the bag-of-words approach. Yet the possibility of deep neural network application to this task has not been fully explored due to the lack of appropriate architecture design and of sufficient training data. In this paper we demonstrate the applicability of deep neural networks by addressing both issues. Specifically we show that a feature pyramid Siamese neural network can achieve state-of-the-art performance on pairwise loop closure detection. The network is trained and tested on large-scale RGB-D datasets with a novel automatic loop closure labeling algorithm. Each image pair is labelled by how much the images overlap, allowing loop closure to be computed directly rather than by labor intensive manual labeling. We present an algorithm to adopt any large-scale generic RGB-D dataset for use in training deep loop-closure networks. We show for the first time that deep neural networks are capable of detecting loop closures, and we provide a method for generating large-scale datasets for use in evaluating and training loop closure detectors.
Tasks	Loop Closure Detection, Simultaneous Localization and Mapping
Published	2018-11-25
URL	http://arxiv.org/abs/1811.09938v1
PDF	http://arxiv.org/pdf/1811.09938v1.pdf
PWC	https://paperswithcode.com/paper/loop-closure-detection-with-rgb-d-feature
Repo
Framework

How much does a word weigh? Weighting word embeddings for word sense induction


Title	How much does a word weigh? Weighting word embeddings for word sense induction
Authors	Nikolay Arefyev, Pavel Ermolaev, Alexander Panchenko
Abstract	The paper describes our participation in the first shared task on word sense induction and disambiguation for the Russian language RUSSE’2018 (Panchenko et al., 2018). For each of several dozens of ambiguous words, the participants were asked to group text fragments containing it according to the senses of this word, which were not provided beforehand, therefore the “induction” part of the task. For instance, a word “bank” and a set of text fragments (also known as “contexts”) in which this word occurs, e.g. “bank is a financial institution that accepts deposits” and “river bank is a slope beside a body of water” were given. A participant was asked to cluster such contexts in the unknown in advance number of clusters corresponding to, in this case, the “company” and the “area” senses of the word “bank”. The organizers proposed three evaluation datasets of varying complexity and text genres based respectively on texts of Wikipedia, Web pages, and a dictionary of the Russian language. We present two experiments: a positive and a negative one, based respectively on clustering of contexts represented as a weighted average of word embeddings and on machine translation using two state-of-the-art production neural machine translation systems. Our team showed the second best result on two datasets and the third best result on the remaining one dataset among 18 participating teams. We managed to substantially outperform competitive state-of-the-art baselines from the previous years based on sense embeddings.
Tasks	Machine Translation, Word Embeddings, Word Sense Induction
Published	2018-05-23
URL	http://arxiv.org/abs/1805.09209v2
PDF	http://arxiv.org/pdf/1805.09209v2.pdf
PWC	https://paperswithcode.com/paper/how-much-does-a-word-weigh-weighting-word
Repo
Framework

Data analysis from empirical moments and the Christoffel function


Title	Data analysis from empirical moments and the Christoffel function
Authors	Edouard Pauwels, Mihai Putinar, Jean-Bernard Lasserre
Abstract	Spectral features of the empirical moment matrix constitute a resourceful tool for unveiling properties of a cloud of points, among which, density, support and latent structures. It is already well known that the empirical moment matrix encodes a great deal of subtle attributes of the underlying measure. Starting from this object as base of observations we combine ideas from statistics, real algebraic geometry, orthogonal polynomials and approximation theory for opening new insights relevant for Machine Learning (ML) problems with data supported on singular sets. Refined concepts and results from real algebraic geometry and approximation theory are empowering a simple tool (the empirical moment matrix) for the task of solving non-trivial questions in data analysis. We provide (1) theoretical support, (2) numerical experiments and, (3) connections to real world data as a validation of the stamina of the empirical moment matrix approach.
Tasks
Published	2018-10-19
URL	https://arxiv.org/abs/1810.08480v2
PDF	https://arxiv.org/pdf/1810.08480v2.pdf
PWC	https://paperswithcode.com/paper/data-analysis-from-empirical-moments-and-the
Repo
Framework

Beyond Backprop: Online Alternating Minimization with Auxiliary Variables


Title	Beyond Backprop: Online Alternating Minimization with Auxiliary Variables
Authors	Anna Choromanska, Benjamin Cowen, Sadhana Kumaravel, Ronny Luss, Mattia Rigotti, Irina Rish, Brian Kingsbury, Paolo DiAchille, Viatcheslav Gurev, Ravi Tejwani, Djallel Bouneffouf
Abstract	Despite significant recent advances in deep neural networks, training them remains a challenge due to the highly non-convex nature of the objective function. State-of-the-art methods rely on error backpropagation, which suffers from several well-known issues, such as vanishing and exploding gradients, inability to handle non-differentiable nonlinearities and to parallelize weight-updates across layers, and biological implausibility. These limitations continue to motivate exploration of alternative training algorithms, including several recently proposed auxiliary-variable methods which break the complex nested objective function into local subproblems. However, those techniques are mainly offline (batch), which limits their applicability to extremely large datasets, as well as to online, continual or reinforcement learning. The main contribution of our work is a novel online (stochastic/mini-batch) alternating minimization (AM) approach for training deep neural networks, together with the first theoretical convergence guarantees for AM in stochastic settings and promising empirical results on a variety of architectures and datasets.
Tasks
Published	2018-06-24
URL	https://arxiv.org/abs/1806.09077v4
PDF	https://arxiv.org/pdf/1806.09077v4.pdf
PWC	https://paperswithcode.com/paper/beyond-backprop-online-alternating
Repo
Framework

Detecting Small, Densely Distributed Objects with Filter-Amplifier Networks and Loss Boosting


Title	Detecting Small, Densely Distributed Objects with Filter-Amplifier Networks and Loss Boosting
Authors	Zhenhua Chen, David Crandall, Robert Templeman
Abstract	Detecting small, densely distributed objects is a significant challenge: small objects often contain less distinctive information compared to larger ones, and finer-grained precision of bounding box boundaries are required. In this paper, we propose two techniques for addressing this problem. First, we estimate the likelihood that each pixel belongs to an object boundary rather than predicting coordinates of bounding boxes (as YOLO, Faster-RCNN and SSD do), by proposing a new architecture called Filter-Amplifier Networks (FANs). Second, we introduce a technique called Loss Boosting (LB) which attempts to soften the loss imbalance problem on each image. We test our algorithm on the problem of detecting electrical components on a new, realistic, diverse dataset of printed circuit boards (PCBs), as well as the problem of detecting vehicles in the Vehicle Detection in Aerial Imagery (VEDAI) dataset. Experiments show that our method works significantly better than current state-of-the-art algorithms with respect to accuracy, recall and average IoU.
Tasks
Published	2018-02-21
URL	http://arxiv.org/abs/1802.07845v2
PDF	http://arxiv.org/pdf/1802.07845v2.pdf
PWC	https://paperswithcode.com/paper/detecting-small-densely-distributed-objects
Repo
Framework

Automated vehicle’s behavior decision making using deep reinforcement learning and high-fidelity simulation environment


Title	Automated vehicle’s behavior decision making using deep reinforcement learning and high-fidelity simulation environment
Authors	Yingjun Ye, Xiaohui Zhang, Jian Sun
Abstract	Automated vehicles are deemed to be the key element for the intelligent transportation system in the future. Many studies have been made to improve the Automated vehicles’ ability of environment recognition and vehicle control, while the attention paid to decision making is not enough though the decision algorithms so far are very preliminary. Therefore, a framework of the decision-making training and learning is put forward in this paper. It consists of two parts: the deep reinforcement learning training program and the high-fidelity virtual simulation environment. Then the basic microscopic behavior, car-following, is trained within this framework. In addition, theoretical analysis and experiments were conducted on setting reward function for accelerating training using deep reinforcement learning. The results show that on the premise of driving comfort, the efficiency of the trained Automated vehicle increases 7.9% compared to the classical traffic model, intelligent driver model. Later on, on a more complex three-lane section, we trained the integrated model combines both car-following and lane-changing behavior, the average speed further grows 2.4%. It indicates that our framework is effective for Automated vehicle’s decision-making learning.
Tasks	Decision Making
Published	2018-04-17
URL	http://arxiv.org/abs/1804.06264v1
PDF	http://arxiv.org/pdf/1804.06264v1.pdf
PWC	https://paperswithcode.com/paper/automated-vehicles-behavior-decision-making
Repo
Framework

Noise Statistics Oblivious GARD For Robust Regression With Sparse Outliers


Title	Noise Statistics Oblivious GARD For Robust Regression With Sparse Outliers
Authors	Sreejith Kallummil, Sheetal Kalyani
Abstract	Linear regression models contaminated by Gaussian noise (inlier) and possibly unbounded sparse outliers are common in many signal processing applications. Sparse recovery inspired robust regression (SRIRR) techniques are shown to deliver high quality estimation performance in such regression models. Unfortunately, most SRIRR techniques assume \textit{a priori} knowledge of noise statistics like inlier noise variance or outlier statistics like number of outliers. Both inlier and outlier noise statistics are rarely known \textit{a priori} and this limits the efficient operation of many SRIRR algorithms. This article proposes a novel noise statistics oblivious algorithm called residual ratio thresholding GARD (RRT-GARD) for robust regression in the presence of sparse outliers. RRT-GARD is developed by modifying the recently proposed noise statistics dependent greedy algorithm for robust de-noising (GARD). Both finite sample and asymptotic analytical results indicate that RRT-GARD performs nearly similar to GARD with \textit{a priori} knowledge of noise statistics. Numerical simulations in real and synthetic data sets also point to the highly competitive performance of RRT-GARD.
Tasks
Published	2018-09-19
URL	http://arxiv.org/abs/1809.07222v1
PDF	http://arxiv.org/pdf/1809.07222v1.pdf
PWC	https://paperswithcode.com/paper/noise-statistics-oblivious-gard-for-robust
Repo
Framework

Multi-Instance Dynamic Ordinal Random Fields for Weakly-supervised Facial Behavior Analysis


Title	Multi-Instance Dynamic Ordinal Random Fields for Weakly-supervised Facial Behavior Analysis
Authors	Adria Ruiz, Ognjen Rudovic, Xavier Binefa, Maja Pantic
Abstract	We propose a Multi-Instance-Learning (MIL) approach for weakly-supervised learning problems, where a training set is formed by bags (sets of feature vectors or instances) and only labels at bag-level are provided. Specifically, we consider the Multi-Instance Dynamic-Ordinal-Regression (MI-DOR) setting, where the instance labels are naturally represented as ordinal variables and bags are structured as temporal sequences. To this end, we propose Multi-Instance Dynamic Ordinal Random Fields (MI-DORF). In this framework, we treat instance-labels as temporally-dependent latent variables in an Undirected Graphical Model. Different MIL assumptions are modelled via newly introduced high-order potentials relating bag and instance-labels within the energy function of the model. We also extend our framework to address the Partially-Observed MI-DOR problems, where a subset of instance labels are available during training. We show on the tasks of weakly-supervised facial behavior analysis, Facial Action Unit (DISFA dataset) and Pain (UNBC dataset) Intensity estimation, that the proposed framework outperforms alternative learning approaches. Furthermore, we show that MIDORF can be employed to reduce the data annotation efforts in this context by large-scale.
Tasks
Published	2018-03-01
URL	http://arxiv.org/abs/1803.00907v1
PDF	http://arxiv.org/pdf/1803.00907v1.pdf
PWC	https://paperswithcode.com/paper/multi-instance-dynamic-ordinal-random-fields-1
Repo
Framework

A GPU-Oriented Algorithm Design for Secant-Based Dimensionality Reduction


Title	A GPU-Oriented Algorithm Design for Secant-Based Dimensionality Reduction
Authors	Henry Kvinge, Elin Farnell, Michael Kirby, Chris Peterson
Abstract	Dimensionality-reduction techniques are a fundamental tool for extracting useful information from high-dimensional data sets. Because secant sets encode manifold geometry, they are a useful tool for designing meaningful data-reduction algorithms. In one such approach, the goal is to construct a projection that maximally avoids secant directions and hence ensures that distinct data points are not mapped too close together in the reduced space. This type of algorithm is based on a mathematical framework inspired by the constructive proof of Whitney’s embedding theorem from differential topology. Computing all (unit) secants for a set of points is by nature computationally expensive, thus opening the door for exploitation of GPU architecture for achieving fast versions of these algorithms. We present a polynomial-time data-reduction algorithm that produces a meaningful low-dimensional representation of a data set by iteratively constructing improved projections within the framework described above. Key to our algorithm design and implementation is the use of GPUs which, among other things, minimizes the computational time required for the calculation of all secant lines. One goal of this report is to share ideas with GPU experts and to discuss a class of mathematical algorithms that may be of interest to the broader GPU community.
Tasks	Dimensionality Reduction
Published	2018-07-10
URL	http://arxiv.org/abs/1807.03425v1
PDF	http://arxiv.org/pdf/1807.03425v1.pdf
PWC	https://paperswithcode.com/paper/a-gpu-oriented-algorithm-design-for-secant
Repo
Framework