January 31, 2020

3158 words 15 mins read

Paper Group ANR 84

Data-driven Discovery of Emergent Behaviors in Collective Dynamics. A New Expert Questioning Approach to More Efficient Fault Localization in Ontologies. A Case for the Score: Identifying Image Anomalies using Variational Autoencoder Gradients. RLTM: An Efficient Neural IR Framework for Long Documents. Reasoning About Human-Object Interactions Thro …

Data-driven Discovery of Emergent Behaviors in Collective Dynamics


Title	Data-driven Discovery of Emergent Behaviors in Collective Dynamics
Authors	Mauro Maggioni, Jason Miller, Ming Zhong
Abstract	Particle- and agent-based systems are a ubiquitous modeling tool in many disciplines. We consider the fundamental problem of inferring interaction kernels from observations of agent-based dynamical systems given observations of trajectories, in particular for collective dynamical systems exhibiting emergent behaviors with complicated interaction kernels, in a nonparametric fashion, and for kernels which are parametrized by a single unknown parameter. We extend the estimators introduced in \cite{PNASLU}, which are based on suitably regularized least squares estimators, to these larger classes of systems. We provide extensive numerical evidence that the estimators provide faithful approximations to the interaction kernels, and provide accurate predictions for trajectories started at new initial conditions, both throughout the training'' time interval in which the observations were made, and often much beyond. We demonstrate these features on prototypical systems displaying collective behaviors, ranging from opinion dynamics, flocking dynamics, self-propelling particle dynamics, synchronized oscillator dynamics, and a gravitational system. Our experiments also suggest that our estimated systems can display the same emergent behaviors of the observed systems, that occur at larger timescales than those used in the training data. Finally, in the case of families of systems governed by a parameterized family of interaction kernels, we introduce novel estimators that estimate the parameterized family of kernels, splitting it into a common interaction kernel and the action of parameters. We demonstrate this in the case of gravity, by learning both the common component’’ $1/r^2$ and the dependency on mass, without any a priori knowledge of either one, from observations of planetary motions in our solar system.
Tasks
Published	2019-12-23
URL	https://arxiv.org/abs/1912.11123v2
PDF	https://arxiv.org/pdf/1912.11123v2.pdf
PWC	https://paperswithcode.com/paper/data-driven-discovery-of-emergent-behaviors
Repo
Framework

A New Expert Questioning Approach to More Efficient Fault Localization in Ontologies


Title	A New Expert Questioning Approach to More Efficient Fault Localization in Ontologies
Authors	Patrick Rodler, Michael Eichholzer
Abstract	When ontologies reach a certain size and complexity, faults such as inconsistencies, unsatisfiable classes or wrong entailments are hardly avoidable. Locating the incorrect axioms that cause these faults is a hard and time-consuming task. Addressing this issue, several techniques for semi-automatic fault localization in ontologies have been proposed. Often, these approaches involve a human expert who provides answers to system-generated questions about the intended (correct) ontology in order to reduce the possible fault locations. To suggest as informative questions as possible, existing methods draw on various algorithmic optimizations as well as heuristics. However, these computations are often based on certain assumptions about the interacting user. In this work, we characterize and discuss different user types and show that existing approaches do not achieve optimal efficiency for all of them. As a remedy, we suggest a new type of expert question which aims at fitting the answering behavior of all analyzed experts. Moreover, we present an algorithm to optimize this new query type which is fully compatible with the (tried and tested) heuristics used in the field. Experiments on faulty real-world ontologies show the potential of the new querying method for minimizing the expert consultation time, independent of the expert type. Besides, the gained insights can inform the design of interactive debugging tools towards better meeting their users’ needs.
Tasks
Published	2019-03-31
URL	http://arxiv.org/abs/1904.00317v1
PDF	http://arxiv.org/pdf/1904.00317v1.pdf
PWC	https://paperswithcode.com/paper/a-new-expert-questioning-approach-to-more
Repo
Framework

A Case for the Score: Identifying Image Anomalies using Variational Autoencoder Gradients


Title	A Case for the Score: Identifying Image Anomalies using Variational Autoencoder Gradients
Authors	David Zimmerer, Jens Petersen, Simon A. A. Kohl, Klaus H. Maier-Hein
Abstract	Through training on unlabeled data, anomaly detection has the potential to impact computer-aided diagnosis by outlining suspicious regions. Previous work on deep-learning-based anomaly detection has primarily focused on the reconstruction error. We argue instead, that pixel-wise anomaly ratings derived from a Variational Autoencoder based score approximation yield a theoretically better grounded and more faithful estimate. In our experiments, Variational Autoencoder gradient-based rating outperforms other approaches on unsupervised pixel-wise tumor detection on the BraTS-2017 dataset with a ROC-AUC of 0.94.
Tasks	Anomaly Detection
Published	2019-11-28
URL	https://arxiv.org/abs/1912.00003v1
PDF	https://arxiv.org/pdf/1912.00003v1.pdf
PWC	https://paperswithcode.com/paper/a-case-for-the-score-identifying-image
Repo
Framework

RLTM: An Efficient Neural IR Framework for Long Documents


Title	RLTM: An Efficient Neural IR Framework for Long Documents
Authors	Chen Zheng, Yu Sun, Shengxian Wan, Dianhai Yu
Abstract	Deep neural networks have achieved significant improvements in information retrieval (IR). However, most existing models are computational costly and can not efficiently scale to long documents. This paper proposes a novel End-to-End neural ranking framework called Reinforced Long Text Matching (RLTM) which matches a query with long documents efficiently and effectively. The core idea behind the framework can be analogous to the human judgment process which firstly locates the relevance parts quickly from the whole document and then matches these parts with the query carefully to obtain the final label. Firstly, we select relevant sentences from the long documents by a coarse and efficient matching model. Secondly, we generate a relevance score by a more sophisticated matching model based on the sentence selected. The whole model is trained jointly with reinforcement learning in a pairwise manner by maximizing the expected score gaps between positive and negative examples. Experimental results demonstrate that RLTM has greatly improved the efficiency and effectiveness of the state-of-the-art models.
Tasks	Information Retrieval, Text Matching
Published	2019-06-22
URL	https://arxiv.org/abs/1906.09404v2
PDF	https://arxiv.org/pdf/1906.09404v2.pdf
PWC	https://paperswithcode.com/paper/rltm-an-efficient-neural-ir-framework-for
Repo
Framework

Reasoning About Human-Object Interactions Through Dual Attention Networks


Title	Reasoning About Human-Object Interactions Through Dual Attention Networks
Authors	Tete Xiao, Quanfu Fan, Dan Gutfreund, Mathew Monfort, Aude Oliva, Bolei Zhou
Abstract	Objects are entities we act upon, where the functionality of an object is determined by how we interact with it. In this work we propose a Dual Attention Network model which reasons about human-object interactions. The dual-attentional framework weights the important features for objects and actions respectively. As a result, the recognition of objects and actions mutually benefit each other. The proposed model shows competitive classification performance on the human-object interaction dataset Something-Something. Besides, it can perform weak spatiotemporal localization and affordance segmentation, despite being trained only with video-level labels. The model not only finds when an action is happening and which object is being manipulated, but also identifies which part of the object is being interacted with. Project page: \url{https://dual-attention-network.github.io/}.
Tasks	Human-Object Interaction Detection
Published	2019-09-10
URL	https://arxiv.org/abs/1909.04743v1
PDF	https://arxiv.org/pdf/1909.04743v1.pdf
PWC	https://paperswithcode.com/paper/reasoning-about-human-object-interactions
Repo
Framework

Position-aware Graph Neural Networks


Title	Position-aware Graph Neural Networks
Authors	Jiaxuan You, Rex Ying, Jure Leskovec
Abstract	Learning node embeddings that capture a node’s position within the broader graph structure is crucial for many prediction tasks on graphs. However, existing Graph Neural Network (GNN) architectures have limited power in capturing the position/location of a given node with respect to all other nodes of the graph. Here we propose Position-aware Graph Neural Networks (P-GNNs), a new class of GNNs for computing position-aware node embeddings. P-GNN first samples sets of anchor nodes, computes the distance of a given target node to each anchor-set,and then learns a non-linear distance-weighted aggregation scheme over the anchor-sets. This way P-GNNs can capture positions/locations of nodes with respect to the anchor nodes. P-GNNs have several advantages: they are inductive, scalable,and can incorporate node feature information. We apply P-GNNs to multiple prediction tasks including link prediction and community detection. We show that P-GNNs consistently outperform state of the art GNNs, with up to 66% improvement in terms of the ROC AUC score.
Tasks	Community Detection, Link Prediction
Published	2019-06-11
URL	https://arxiv.org/abs/1906.04817v2
PDF	https://arxiv.org/pdf/1906.04817v2.pdf
PWC	https://paperswithcode.com/paper/position-aware-graph-neural-networks
Repo
Framework

On the well-posedness of Bayesian inverse problems


Title	On the well-posedness of Bayesian inverse problems
Authors	Jonas Latz
Abstract	The subject of this article is the introduction of a new concept of well-posedness of Bayesian inverse problems. The conventional concept of (Lipschitz, Hellinger) well-posedness in [Stuart 2010, Acta Numerica 19, pp. 451-559] is difficult to verify in practice and may be inappropriate in some contexts. Our concept simply replaces the Lipschitz continuity of the posterior measure in the Hellinger distance by continuity in an appropriate distance between probability measures. Aside from the Hellinger distance, we investigate well-posedness with respect to weak convergence, the total variation distance, the Wasserstein distance, and also the Kullback–Leibler divergence. We demonstrate that the weakening to continuity is tolerable and that the generalisation to other distances is important. The main results of this article are proofs of well-posedness with respect to some of the aforementioned distances for large classes of Bayesian inverse problems. Here, little or no information about the underlying model is necessary; making these results particularly interesting for practitioners using black-box models. We illustrate our findings with numerical examples motivated from machine learning and image processing.
Tasks
Published	2019-02-26
URL	https://arxiv.org/abs/1902.10257v4
PDF	https://arxiv.org/pdf/1902.10257v4.pdf
PWC	https://paperswithcode.com/paper/on-the-well-posedness-of-bayesian-inverse
Repo
Framework

Qualitative Numeric Planning: Reductions and Complexity


Title	Qualitative Numeric Planning: Reductions and Complexity
Authors	Blai Bonet, Hector Geffner
Abstract	Qualitative numerical planning is classical planning extended with non-negative real variables that can be increased or decreased “qualitatively”, i.e., by positive random amounts. While deterministic planning with numerical variables is undecidable in general, qualitative numerical planning is decidable and provides a convenient abstract model for generalized planning. Qualitative numerical planning, introduced by Srivastava, Zilberstein, Immerman, and Geffner (2011), showed that solutions to qualitative numerical problems (QNPs) correspond to the strong cyclic solutions of an associated fully observable non-deterministic (FOND) problem that terminate. The approach leads to a generate-and-test algorithm for solving QNPs where solutions to a FOND problem are generated one by one and tested for termination. The computational shortcomings of this approach, however, are that it is not simple to amend FOND planners to generate all solutions, and that the number of solutions to check can be doubly exponential in the number of variables. In this work we address these limitations, while providing additional insights on QNPs. More precisely, we introduce two reductions, one from QNPs to FOND problems and the other from FOND problems to QNPs both of which do not involve termination tests. A result of these reductions is that QNPs are shown to have the same expressive power and the same complexity as FOND problems.
Tasks
Published	2019-12-10
URL	https://arxiv.org/abs/1912.04816v1
PDF	https://arxiv.org/pdf/1912.04816v1.pdf
PWC	https://paperswithcode.com/paper/qualitative-numeric-planning-reductions-and
Repo
Framework

Policy Optimization for $\mathcal{H}2$ Linear Control with $\mathcal{H}\infty$ Robustness Guarantee: Implicit Regularization and Global Convergence


Title	Policy Optimization for $\mathcal{H}2$ Linear Control with $\mathcal{H}\infty$ Robustness Guarantee: Implicit Regularization and Global Convergence
Authors	Kaiqing Zhang, Bin Hu, Tamer Başar
Abstract	Policy optimization (PO) is a key ingredient for reinforcement learning (RL). For control design, certain constraints are usually enforced on the policies to optimize, accounting for either the stability, robustness, or safety concerns on the system. Hence, PO is by nature a constrained (nonconvex) optimization in most cases, whose global convergence is challenging to analyze in general. More importantly, some constraints that are safety-critical, e.g., the $\mathcal{H}\infty$-norm constraint that guarantees the system robustness, are difficult to enforce as the PO methods proceed. Recently, policy gradient methods have been shown to converge to the global optimum of linear quadratic regulator (LQR), a classical optimal control problem, without regularizing/projecting the control iterates onto the stabilizing set (Fazel et al., 2018), its (implicit) feasible set. This striking result is built upon the coercive property of the cost, ensuring that the iterates remain feasible as the cost decreases. In this paper, we study the convergence theory of PO for $\mathcal{H}2$ linear control with $\mathcal{H}\infty$-norm robustness guarantee. One significant new feature of this problem is the lack of coercivity, i.e., the cost may have finite value around the feasible set boundary, breaking the existing analysis for LQR. Interestingly, we show that two PO methods enjoy the implicit regularization property, i.e., the iterates preserve the $\mathcal{H}\infty$ robustness constraint as if they are regularized by the algorithms. Furthermore, convergence to the globally optimal policies with globally sublinear and locally (super-)linear rates are provided under certain conditions, despite the nonconvexity of the problem. To the best of our knowledge, our work offers the first results on the implicit regularization property and global convergence of PO methods for robust/risk-sensitive control.
Tasks	Policy Gradient Methods
Published	2019-10-21
URL	https://arxiv.org/abs/1910.09496v2
PDF	https://arxiv.org/pdf/1910.09496v2.pdf
PWC	https://paperswithcode.com/paper/policy-optimization-for-mathcalh_2-linear
Repo
Framework

The Discrete Langevin Machine: Bridging the Gap Between Thermodynamic and Neuromorphic Systems


Title	The Discrete Langevin Machine: Bridging the Gap Between Thermodynamic and Neuromorphic Systems
Authors	Lukas Kades, Jan M. Pawlowski
Abstract	A formulation of Langevin dynamics for discrete systems is derived as a new class of generic stochastic processes. The dynamics simplify for a two-state system and suggest a novel network architecture which is implemented by the Langevin machine. The Langevin machine represents a promising approach to compute successfully quantitative exact results of Boltzmann distributed systems by LIF neurons. Besides a detailed introduction of the new dynamics, different simplified models of a neuromorphic hardware system are studied with respect to a control of emerging sources of errors.
Tasks
Published	2019-01-16
URL	http://arxiv.org/abs/1901.05214v2
PDF	http://arxiv.org/pdf/1901.05214v2.pdf
PWC	https://paperswithcode.com/paper/the-discrete-langevin-machine-bridging-the
Repo
Framework

Characterizing Bias in Classifiers using Generative Models


Title	Characterizing Bias in Classifiers using Generative Models
Authors	Daniel McDuff, Shuang Ma, Yale Song, Ashish Kapoor
Abstract	Models that are learned from real-world data are often biased because the data used to train them is biased. This can propagate systemic human biases that exist and ultimately lead to inequitable treatment of people, especially minorities. To characterize bias in learned classifiers, existing approaches rely on human oracles labeling real-world examples to identify the “blind spots” of the classifiers; these are ultimately limited due to the human labor required and the finite nature of existing image examples. We propose a simulation-based approach for interrogating classifiers using generative adversarial models in a systematic manner. We incorporate a progressive conditional generative model for synthesizing photo-realistic facial images and Bayesian Optimization for an efficient interrogation of independent facial image classification systems. We show how this approach can be used to efficiently characterize racial and gender biases in commercial systems.
Tasks	Image Classification
Published	2019-05-30
URL	https://arxiv.org/abs/1906.11891v1
PDF	https://arxiv.org/pdf/1906.11891v1.pdf
PWC	https://paperswithcode.com/paper/characterizing-bias-in-classifiers-using
Repo
Framework

DensePhysNet: Learning Dense Physical Object Representations via Multi-step Dynamic Interactions


Title	DensePhysNet: Learning Dense Physical Object Representations via Multi-step Dynamic Interactions
Authors	Zhenjia Xu, Jiajun Wu, Andy Zeng, Joshua B. Tenenbaum, Shuran Song
Abstract	We study the problem of learning physical object representations for robot manipulation. Understanding object physics is critical for successful object manipulation, but also challenging because physical object properties can rarely be inferred from the object’s static appearance. In this paper, we propose DensePhysNet, a system that actively executes a sequence of dynamic interactions (e.g., sliding and colliding), and uses a deep predictive model over its visual observations to learn dense, pixel-wise representations that reflect the physical properties of observed objects. Our experiments in both simulation and real settings demonstrate that the learned representations carry rich physical information, and can directly be used to decode physical object properties such as friction and mass. The use of dense representation enables DensePhysNet to generalize well to novel scenes with more objects than in training. With knowledge of object physics, the learned representation also leads to more accurate and efficient manipulation in downstream tasks than the state-of-the-art.
Tasks
Published	2019-06-10
URL	https://arxiv.org/abs/1906.03853v2
PDF	https://arxiv.org/pdf/1906.03853v2.pdf
PWC	https://paperswithcode.com/paper/densephysnet-learning-dense-physical-object
Repo
Framework

Self-Attention Transducers for End-to-End Speech Recognition


Title	Self-Attention Transducers for End-to-End Speech Recognition
Authors	Zhengkun Tian, Jiangyan Yi, Jianhua Tao, Ye Bai, Zhengqi Wen
Abstract	Recurrent neural network transducers (RNN-T) have been successfully applied in end-to-end speech recognition. However, the recurrent structure makes it difficult for parallelization . In this paper, we propose a self-attention transducer (SA-T) for speech recognition. RNNs are replaced with self-attention blocks, which are powerful to model long-term dependencies inside sequences and able to be efficiently parallelized. Furthermore, a path-aware regularization is proposed to assist SA-T to learn alignments and improve the performance. Additionally, a chunk-flow mechanism is utilized to achieve online decoding. All experiments are conducted on a Mandarin Chinese dataset AISHELL-1. The results demonstrate that our proposed approach achieves a 21.3% relative reduction in character error rate compared with the baseline RNN-T. In addition, the SA-T with chunk-flow mechanism can perform online decoding with only a little degradation of the performance.
Tasks	End-To-End Speech Recognition, Speech Recognition
Published	2019-09-28
URL	https://arxiv.org/abs/1909.13037v1
PDF	https://arxiv.org/pdf/1909.13037v1.pdf
PWC	https://paperswithcode.com/paper/self-attention-transducers-for-end-to-end
Repo
Framework

Neurons Activation Visualization and Information Theoretic Analysis


Title	Neurons Activation Visualization and Information Theoretic Analysis
Authors	Longwei Wang, Peijie Chen
Abstract	Understanding the inner working mechanism of deep neural networks (DNNs) is essential and important for researchers to design and improve the performance of DNNs. In this work, the entropy analysis is leveraged to study the neurons activation behavior of the fully connected layers of DNNs. The entropy of the activation patterns of each layer can provide a performance metric for the evaluation of the network model accuracy. The study is conducted based on a well trained network model. The activation patterns of shallow and deep layers of the fully connected layers are analyzed by inputting the images of a single class. It is found that for the well trained deep neural networks model, the entropy of the neuron activation pattern is monotonically reduced with the depth of the layers. That is, the neuron activation patterns become more and more stable with the depth of the fully connected layers. The entropy pattern of the fully connected layers can also provide guidelines as to how many fully connected layers are needed to guarantee the accuracy of the model. The study in this work provides a new perspective on the analysis of DNN, which shows some interesting results.
Tasks
Published	2019-05-14
URL	https://arxiv.org/abs/1905.08618v3
PDF	https://arxiv.org/pdf/1905.08618v3.pdf
PWC	https://paperswithcode.com/paper/190508618
Repo
Framework

Binarized Collaborative Filtering with Distilling Graph Convolutional Networks


Title	Binarized Collaborative Filtering with Distilling Graph Convolutional Networks
Authors	Haoyu Wang, Defu Lian, Yong Ge
Abstract	The efficiency of top-K item recommendation based on implicit feedback are vital to recommender systems in real world, but it is very challenging due to the lack of negative samples and the large number of candidate items. To address the challenges, we firstly introduce an improved Graph Convolutional Network~(GCN) model with high-order feature interaction considered. Then we distill the ranking information derived from GCN into binarized collaborative filtering, which makes use of binary representation to improve the efficiency of online recommendation. However, binary codes are not only hard to be optimized but also likely to incur the loss of information during the training processing. Therefore, we propose a novel framework to convert the binary constrained optimization problem into an equivalent continuous optimization problem with a stochastic penalty. The binarized collaborative filtering model is then easily optimized by many popular solvers like SGD and Adam. The proposed algorithm is finally evaluated on three real-world datasets and shown the superiority to the competing baselines.
Tasks	Recommendation Systems
Published	2019-06-05
URL	https://arxiv.org/abs/1906.01829v1
PDF	https://arxiv.org/pdf/1906.01829v1.pdf
PWC	https://paperswithcode.com/paper/binarized-collaborative-filtering-with
Repo
Framework