April 2, 2020

2952 words 14 mins read

Paper Group ANR 383

EPINE: Enhanced Proximity Information Network Embedding. A Relaxed Inertial Forward-Backward-Forward Algorithm for Solving Monotone Inclusions with Application to GANs. Towards Effective Human-AI Collaboration in GUI-Based Interactive Task Learning Agents. Self-Supervised Graph Representation Learning via Global Context Prediction. Robust Gaussian …

EPINE: Enhanced Proximity Information Network Embedding


Title	EPINE: Enhanced Proximity Information Network Embedding
Authors	Luoyi Zhang, Ming Xu
Abstract	Unsupervised homogeneous network embedding (NE) represents every vertex of networks into a low-dimensional vector and meanwhile preserves the network information. Adjacency matrices retain most of the network information, and directly charactrize the first-order proximity. In this work, we devote to mining valuable information in adjacency matrices at a deeper level. Under the same objective, many NE methods calculate high-order proximity by the powers of adjacency matrices, which is not accurate and well-designed enough. Instead, we propose to redefine high-order proximity in a more intuitive manner. Besides, we design a novel algorithm for calculation, which alleviates the scalability problem in the field of accurate calculation for high-order proximity. Comprehensive experiments on real-world network datasets demonstrate the effectiveness of our method in downstream machine learning tasks such as network reconstruction, link prediction and node classification.
Tasks	Link Prediction, Network Embedding, Node Classification
Published	2020-03-04
URL	https://arxiv.org/abs/2003.02689v1
PDF	https://arxiv.org/pdf/2003.02689v1.pdf
PWC	https://paperswithcode.com/paper/epine-enhanced-proximity-information-network
Repo
Framework

A Relaxed Inertial Forward-Backward-Forward Algorithm for Solving Monotone Inclusions with Application to GANs


Title	A Relaxed Inertial Forward-Backward-Forward Algorithm for Solving Monotone Inclusions with Application to GANs
Authors	Radu Ioan Bot, Michael Sedlmayer, Phan Tu Vuong
Abstract	We introduce a relaxed inertial forward-backward-forward (RIFBF) splitting algorithm for approaching the set of zeros of the sum of a maximally monotone operator and a single-valued monotone and Lipschitz continuous operator. This work aims to extend Tseng’s forward-backward-forward method by both using inertial effects as well as relaxation parameters. We formulate first a second order dynamical system which approaches the solution set of the monotone inclusion problem to be solved and provide an asymptotic analysis for its trajectories. We provide for RIFBF, which follows by explicit time discretization, a convergence analysis in the general monotone case as well as when applied to the solving of pseudo-monotone variational inequalities. We illustrate the proposed method by applications to a bilinear saddle point problem, in the context of which we also emphasize the interplay between the inertial and the relaxation parameters, and to the training of Generative Adversarial Networks (GANs).
Tasks
Published	2020-03-17
URL	https://arxiv.org/abs/2003.07886v2
PDF	https://arxiv.org/pdf/2003.07886v2.pdf
PWC	https://paperswithcode.com/paper/a-relaxed-inertial-forward-backward-forward
Repo
Framework

Towards Effective Human-AI Collaboration in GUI-Based Interactive Task Learning Agents


Title	Towards Effective Human-AI Collaboration in GUI-Based Interactive Task Learning Agents
Authors	Toby Jia-Jun Li, Jingya Chen, Tom M. Mitchell, Brad A. Myers
Abstract	We argue that a key challenge in enabling usable and useful interactive task learning for intelligent agents is to facilitate effective Human-AI collaboration. We reflect on our past 5 years of efforts on designing, developing and studying the SUGILITE system, discuss the issues on incorporating recent advances in AI with HCI principles in mixed-initiative interactions and multi-modal interactions, and summarize the lessons we learned. Lastly, we identify several challenges and opportunities, and describe our ongoing work
Tasks
Published	2020-03-05
URL	https://arxiv.org/abs/2003.02622v1
PDF	https://arxiv.org/pdf/2003.02622v1.pdf
PWC	https://paperswithcode.com/paper/towards-effective-human-ai-collaboration-in
Repo
Framework

Self-Supervised Graph Representation Learning via Global Context Prediction


Title	Self-Supervised Graph Representation Learning via Global Context Prediction
Authors	Zhen Peng, Yixiang Dong, Minnan Luo, Xiao-Ming Wu, Qinghua Zheng
Abstract	To take full advantage of fast-growing unlabeled networked data, this paper introduces a novel self-supervised strategy for graph representation learning by exploiting natural supervision provided by the data itself. Inspired by human social behavior, we assume that the global context of each node is composed of all nodes in the graph since two arbitrary entities in a connected network could interact with each other via paths of varying length. Based on this, we investigate whether the global context can be a source of free and effective supervisory signals for learning useful node representations. Specifically, we randomly select pairs of nodes in a graph and train a well-designed neural net to predict the contextual position of one node relative to the other. Our underlying hypothesis is that the representations learned from such within-graph context would capture the global topology of the graph and finely characterize the similarity and differentiation between nodes, which is conducive to various downstream learning tasks. Extensive benchmark experiments including node classification, clustering, and link prediction demonstrate that our approach outperforms many state-of-the-art unsupervised methods and sometimes even exceeds the performance of supervised counterparts.
Tasks	Graph Representation Learning, Link Prediction, Node Classification, Representation Learning
Published	2020-03-03
URL	https://arxiv.org/abs/2003.01604v1
PDF	https://arxiv.org/pdf/2003.01604v1.pdf
PWC	https://paperswithcode.com/paper/self-supervised-graph-representation-learning
Repo
Framework

Robust Gaussian Process Regression with a Bias Model


Title	Robust Gaussian Process Regression with a Bias Model
Authors	Chiwoo Park, David J. Borth, Nicholas S. Wilson, Chad N. Hunter, Fritz J. Friedersdorf
Abstract	This paper presents a new approach to a robust Gaussian process (GP) regression. Most existing approaches replace an outlier-prone Gaussian likelihood with a non-Gaussian likelihood induced from a heavy tail distribution, such as the Laplace distribution and Student-t distribution. However, the use of a non-Gaussian likelihood would incur the need for a computationally expensive Bayesian approximate computation in the posterior inferences. The proposed approach models an outlier as a noisy and biased observation of an unknown regression function, and accordingly, the likelihood contains bias terms to explain the degree of deviations from the regression function. We entail how the biases can be estimated accurately with other hyperparameters by a regularized maximum likelihood estimation. Conditioned on the bias estimates, the robust GP regression can be reduced to a standard GP regression problem with analytical forms of the predictive mean and variance estimates. Therefore, the proposed approach is simple and very computationally attractive. It also gives a very robust and accurate GP estimate for many tested scenarios. For the numerical evaluation, we perform a comprehensive simulation study to evaluate the proposed approach with the comparison to the existing robust GP approaches under various simulated scenarios of different outlier proportions and different noise levels. The approach is applied to data from two measurement systems, where the predictors are based on robust environmental parameter measurements and the response variables utilize more complex chemical sensing methods that contain a certain percentage of outliers. The utility of the measurement systems and value of the environmental data are improved through the computationally efficient GP regression and bias model.
Tasks
Published	2020-01-14
URL	https://arxiv.org/abs/2001.04639v1
PDF	https://arxiv.org/pdf/2001.04639v1.pdf
PWC	https://paperswithcode.com/paper/robust-gaussian-process-regression-with-a
Repo
Framework

Inductive Representation Learning on Temporal Graphs


Title	Inductive Representation Learning on Temporal Graphs
Authors	Da Xu, Chuanwei Ruan, Evren Korpeoglu, Sushant Kumar, Kannan Achan
Abstract	Inductive representation learning on temporal graphs is an important step toward salable machine learning on real-world dynamic networks. The evolving nature of temporal dynamic graphs requires handling new nodes as well as capturing temporal patterns. The node embeddings, which are now functions of time, should represent both the static node features and the evolving topological structures. Moreover, node and topological features can be temporal as well, whose patterns the node embeddings should also capture. We propose the temporal graph attention (TGAT) layer to efficiently aggregate temporal-topological neighborhood features as well as to learn the time-feature interactions. For TGAT, we use the self-attention mechanism as building block and develop a novel functional time encoding technique based on the classical Bochner’s theorem from harmonic analysis. By stacking TGAT layers, the network recognizes the node embeddings as functions of time and is able to inductively infer embeddings for both new and observed nodes as the graph evolves. The proposed approach handles both node classification and link prediction task, and can be naturally extended to include the temporal edge features. We evaluate our method with transductive and inductive tasks under temporal settings with two benchmark and one industrial dataset. Our TGAT model compares favorably to state-of-the-art baselines as well as the previous temporal graph embedding approaches.
Tasks	Graph Embedding, Link Prediction, Node Classification, Representation Learning
Published	2020-02-19
URL	https://arxiv.org/abs/2002.07962v1
PDF	https://arxiv.org/pdf/2002.07962v1.pdf
PWC	https://paperswithcode.com/paper/inductive-representation-learning-on-temporal-1
Repo
Framework

RecipeGPT: Generative Pre-training Based Cooking Recipe Generation and Evaluation System


Title	RecipeGPT: Generative Pre-training Based Cooking Recipe Generation and Evaluation System
Authors	Helena H. Lee, Ke Shu, Palakorn Achananuparp, Philips Kokoh Prasetyo, Yue Liu, Ee-Peng Lim, Lav R. Varshney
Abstract	Interests in the automatic generation of cooking recipes have been growing steadily over the past few years thanks to a large amount of online cooking recipes. We present RecipeGPT, a novel online recipe generation and evaluation system. The system provides two modes of text generations: (1) instruction generation from given recipe title and ingredients; and (2) ingredient generation from recipe title and cooking instructions. Its back-end text generation module comprises a generative pre-trained language model GPT-2 fine-tuned on a large cooking recipe dataset. Moreover, the recipe evaluation module allows the users to conveniently inspect the quality of the generated recipe contents and store the results for future reference. RecipeGPT can be accessed online at https://recipegpt.org/.
Tasks	Language Modelling, Recipe Generation, Text Generation
Published	2020-03-05
URL	https://arxiv.org/abs/2003.02498v1
PDF	https://arxiv.org/pdf/2003.02498v1.pdf
PWC	https://paperswithcode.com/paper/recipegpt-generative-pre-training-based
Repo
Framework

Memory capacity of neural networks with threshold and ReLU activations


Title	Memory capacity of neural networks with threshold and ReLU activations
Authors	Roman Vershynin
Abstract	Overwhelming theoretical and empirical evidence shows that mildly overparametrized neural networks – those with more connections than the size of the training data – are often able to memorize the training data with $100%$ accuracy. This was rigorously proved for networks with sigmoid activation functions and, very recently, for ReLU activations. Addressing a 1988 open question of Baum, we prove that this phenomenon holds for general multilayered perceptrons, i.e. neural networks with threshold activation functions, or with any mix of threshold and ReLU activations. Our construction is probabilistic and exploits sparsity.
Tasks
Published	2020-01-20
URL	https://arxiv.org/abs/2001.06938v1
PDF	https://arxiv.org/pdf/2001.06938v1.pdf
PWC	https://paperswithcode.com/paper/memory-capacity-of-neural-networks-with
Repo
Framework

PulseSatellite: A tool using human-AI feedback loops for satellite image analysis in humanitarian contexts


Title	PulseSatellite: A tool using human-AI feedback loops for satellite image analysis in humanitarian contexts
Authors	Tomaz Logar, Joseph Bullock, Edoardo Nemni, Lars Bromley, John A. Quinn, Miguel Luengo-Oroz
Abstract	Humanitarian response to natural disasters and conflicts can be assisted by satellite image analysis. In a humanitarian context, very specific satellite image analysis tasks must be done accurately and in a timely manner to provide operational support. We present PulseSatellite, a collaborative satellite image analysis tool which leverages neural network models that can be retrained on-the fly and adapted to specific humanitarian contexts and geographies. We present two case studies, in mapping shelters and floods respectively, that illustrate the capabilities of PulseSatellite.
Tasks
Published	2020-01-29
URL	https://arxiv.org/abs/2001.10685v1
PDF	https://arxiv.org/pdf/2001.10685v1.pdf
PWC	https://paperswithcode.com/paper/pulsesatellite-a-tool-using-human-ai-feedback
Repo
Framework

Asymptotically Efficient Off-Policy Evaluation for Tabular Reinforcement Learning


Title	Asymptotically Efficient Off-Policy Evaluation for Tabular Reinforcement Learning
Authors	Ming Yin, Yu-Xiang Wang
Abstract	We consider the problem of off-policy evaluation for reinforcement learning, where the goal is to estimate the expected reward of a target policy $\pi$ using offline data collected by running a logging policy $\mu$. Standard importance-sampling based approaches for this problem suffer from a variance that scales exponentially with time horizon $H$, which motivates a splurge of recent interest in alternatives that break the “Curse of Horizon” (Liu et al. 2018, Xie et al. 2019). In particular, it was shown that a marginalized importance sampling (MIS) approach can be used to achieve an estimation error of order $O(H^3/ n)$ in mean square error (MSE) under an episodic Markov Decision Process model with finite states and potentially infinite actions. The MSE bound however is still a factor of $H$ away from a Cramer-Rao lower bound of order $\Omega(H^2/n)$. In this paper, we prove that with a simple modification to the MIS estimator, we can asymptotically attain the Cramer-Rao lower bound, provided that the action space is finite. We also provide a general method for constructing MIS estimators with high-probability error bounds.
Tasks
Published	2020-01-29
URL	https://arxiv.org/abs/2001.10742v1
PDF	https://arxiv.org/pdf/2001.10742v1.pdf
PWC	https://paperswithcode.com/paper/asymptotically-efficient-off-policy
Repo
Framework

MOTS: Minimax Optimal Thompson Sampling


Title	MOTS: Minimax Optimal Thompson Sampling
Authors	Tianyuan Jin, Pan Xu, Jieming Shi, Xiaokui Xiao, Quanquan Gu
Abstract	Thompson sampling is one of the most widely used algorithms for many online decision problems, due to its simplicity in implementation and superior empirical performance over other state-of-the-art methods. Despite its popularity and empirical success, it has remained an open problem whether Thompson sampling can achieve the minimax optimal regret $O(\sqrt{KT})$ for $K$-armed bandit problems, where $T$ is the total time horizon. In this paper, we solve this long open problem by proposing a new Thompson sampling algorithm called MOTS that adaptively truncates the sampling result of the chosen arm at each time step. We prove that this simple variant of Thompson sampling achieves the minimax optimal regret bound $O(\sqrt{KT})$ for finite time horizon $T$ and also the asymptotic optimal regret bound when $T$ grows to infinity as well. This is the first time that the minimax optimality of multi-armed bandit problems has been attained by Thompson sampling type of algorithms.
Tasks
Published	2020-03-03
URL	https://arxiv.org/abs/2003.01803v1
PDF	https://arxiv.org/pdf/2003.01803v1.pdf
PWC	https://paperswithcode.com/paper/mots-minimax-optimal-thompson-sampling
Repo
Framework

Optimal by Design: Model-Driven Synthesis of Adaptation Strategies for Autonomous Systems


Title	Optimal by Design: Model-Driven Synthesis of Adaptation Strategies for Autonomous Systems
Authors	Yehia Elrakaiby, Paola Spoletini, Bashar Nuseibeh
Abstract	Many software systems have become too large and complex to be managed efficiently by human administrators, particularly when they operate in uncertain and dynamic environments and require frequent changes. Requirements-driven adaptation techniques have been proposed to endow systems with the necessary means to autonomously decide ways to satisfy their requirements. However, many current approaches rely on general-purpose languages, models and/or frameworks to design, develop and analyze autonomous systems. Unfortunately, these tools are not tailored towards the characteristics of adaptation problems in autonomous systems. In this paper, we present Optimal by Design (ObD ), a framework for model-based requirements-driven synthesis of optimal adaptation strategies for autonomous systems. ObD proposes a model (and a language) for the high-level description of the basic elements of self-adaptive systems, namely the system, capabilities, requirements and environment. Based on those elements, a Markov Decision Process (MDP) is constructed to compute the optimal strategy or the most rewarding system behaviour. Furthermore, this defines a reflex controller that can ensure timely responses to changes. One novel feature of the framework is that it benefits both from goal-oriented techniques, developed for requirement elicitation, refinement and analysis, and synthesis capabilities and extensive research around MDPs, their extensions and tools. Our preliminary evaluation results demonstrate the practicality and advantages of the framework.
Tasks
Published	2020-01-16
URL	https://arxiv.org/abs/2001.08525v1
PDF	https://arxiv.org/pdf/2001.08525v1.pdf
PWC	https://paperswithcode.com/paper/optimal-by-design-model-driven-synthesis-of
Repo
Framework

A point-wise linear model reveals reasons for 30-day readmission of heart failure patients


Title	A point-wise linear model reveals reasons for 30-day readmission of heart failure patients
Authors	Yasuho Yamashita, Takuma Shibahara, Junichi Kuwata
Abstract	Heart failures in the United States cost an estimated 30.7 billion dollars annually and predictive analysis can decrease costs due to readmission of heart failure patients. Deep learning can predict readmissions but does not give reasons for its predictions. Ours is the first study on a deep-learning approach to explaining decisions behind readmission predictions. Additionally, it provides an automatic patient stratification to explain cohorts of readmitted patients. The new deep-learning model called a point-wise linear model is a meta-learning machine of linear models. It generates a logistic regression model to predict early readmission for each patient. The custom-made prediction models allow us to analyze feature importance. We evaluated the approach using a dataset that had 30-days readmission patients with heart failures. This study has been submitted in PLOS ONE. In advance, we would like to share the theoretical aspect of the point-wise linear model as a part of our study.
Tasks	Feature Importance, Meta-Learning
Published	2020-01-20
URL	https://arxiv.org/abs/2001.06988v1
PDF	https://arxiv.org/pdf/2001.06988v1.pdf
PWC	https://paperswithcode.com/paper/a-point-wise-linear-model-reveals-reasons-for
Repo
Framework

Improving the Robustness of Graphs through Reinforcement Learning and Graph Neural Networks


Title	Improving the Robustness of Graphs through Reinforcement Learning and Graph Neural Networks
Authors	Victor-Alexandru Darvariu, Stephen Hailes, Mirco Musolesi
Abstract	Graphs can be used to represent and reason about real world systems. A variety of metrics have been devised to quantify their global characteristics. In general, prior work focuses on measuring the properties of existing graphs rather than the problem of dynamically modifying them (for example, by adding edges) in order to improve the value of an objective function. In this paper, we present RNet-DQN, a solution for improving graph robustness based on Graph Neural Network architectures and Deep Reinforcement Learning. We investigate the application of this approach for improving graph robustness, which is relevant to infrastructure and communication networks. We capture robustness using two objective functions and use changes in their values as the reward signal. Our experiments show that our approach can learn edge addition policies for improving robustness that perform significantly better than random and, in some cases, exceed the performance of a greedy baseline. Crucially, the learned policies generalize to different graphs including those larger than the ones on which they were trained. This is important because the naive greedy solution can be prohibitively expensive to compute for large graphs; our approach offers an $O(V^3)$ speed-up with respect to it.
Tasks
Published	2020-01-30
URL	https://arxiv.org/abs/2001.11279v1
PDF	https://arxiv.org/pdf/2001.11279v1.pdf
PWC	https://paperswithcode.com/paper/improving-the-robustness-of-graphs-through
Repo
Framework

How Powerful Are Randomly Initialized Pointcloud Set Functions?


Title	How Powerful Are Randomly Initialized Pointcloud Set Functions?
Authors	Aditya Sanghi, Pradeep Kumar Jayaraman
Abstract	We study random embeddings produced by untrained neural set functions, and show that they are powerful representations which well capture the input features for downstream tasks such as classification, and are often linearly separable. We obtain surprising results that show that random set functions can often obtain close to or even better accuracy than fully trained models. We investigate factors that affect the representative power of such embeddings quantitatively and qualitatively.
Tasks
Published	2020-03-11
URL	https://arxiv.org/abs/2003.05410v1
PDF	https://arxiv.org/pdf/2003.05410v1.pdf
PWC	https://paperswithcode.com/paper/how-powerful-are-randomly-initialized
Repo
Framework