January 28, 2020

3056 words 15 mins read

Paper Group ANR 940

Learning Symbolic Physics with Graph Networks. Partial Compilation of ASP Programs. Earlier Attention? Aspect-Aware LSTM for Aspect-Based Sentiment Analysis. RefNet: A Reference-aware Network for Background Based Conversation. Learn How to Cook a New Recipe in a New House: Using Map Familiarization, Curriculum Learning, and Bandit Feedback to Learn …

Learning Symbolic Physics with Graph Networks


Title	Learning Symbolic Physics with Graph Networks
Authors	Miles D. Cranmer, Rui Xu, Peter Battaglia, Shirley Ho
Abstract	We introduce an approach for imposing physically motivated inductive biases on graph networks to learn interpretable representations and improved zero-shot generalization. Our experiments show that our graph network models, which implement this inductive bias, can learn message representations equivalent to the true force vector when trained on n-body gravitational and spring-like simulations. We use symbolic regression to fit explicit algebraic equations to our trained model’s message function and recover the symbolic form of Newton’s law of gravitation without prior knowledge. We also show that our model generalizes better at inference time to systems with more bodies than had been experienced during training. Our approach is extensible, in principle, to any unknown interaction law learned by a graph network, and offers a valuable technique for interpreting and inferring explicit causal theories about the world from implicit knowledge captured by deep learning.
Tasks
Published	2019-09-12
URL	https://arxiv.org/abs/1909.05862v2
PDF	https://arxiv.org/pdf/1909.05862v2.pdf
PWC	https://paperswithcode.com/paper/learning-symbolic-physics-with-graph-networks
Repo
Framework

Partial Compilation of ASP Programs


Title	Partial Compilation of ASP Programs
Authors	Bernardo Cuteri, Carmine Dodaro, Francesco Ricca, Peter Schüller
Abstract	Answer Set Programming (ASP) is a well-known declarative formalism in logic programming. Efficient implementations made it possible to apply ASP in many scenarios, ranging from deductive databases applications to the solution of hard combinatorial problems. State-of-the-art ASP systems are based on the traditional ground&solve approach and are general-purpose implementations, i.e., they are essentially built once for any kind of input program. In this paper, we propose an extended architecture for ASP systems, in which parts of the input program are compiled into an ad-hoc evaluation algorithm (i.e., we obtain a specific binary for a given program), and might not be subject to the grounding step. To this end, we identify a condition that allows the compilation of a sub-program, and present the related partial compilation technique. Importantly, we have implemented the new approach on top of a well-known ASP solver and conducted an experimental analysis on publicly-available benchmarks. Results show that our compilation-based approach improves on the state of the art in various scenarios, including cases in which the input program is stratified or the grounding blow-up makes the evaluation unpractical with traditional ASP systems.
Tasks
Published	2019-07-24
URL	https://arxiv.org/abs/1907.10469v1
PDF	https://arxiv.org/pdf/1907.10469v1.pdf
PWC	https://paperswithcode.com/paper/partial-compilation-of-asp-programs
Repo
Framework

Earlier Attention? Aspect-Aware LSTM for Aspect-Based Sentiment Analysis


Title	Earlier Attention? Aspect-Aware LSTM for Aspect-Based Sentiment Analysis
Authors	Bowen Xing, Lejian Liao, Dandan Song, Jingang Wang, Fuzheng Zhang, Zhongyuan Wang, Heyan Huang
Abstract	Aspect-based sentiment analysis (ABSA) aims to predict fine-grained sentiments of comments with respect to given aspect terms or categories. In previous ABSA methods, the importance of aspect has been realized and verified. Most existing LSTM-based models take aspect into account via the attention mechanism, where the attention weights are calculated after the context is modeled in the form of contextual vectors. However, aspect-related information may be already discarded and aspect-irrelevant information may be retained in classic LSTM cells in the context modeling process, which can be improved to generate more effective context representations. This paper proposes a novel variant of LSTM, termed as aspect-aware LSTM (AA-LSTM), which incorporates aspect information into LSTM cells in the context modeling stage before the attention mechanism. Therefore, our AA-LSTM can dynamically produce aspect-aware contextual representations. We experiment with several representative LSTM-based models by replacing the classic LSTM cells with the AA-LSTM cells. Experimental results on SemEval-2014 Datasets demonstrate the effectiveness of AA-LSTM.
Tasks	Aspect-Based Sentiment Analysis, Sentiment Analysis
Published	2019-05-19
URL	https://arxiv.org/abs/1905.07719v2
PDF	https://arxiv.org/pdf/1905.07719v2.pdf
PWC	https://paperswithcode.com/paper/earlier-attention-aspect-aware-lstm-for
Repo
Framework

RefNet: A Reference-aware Network for Background Based Conversation


Title	RefNet: A Reference-aware Network for Background Based Conversation
Authors	Chuan Meng, Pengjie Ren, Zhumin Chen, Christof Monz, Jun Ma, Maarten de Rijke
Abstract	Existing conversational systems tend to generate generic responses. Recently, Background Based Conversations (BBCs) have been introduced to address this issue. Here, the generated responses are grounded in some background information. The proposed methods for BBCs are able to generate more informative responses, they either cannot generate natural responses or have difficulty in locating the right background information. In this paper, we propose a Reference-aware Network (RefNet) to address the two issues. Unlike existing methods that generate responses token by token, RefNet incorporates a novel reference decoder that provides an alternative way to learn to directly cite a semantic unit (e.g., a span containing complete semantic information) from the background. Experimental results show that RefNet significantly outperforms state-of-the-art methods in terms of both automatic and human evaluations, indicating that RefNet can generate more appropriate and human-like responses.
Tasks
Published	2019-08-18
URL	https://arxiv.org/abs/1908.06449v2
PDF	https://arxiv.org/pdf/1908.06449v2.pdf
PWC	https://paperswithcode.com/paper/refnet-a-reference-aware-network-for
Repo
Framework

Learn How to Cook a New Recipe in a New House: Using Map Familiarization, Curriculum Learning, and Bandit Feedback to Learn Families of Text-Based Adventure Games


Title	Learn How to Cook a New Recipe in a New House: Using Map Familiarization, Curriculum Learning, and Bandit Feedback to Learn Families of Text-Based Adventure Games
Authors	Xusen Yin, Jonathan May
Abstract	We consider the task of learning to play families of text-based computer adventure games, i.e., fully textual environments with a common theme (e.g. cooking) and goal (e.g. prepare a meal from a recipe) but with different specifics; new instances of such games are relatively straightforward for humans to master after a brief exposure to the genre but have been curiously difficult for computer agents to learn. We find that the deep Q-learning strategies that have been successfully leveraged for superhuman performance in single-instance action video games can be applied to learn families of text video games when adopting simple strategies that correlate with human-like learning behavior. Specifically, we build agents that learn to tackle simple scenarios before more complex ones using curriculum learning, that familiarize themselves in an unfamiliar environment by navigating before acting, and that explore uncertain environment more thoroughly using multi-armed bandit decision policies. We demonstrate improved task completion rates over reasonable baselines when evaluating on never-before-seen games of that theme.
Tasks	Common Sense Reasoning, Q-Learning
Published	2019-08-13
URL	https://arxiv.org/abs/1908.04777v2
PDF	https://arxiv.org/pdf/1908.04777v2.pdf
PWC	https://paperswithcode.com/paper/learn-how-to-cook-a-new-recipe-in-a-new-house
Repo
Framework

Synthesizing Images from Spatio-Temporal Representations using Spike-based Backpropagation


Title	Synthesizing Images from Spatio-Temporal Representations using Spike-based Backpropagation
Authors	Deboleena Roy, Priyadarshini Panda, Kaushik Roy
Abstract	Spiking neural networks (SNNs) offer a promising alternative to current artificial neural networks to enable low-power event-driven neuromorphic hardware. Spike-based neuromorphic applications require processing and extracting meaningful information from spatio-temporal data, represented as series of spike trains over time. In this paper, we propose a method to synthesize images from multiple modalities in a spike-based environment. We use spiking auto-encoders to convert image and audio inputs into compact spatio-temporal representations that is then decoded for image synthesis. For this, we use a direct training algorithm that computes loss on the membrane potential of the output layer and back-propagates it by using a sigmoid approximation of the neuron’s activation function to enable differentiability. The spiking autoencoders are benchmarked on MNIST and Fashion-MNIST and achieve very low reconstruction loss, comparable to ANNs. Then, spiking autoencoders are trained to learn meaningful spatio-temporal representations of the data, across the two modalities - audio and visual. We synthesize images from audio in a spike-based environment by first generating, and then utilizing such shared multi-modal spatio-temporal representations. Our audio to image synthesis model is tested on the task of converting TI-46 digits audio samples to MNIST images. We are able to synthesize images with high fidelity and the model achieves competitive performance against ANNs.
Tasks	Image Generation
Published	2019-05-24
URL	https://arxiv.org/abs/1906.08861v1
PDF	https://arxiv.org/pdf/1906.08861v1.pdf
PWC	https://paperswithcode.com/paper/synthesizing-images-from-spatio-temporal
Repo
Framework

Targeted Sentiment Analysis: A Data-Driven Categorization


Title	Targeted Sentiment Analysis: A Data-Driven Categorization
Authors	Jiaxin Pei, Aixin Sun, Chenliang Li
Abstract	Targeted sentiment analysis (TSA), also known as aspect based sentiment analysis (ABSA), aims at detecting fine-grained sentiment polarity towards targets in a given opinion document. Due to the lack of labeled datasets and effective technology, TSA had been intractable for many years. The newly released datasets and the rapid development of deep learning technologies are key enablers for the recent significant progress made in this area. However, the TSA tasks have been defined in various ways with different understandings towards basic concepts like `target' and` aspect’. In this paper, we categorize the different tasks and highlight the differences in the available datasets and their specific tasks. We then further discuss the challenges related to data collection and data annotation which are overlooked in many previous studies.
Tasks	Aspect-Based Sentiment Analysis, Sentiment Analysis
Published	2019-05-09
URL	https://arxiv.org/abs/1905.03423v1
PDF	https://arxiv.org/pdf/1905.03423v1.pdf
PWC	https://paperswithcode.com/paper/190503423
Repo
Framework

Efficient Prealignment of CT Scans for Registration through a Bodypart Regressor


Title	Efficient Prealignment of CT Scans for Registration through a Bodypart Regressor
Authors	Hans Meine, Alessa Hering
Abstract	Convolutional neural networks have not only been applied for classification of voxels, objects, or images, for instance, but have also been proposed as a bodypart regressor. We pick up this underexplored idea and evaluate its value for registration: A CNN is trained to output the relative height within the human body in axial CT scans, and the resulting scores are used for quick alignment between different timepoints. Preliminary results confirm that this allows both fast and robust prealignment compared with iterative approaches.
Tasks
Published	2019-09-19
URL	https://arxiv.org/abs/1909.08898v1
PDF	https://arxiv.org/pdf/1909.08898v1.pdf
PWC	https://paperswithcode.com/paper/efficient-prealignment-of-ct-scans-for
Repo
Framework

Approximation in $L^p(μ)$ with deep ReLU neural networks


Title	Approximation in $L^p(μ)$ with deep ReLU neural networks
Authors	Felix Voigtlaender, Philipp Petersen
Abstract	We discuss the expressive power of neural networks which use the non-smooth ReLU activation function $\varrho(x) = \max{0,x}$ by analyzing the approximation theoretic properties of such networks. The existing results mainly fall into two categories: approximation using ReLU networks with a fixed depth, or using ReLU networks whose depth increases with the approximation accuracy. After reviewing these findings, we show that the results concerning networks with fixed depth— which up to now only consider approximation in $L^p(\lambda)$ for the Lebesgue measure $\lambda$— can be generalized to approximation in $L^p(\mu)$, for any finite Borel measure $\mu$. In particular, the generalized results apply in the usual setting of statistical learning theory, where one is interested in approximation in $L^2(\mathbb{P})$, with the probability measure $\mathbb{P}$ describing the distribution of the data.
Tasks
Published	2019-04-09
URL	http://arxiv.org/abs/1904.04789v1
PDF	http://arxiv.org/pdf/1904.04789v1.pdf
PWC	https://paperswithcode.com/paper/approximation-in-lp-with-deep-relu-neural
Repo
Framework

Solving Arithmetic Word Problems Automatically Using Transformer and Unambiguous Representations


Title	Solving Arithmetic Word Problems Automatically Using Transformer and Unambiguous Representations
Authors	Kaden Griffith, Jugal Kalita
Abstract	Constructing accurate and automatic solvers of math word problems has proven to be quite challenging. Prior attempts using machine learning have been trained on corpora specific to math word problems to produce arithmetic expressions in infix notation before answer computation. We find that custom-built neural networks have struggled to generalize well. This paper outlines the use of Transformer networks trained to translate math word problems to equivalent arithmetic expressions in infix, prefix, and postfix notations. In addition to training directly on domain-specific corpora, we use an approach that pre-trains on a general text corpus to provide foundational language abilities to explore if it improves performance. We compare results produced by a large number of neural configurations and find that most configurations outperform previously reported approaches on three of four datasets with significant increases in accuracy of over 20 percentage points. The best neural approaches boost accuracy by almost 10% on average when compared to the previous state of the art.
Tasks
Published	2019-12-02
URL	https://arxiv.org/abs/1912.00871v1
PDF	https://arxiv.org/pdf/1912.00871v1.pdf
PWC	https://paperswithcode.com/paper/solving-arithmetic-word-problems
Repo
Framework

Listen to Look: Action Recognition by Previewing Audio


Title	Listen to Look: Action Recognition by Previewing Audio
Authors	Ruohan Gao, Tae-Hyun Oh, Kristen Grauman, Lorenzo Torresani
Abstract	In the face of the video data deluge, today’s expensive clip-level classifiers are increasingly impractical. We propose a framework for efficient action recognition in untrimmed video that uses audio as a preview mechanism to eliminate both short-term and long-term visual redundancies. First, we devise an ImgAud2Vid framework that hallucinates clip-level features by distilling from lighter modalities—a single frame and its accompanying audio—reducing short-term temporal redundancy for efficient clip-level recognition. Second, building on ImgAud2Vid, we further propose ImgAud-Skimming, an attention-based long short-term memory network that iteratively selects useful moments in untrimmed videos, reducing long-term temporal redundancy for efficient video-level recognition. Extensive experiments on four action recognition datasets demonstrate that our method achieves the state-of-the-art in terms of both recognition accuracy and speed.
Tasks
Published	2019-12-10
URL	https://arxiv.org/abs/1912.04487v3
PDF	https://arxiv.org/pdf/1912.04487v3.pdf
PWC	https://paperswithcode.com/paper/listen-to-look-action-recognition-by
Repo
Framework

Large-scale Traffic Signal Control Using a Novel Multi-Agent Reinforcement Learning


Title	Large-scale Traffic Signal Control Using a Novel Multi-Agent Reinforcement Learning
Authors	Xiaoqiang Wang, Liangjun Ke, Zhimin Qiao, Xinghua Chai
Abstract	Finding the optimal signal timing strategy is a difficult task for the problem of large-scale traffic signal control (TSC). Multi-Agent Reinforcement Learning (MARL) is a promising method to solve this problem. However, there is still room for improvement in extending to large-scale problems and modeling the behaviors of other agents for each individual agent. In this paper, a new MARL, called Cooperative double Q-learning (Co-DQL), is proposed, which has several prominent features. It uses a highly scalable independent double Q-learning method based on double estimators and the UCB policy, which can eliminate the over-estimation problem existing in traditional independent Q-learning while ensuring exploration. It uses mean field approximation to model the interaction among agents, thereby making agents learn a better cooperative strategy. In order to improve the stability and robustness of the learning process, we introduce a new reward allocation mechanism and a local state sharing method. In addition, we analyze the convergence properties of the proposed algorithm. Co-DQL is applied on TSC and tested on a multi-traffic signal simulator. According to the results obtained on several traffic scenarios, Co- DQL outperforms several state-of-the-art decentralized MARL algorithms. It can effectively shorten the average waiting time of the vehicles in the whole road system.
Tasks	Multi-agent Reinforcement Learning, Q-Learning
Published	2019-08-10
URL	https://arxiv.org/abs/1908.03761v1
PDF	https://arxiv.org/pdf/1908.03761v1.pdf
PWC	https://paperswithcode.com/paper/large-scale-traffic-signal-control-using-a
Repo
Framework

Spatio-Temporal Deep Learning-Based Undersampling Artefact Reduction for 2D Radial Cine MRI with Limited Data


Title	Spatio-Temporal Deep Learning-Based Undersampling Artefact Reduction for 2D Radial Cine MRI with Limited Data
Authors	Andreas Kofler, Marc Dewey, Tobias Schaeffter, Christian Wald, Christoph Kolbitsch
Abstract	In this work we reduce undersampling artefacts in two-dimensional ($2D$) golden-angle radial cine cardiac MRI by applying a modified version of the U-net. We train the network on $2D$ spatio-temporal slices which are previously extracted from the image sequences. We compare our approach to two $2D$ and a $3D$ Deep Learning-based post processing methods and to three iterative reconstruction methods for dynamic cardiac MRI. Our method outperforms the $2D$ spatially trained U-net and the $2D$ spatio-temporal U-net. Compared to the $3D$ spatio-temporal U-net, our method delivers comparable results, but with shorter training times and less training data. Compared to the Compressed Sensing-based methods $kt$-FOCUSS and a total variation regularised reconstruction approach, our method improves image quality with respect to all reported metrics. Further, it achieves competitive results when compared to an iterative reconstruction method based on adaptive regularization with Dictionary Learning and total variation, while only requiring a small fraction of the computational time. A persistent homology analysis demonstrates that the data manifold of the spatio-temporal domain has a lower complexity than the spatial domain and therefore, the learning of a projection-like mapping is facilitated. Even when trained on only one single subject without data-augmentation, our approach yields results which are similar to the ones obtained on a large training dataset. This makes the method particularly suitable for training a network on limited training data. Finally, in contrast to the spatial $2D$ U-net, our proposed method is shown to be naturally robust with respect to image rotation in image space and almost achieves rotation-equivariance where neither data-augmentation nor a particular network design are required.
Tasks	Data Augmentation, Dictionary Learning
Published	2019-04-01
URL	https://arxiv.org/abs/1904.01574v2
PDF	https://arxiv.org/pdf/1904.01574v2.pdf
PWC	https://paperswithcode.com/paper/spatio-temporal-deep-learning-based
Repo
Framework

Joint Learning of Discriminative Low-dimensional Image Representations Based on Dictionary Learning and Two-layer Orthogonal Projections


Title	Joint Learning of Discriminative Low-dimensional Image Representations Based on Dictionary Learning and Two-layer Orthogonal Projections
Authors	Xian Wei, Hao Shen, Yuanxiang Li, Xuan Tang, Bo Jin, Lijun Zhao, Yi Lu Murphey
Abstract	There are some inadequacies in the language description of this paper that require further improvement. This paper is based on a revision of a conference paper. It is now necessary to further explain the difference between the contributions of the two papers.
Tasks	Dictionary Learning, Image Classification
Published	2019-03-24
URL	http://arxiv.org/abs/1903.09977v2
PDF	http://arxiv.org/pdf/1903.09977v2.pdf
PWC	https://paperswithcode.com/paper/joint-learning-of-discriminative-low
Repo
Framework

MRNN: A Multi-Resolution Neural Network with Duplex Attention for Document Retrieval in the Context of Question Answering


Title	MRNN: A Multi-Resolution Neural Network with Duplex Attention for Document Retrieval in the Context of Question Answering
Authors	Tolgahan Cakaloglu, Xiaowei Xu
Abstract	The primary goal of ad-hoc retrieval (document retrieval in the context of question answering) is to find relevant documents satisfied the information need posted in a natural language query. It requires a good understanding of the query and all the documents in a corpus, which is difficult because the meaning of natural language texts depends on the context, syntax,and semantics. Recently deep neural networks have been used to rank search results in response to a query. In this paper, we devise a multi-resolution neural network(MRNN) to leverage the whole hierarchy of representations for document retrieval. The proposed MRNN model is capable of deriving a representation that integrates representations of different levels of abstraction from all the layers of the learned hierarchical representation.Moreover, a duplex attention component is designed to refinethe multi-resolution representation so that an optimal contextfor matching the query and document can be determined. More specifically, the first attention mechanism determines optimal context from the learned multi-resolution representation for the query and document. The latter attention mechanism aims to fine-tune the representation so that the query and the relevant document are closer in proximity. The empirical study shows that MRNN with the duplex attention is significantly superior to existing models used for ad-hoc retrieval on benchmark datasets including SQuAD, WikiQA, QUASAR, and TrecQA.
Tasks	Question Answering
Published	2019-11-03
URL	https://arxiv.org/abs/1911.00964v1
PDF	https://arxiv.org/pdf/1911.00964v1.pdf
PWC	https://paperswithcode.com/paper/mrnn-a-multi-resolution-neural-network-with
Repo
Framework