October 21, 2019

2807 words 14 mins read

Paper Group AWR 140

Toward Diverse Text Generation with Inverse Reinforcement Learning. Multi-Agent Reinforcement Learning: A Report on Challenges and Approaches. Hierarchical Graph Clustering using Node Pair Sampling. Mem2Seq: Effectively Incorporating Knowledge Bases into End-to-End Task-Oriented Dialog Systems. Norm-Preservation: Why Residual Networks Can Become Ex …

Toward Diverse Text Generation with Inverse Reinforcement Learning


Title	Toward Diverse Text Generation with Inverse Reinforcement Learning
Authors	Zhan Shi, Xinchi Chen, Xipeng Qiu, Xuanjing Huang
Abstract	Text generation is a crucial task in NLP. Recently, several adversarial generative models have been proposed to improve the exposure bias problem in text generation. Though these models gain great success, they still suffer from the problems of reward sparsity and mode collapse. In order to address these two problems, in this paper, we employ inverse reinforcement learning (IRL) for text generation. Specifically, the IRL framework learns a reward function on training data, and then an optimal policy to maximum the expected total reward. Similar to the adversarial models, the reward and policy function in IRL are optimized alternately. Our method has two advantages: (1) the reward function can produce more dense reward signals. (2) the generation policy, trained by “entropy regularized” policy gradient, encourages to generate more diversified texts. Experiment results demonstrate that our proposed method can generate higher quality texts than the previous methods.
Tasks	Text Generation
Published	2018-04-30
URL	http://arxiv.org/abs/1804.11258v3
PDF	http://arxiv.org/pdf/1804.11258v3.pdf
PWC	https://paperswithcode.com/paper/toward-diverse-text-generation-with-inverse
Repo	https://github.com/FudanNLP/Irl_gen
Framework	tf

Multi-Agent Reinforcement Learning: A Report on Challenges and Approaches


Title	Multi-Agent Reinforcement Learning: A Report on Challenges and Approaches
Authors	Sanyam Kapoor
Abstract	Reinforcement Learning (RL) is a learning paradigm concerned with learning to control a system so as to maximize an objective over the long term. This approach to learning has received immense interest in recent times and success manifests itself in the form of human-level performance on games like \textit{Go}. While RL is emerging as a practical component in real-life systems, most successes have been in Single Agent domains. This report will instead specifically focus on challenges that are unique to Multi-Agent Systems interacting in mixed cooperative and competitive environments. The report concludes with advances in the paradigm of training Multi-Agent Systems called \textit{Decentralized Actor, Centralized Critic}, based on an extension of MDPs called \textit{Decentralized Partially Observable MDP}s, which has seen a renewed interest lately.
Tasks	Multi-agent Reinforcement Learning
Published	2018-07-25
URL	http://arxiv.org/abs/1807.09427v1
PDF	http://arxiv.org/pdf/1807.09427v1.pdf
PWC	https://paperswithcode.com/paper/multi-agent-reinforcement-learning-a-report
Repo	https://github.com/activatedgeek/marl-challenges-approaches
Framework	none

Hierarchical Graph Clustering using Node Pair Sampling


Title	Hierarchical Graph Clustering using Node Pair Sampling
Authors	Thomas Bonald, Bertrand Charpentier, Alexis Galland, Alexandre Hollocou
Abstract	We present a novel hierarchical graph clustering algorithm inspired by modularity-based clustering techniques. The algorithm is agglomerative and based on a simple distance between clusters induced by the probability of sampling node pairs. We prove that this distance is reducible, which enables the use of the nearest-neighbor chain to speed up the agglomeration. The output of the algorithm is a regular dendrogram, which reveals the multi-scale structure of the graph. The results are illustrated on both synthetic and real datasets.
Tasks	Graph Clustering
Published	2018-06-05
URL	http://arxiv.org/abs/1806.01664v2
PDF	http://arxiv.org/pdf/1806.01664v2.pdf
PWC	https://paperswithcode.com/paper/hierarchical-graph-clustering-using-node-pair
Repo	https://github.com/tbonald/paris
Framework	none

Mem2Seq: Effectively Incorporating Knowledge Bases into End-to-End Task-Oriented Dialog Systems


Title	Mem2Seq: Effectively Incorporating Knowledge Bases into End-to-End Task-Oriented Dialog Systems
Authors	Andrea Madotto, Chien-Sheng Wu, Pascale Fung
Abstract	End-to-end task-oriented dialog systems usually suffer from the challenge of incorporating knowledge bases. In this paper, we propose a novel yet simple end-to-end differentiable model called memory-to-sequence (Mem2Seq) to address this issue. Mem2Seq is the first neural generative model that combines the multi-hop attention over memories with the idea of pointer network. We empirically show how Mem2Seq controls each generation step, and how its multi-hop attention mechanism helps in learning correlations between memories. In addition, our model is quite general without complicated task-specific designs. As a result, we show that Mem2Seq can be trained faster and attain the state-of-the-art performance on three different task-oriented dialog datasets.
Tasks
Published	2018-04-23
URL	http://arxiv.org/abs/1804.08217v3
PDF	http://arxiv.org/pdf/1804.08217v3.pdf
PWC	https://paperswithcode.com/paper/mem2seq-effectively-incorporating-knowledge
Repo	https://github.com/HLTCHKUST/Mem2Seq
Framework	pytorch

Norm-Preservation: Why Residual Networks Can Become Extremely Deep?


Title	Norm-Preservation: Why Residual Networks Can Become Extremely Deep?
Authors	Alireza Zaeemzadeh, Nazanin Rahnavard, Mubarak Shah
Abstract	Augmenting neural networks with skip connections, as introduced in the so-called ResNet architecture, surprised the community by enabling the training of networks of more than 1,000 layers with significant performance gains. This paper deciphers ResNet by analyzing the effect of skip connections, and puts forward new theoretical results on the advantages of identity skip connections in neural networks. We prove that the skip connections in the residual blocks facilitate preserving the norm of the gradient, and lead to stable back-propagation, which is desirable from optimization perspective. We also show that, perhaps surprisingly, as more residual blocks are stacked, the norm-preservation of the network is enhanced. Our theoretical arguments are supported by extensive empirical evidence. Can we push for extra norm-preservation? We answer this question by proposing an efficient method to regularize the singular values of the convolution operator and making the ResNet’s transition layers extra norm-preserving. Our numerical investigations demonstrate that the learning dynamics and the classification performance of ResNet can be improved by making it even more norm preserving. Our results and the introduced modification for ResNet, referred to as Procrustes ResNets, can be used as a guide for training deeper networks and can also inspire new deeper architectures.
Tasks
Published	2018-05-18
URL	https://arxiv.org/abs/1805.07477v4
PDF	https://arxiv.org/pdf/1805.07477v4.pdf
PWC	https://paperswithcode.com/paper/norm-preservation-why-residual-networks-can
Repo	https://github.com/zaeemzadeh/ProcResNet
Framework	pytorch

The WiLI benchmark dataset for written language identification


Title	The WiLI benchmark dataset for written language identification
Authors	Martin Thoma
Abstract	This paper describes the WiLI-2018 benchmark dataset for monolingual written natural language identification. WiLI-2018 is a publicly available, free of charge dataset of short text extracts from Wikipedia. It contains 1000 paragraphs of 235 languages, totaling in 23500 paragraphs. WiLI is a classification dataset: Given an unknown paragraph written in one dominant language, it has to be decided which language it is.
Tasks	Language Identification
Published	2018-01-23
URL	http://arxiv.org/abs/1801.07779v1
PDF	http://arxiv.org/pdf/1801.07779v1.pdf
PWC	https://paperswithcode.com/paper/the-wili-benchmark-dataset-for-written
Repo	https://github.com/LauraRuis/LanguageIdentification
Framework	pytorch

Heated-Up Softmax Embedding


Title	Heated-Up Softmax Embedding
Authors	Xu Zhang, Felix Xinnan Yu, Svebor Karaman, Wei Zhang, Shih-Fu Chang
Abstract	Metric learning aims at learning a distance which is consistent with the semantic meaning of the samples. The problem is generally solved by learning an embedding for each sample such that the embeddings of samples of the same category are compact while the embeddings of samples of different categories are spread-out in the feature space. We study the features extracted from the second last layer of a deep neural network based classifier trained with the cross entropy loss on top of the softmax layer. We show that training classifiers with different temperature values of softmax function leads to features with different levels of compactness. Leveraging these insights, we propose a “heating-up” strategy to train a classifier with increasing temperatures, leading the corresponding embeddings to achieve state-of-the-art performance on a variety of metric learning benchmarks.
Tasks	Metric Learning
Published	2018-09-11
URL	http://arxiv.org/abs/1809.04157v1
PDF	http://arxiv.org/pdf/1809.04157v1.pdf
PWC	https://paperswithcode.com/paper/heated-up-softmax-embedding
Repo	https://github.com/ColumbiaDVMM/Heated_Up_Softmax_Embedding
Framework	tf

Graph2Seq: Graph to Sequence Learning with Attention-based Neural Networks


Title	Graph2Seq: Graph to Sequence Learning with Attention-based Neural Networks
Authors	Kun Xu, Lingfei Wu, Zhiguo Wang, Yansong Feng, Michael Witbrock, Vadim Sheinin
Abstract	The celebrated Sequence to Sequence learning (Seq2Seq) technique and its numerous variants achieve excellent performance on many tasks. However, many machine learning tasks have inputs naturally represented as graphs; existing Seq2Seq models face a significant challenge in achieving accurate conversion from graph form to the appropriate sequence. To address this challenge, we introduce a novel general end-to-end graph-to-sequence neural encoder-decoder model that maps an input graph to a sequence of vectors and uses an attention-based LSTM method to decode the target sequence from these vectors. Our method first generates the node and graph embeddings using an improved graph-based neural network with a novel aggregation strategy to incorporate edge direction information in the node embeddings. We further introduce an attention mechanism that aligns node embeddings and the decoding sequence to better cope with large graphs. Experimental results on bAbI, Shortest Path, and Natural Language Generation tasks demonstrate that our model achieves state-of-the-art performance and significantly outperforms existing graph neural networks, Seq2Seq, and Tree2Seq models; using the proposed bi-directional node embedding aggregation strategy, the model can converge rapidly to the optimal performance.
Tasks	Graph-to-Sequence, SQL-to-Text, Text Generation
Published	2018-04-03
URL	http://arxiv.org/abs/1804.00823v4
PDF	http://arxiv.org/pdf/1804.00823v4.pdf
PWC	https://paperswithcode.com/paper/graph2seq-graph-to-sequence-learning-with
Repo	https://github.com/IBM/Graph2Seq
Framework	tf

End-to-End Video Captioning with Multitask Reinforcement Learning


Title	End-to-End Video Captioning with Multitask Reinforcement Learning
Authors	Lijun Li, Boqing Gong
Abstract	Although end-to-end (E2E) learning has led to impressive progress on a variety of visual understanding tasks, it is often impeded by hardware constraints (e.g., GPU memory) and is prone to overfitting. When it comes to video captioning, one of the most challenging benchmark tasks in computer vision, those limitations of E2E learning are especially amplified by the fact that both the input videos and output captions are lengthy sequences. Indeed, state-of-the-art methods for video captioning process video frames by convolutional neural networks and generate captions by unrolling recurrent neural networks. If we connect them in an E2E manner, the resulting model is both memory-consuming and data-hungry, making it extremely hard to train. In this paper, we propose a multitask reinforcement learning approach to training an E2E video captioning model. The main idea is to mine and construct as many effective tasks (e.g., attributes, rewards, and the captions) as possible from the human captioned videos such that they can jointly regulate the search space of the E2E neural network, from which an E2E video captioning model can be found and generalized to the testing phase. To the best of our knowledge, this is the first video captioning model that is trained end-to-end from the raw video input to the caption output. Experimental results show that such a model outperforms existing ones to a large margin on two benchmark video captioning datasets.
Tasks	Video Captioning
Published	2018-03-21
URL	http://arxiv.org/abs/1803.07950v2
PDF	http://arxiv.org/pdf/1803.07950v2.pdf
PWC	https://paperswithcode.com/paper/end-to-end-video-captioning-with-multitask
Repo	https://github.com/adwardlee/multitask-end-to-end-video-captioning
Framework	tf

Near-Optimal Representation Learning for Hierarchical Reinforcement Learning


Title	Near-Optimal Representation Learning for Hierarchical Reinforcement Learning
Authors	Ofir Nachum, Shixiang Gu, Honglak Lee, Sergey Levine
Abstract	We study the problem of representation learning in goal-conditioned hierarchical reinforcement learning. In such hierarchical structures, a higher-level controller solves tasks by iteratively communicating goals which a lower-level policy is trained to reach. Accordingly, the choice of representation – the mapping of observation space to goal space – is crucial. To study this problem, we develop a notion of sub-optimality of a representation, defined in terms of expected reward of the optimal hierarchical policy using this representation. We derive expressions which bound the sub-optimality and show how these expressions can be translated to representation learning objectives which may be optimized in practice. Results on a number of difficult continuous-control tasks show that our approach to representation learning yields qualitatively better representations as well as quantitatively better hierarchical policies, compared to existing methods (see videos at https://sites.google.com/view/representation-hrl).
Tasks	Continuous Control, Hierarchical Reinforcement Learning, Representation Learning
Published	2018-10-02
URL	http://arxiv.org/abs/1810.01257v2
PDF	http://arxiv.org/pdf/1810.01257v2.pdf
PWC	https://paperswithcode.com/paper/near-optimal-representation-learning-for
Repo	https://github.com/josherich/efficient-hrl
Framework	tf

Testing Deep Neural Networks


Title	Testing Deep Neural Networks
Authors	Youcheng Sun, Xiaowei Huang, Daniel Kroening, James Sharp, Matthew Hill, Rob Ashmore
Abstract	Deep neural networks (DNNs) have a wide range of applications, and software employing them must be thoroughly tested, especially in safety-critical domains. However, traditional software test coverage metrics cannot be applied directly to DNNs. In this paper, inspired by the MC/DC coverage criterion, we propose a family of four novel test criteria that are tailored to structural features of DNNs and their semantics. We validate the criteria by demonstrating that the generated test inputs guided via our proposed coverage criteria are able to capture undesired behaviours in a DNN. Test cases are generated using a symbolic approach and a gradient-based heuristic search. By comparing them with existing methods, we show that our criteria achieve a balance between their ability to find bugs (proxied using adversarial examples) and the computational cost of test case generation. Our experiments are conducted on state-of-the-art DNNs obtained using popular open source datasets, including MNIST, CIFAR-10 and ImageNet.
Tasks
Published	2018-03-10
URL	http://arxiv.org/abs/1803.04792v4
PDF	http://arxiv.org/pdf/1803.04792v4.pdf
PWC	https://paperswithcode.com/paper/testing-deep-neural-networks
Repo	https://github.com/theyoucheng/deepcover
Framework	none

Fast Symbolic 3D Registration Solution


Title	Fast Symbolic 3D Registration Solution
Authors	Jin Wu, Ming Liu, Zebo Zhou, Rui Li
Abstract	3D registration has always been performed invoking singular value decomposition (SVD) or eigenvalue decomposition (EIG) in real engineering practices. However, these numerical algorithms suffer from uncertainty of convergence in many cases. A novel fast symbolic solution is proposed in this paper by following our recent publication in this journal. The equivalence analysis shows that our previous solver can be converted to deal with the 3D registration problem. Rather, the computation procedure is studied for further simplification of computing without complex-number support. Experimental results show that the proposed solver does not loose accuracy and robustness but improves the execution speed to a large extent by almost %50 to %80, on both personal computer and embedded processor.
Tasks
Published	2018-05-12
URL	https://arxiv.org/abs/1805.08703v3
PDF	https://arxiv.org/pdf/1805.08703v3.pdf
PWC	https://paperswithcode.com/paper/fast-symbolic-3d-registration-solution
Repo	https://github.com/zarathustr/FS3R-Matlab
Framework	none

Automatic Estimation of Simultaneous Interpreter Performance


Title	Automatic Estimation of Simultaneous Interpreter Performance
Authors	Craig Stewart, Nikolai Vogler, Junjie Hu, Jordan Boyd-Graber, Graham Neubig
Abstract	Simultaneous interpretation, translation of the spoken word in real-time, is both highly challenging and physically demanding. Methods to predict interpreter confidence and the adequacy of the interpreted message have a number of potential applications, such as in computer-assisted interpretation interfaces or pedagogical tools. We propose the task of predicting simultaneous interpreter performance by building on existing methodology for quality estimation (QE) of machine translation output. In experiments over five settings in three language pairs, we extend a QE pipeline to estimate interpreter performance (as approximated by the METEOR evaluation metric) and propose novel features reflecting interpretation strategy and evaluation measures that further improve prediction accuracy.
Tasks	Machine Translation
Published	2018-05-10
URL	http://arxiv.org/abs/1805.04016v2
PDF	http://arxiv.org/pdf/1805.04016v2.pdf
PWC	https://paperswithcode.com/paper/automatic-estimation-of-simultaneous
Repo	https://github.com/craigastewart/qe_sim_interp
Framework	none

Fréchet ChemNet Distance: A metric for generative models for molecules in drug discovery


Title	Fréchet ChemNet Distance: A metric for generative models for molecules in drug discovery
Authors	Kristina Preuer, Philipp Renz, Thomas Unterthiner, Sepp Hochreiter, Günter Klambauer
Abstract	The new wave of successful generative models in machine learning has increased the interest in deep learning driven de novo drug design. However, assessing the performance of such generative models is notoriously difficult. Metrics that are typically used to assess the performance of such generative models are the percentage of chemically valid molecules or the similarity to real molecules in terms of particular descriptors, such as the partition coefficient (logP) or druglikeness. However, method comparison is difficult because of the inconsistent use of evaluation metrics, the necessity for multiple metrics, and the fact that some of these measures can easily be tricked by simple rule-based systems. We propose a novel distance measure between two sets of molecules, called Fr'echet ChemNet distance (FCD), that can be used as an evaluation metric for generative models. The FCD is similar to a recently established performance metric for comparing image generation methods, the Fr'echet Inception Distance (FID). Whereas the FID uses one of the hidden layers of InceptionNet, the FCD utilizes the penultimate layer of a deep neural network called ChemNet, which was trained to predict drug activities. Thus, the FCD metric takes into account chemically and biologically relevant information about molecules, and also measures the diversity of the set via the distribution of generated molecules. The FCD’s advantage over previous metrics is that it can detect if generated molecules are a) diverse and have similar b) chemical and c) biological properties as real molecules. We further provide an easy-to-use implementation that only requires the SMILES representation of the generated molecules as input to calculate the FCD. Implementations are available at: https://www.github.com/bioinf-jku/FCD
Tasks	Drug Discovery
Published	2018-03-26
URL	http://arxiv.org/abs/1803.09518v3
PDF	http://arxiv.org/pdf/1803.09518v3.pdf
PWC	https://paperswithcode.com/paper/frechet-chemnet-distance-a-metric-for
Repo	https://github.com/bioinf-jku/FCD
Framework	tf

A Neural Multi-sequence Alignment TeCHnique (NeuMATCH)


Title	A Neural Multi-sequence Alignment TeCHnique (NeuMATCH)
Authors	Pelin Dogan, Boyang Li, Leonid Sigal, Markus Gross
Abstract	The alignment of heterogeneous sequential data (video to text) is an important and challenging problem. Standard techniques for this task, including Dynamic Time Warping (DTW) and Conditional Random Fields (CRFs), suffer from inherent drawbacks. Mainly, the Markov assumption implies that, given the immediate past, future alignment decisions are independent of further history. The separation between similarity computation and alignment decision also prevents end-to-end training. In this paper, we propose an end-to-end neural architecture where alignment actions are implemented as moving data between stacks of Long Short-term Memory (LSTM) blocks. This flexible architecture supports a large variety of alignment tasks, including one-to-one, one-to-many, skipping unmatched elements, and (with extensions) non-monotonic alignment. Extensive experiments on semi-synthetic and real datasets show that our algorithm outperforms state-of-the-art baselines.
Tasks
Published	2018-02-19
URL	http://arxiv.org/abs/1803.00057v2
PDF	http://arxiv.org/pdf/1803.00057v2.pdf
PWC	https://paperswithcode.com/paper/a-neural-multi-sequence-alignment-technique
Repo	https://github.com/pelindogan/NeuMATCH
Framework	none