July 30, 2019

3230 words 16 mins read

Paper Group AWR 57

Multi-agent Reinforcement Learning in Sequential Social Dilemmas. Breaking the Softmax Bottleneck: A High-Rank RNN Language Model. Argotario: Computational Argumentation Meets Serious Games. RECOD Titans at ISIC Challenge 2017. Biased Importance Sampling for Deep Neural Network Training. A Bayesian Method for Joint Clustering of Vectorial Data and …


Title	Multi-agent Reinforcement Learning in Sequential Social Dilemmas
Authors	Joel Z. Leibo, Vinicius Zambaldi, Marc Lanctot, Janusz Marecki, Thore Graepel
Abstract	Matrix games like Prisoner’s Dilemma have guided research on social dilemmas for decades. However, they necessarily treat the choice to cooperate or defect as an atomic action. In real-world social dilemmas these choices are temporally extended. Cooperativeness is a property that applies to policies, not elementary actions. We introduce sequential social dilemmas that share the mixed incentive structure of matrix game social dilemmas but also require agents to learn policies that implement their strategic intentions. We analyze the dynamics of policies learned by multiple self-interested independent learning agents, each using its own deep Q-network, on two Markov games we introduce here: 1. a fruit Gathering game and 2. a Wolfpack hunting game. We characterize how learned behavior in each domain changes as a function of environmental factors including resource abundance. Our experiments show how conflict can emerge from competition over shared resources and shed light on how the sequential nature of real world social dilemmas affects cooperation.
Tasks	Multi-agent Reinforcement Learning
Published	2017-02-10
URL	http://arxiv.org/abs/1702.03037v1
PDF	http://arxiv.org/pdf/1702.03037v1.pdf
PWC	https://paperswithcode.com/paper/multi-agent-reinforcement-learning-in
Repo	https://github.com/eugenevinitsky/sequential_social_dilemma_games
Framework	none

Breaking the Softmax Bottleneck: A High-Rank RNN Language Model


Title	Breaking the Softmax Bottleneck: A High-Rank RNN Language Model
Authors	Zhilin Yang, Zihang Dai, Ruslan Salakhutdinov, William W. Cohen
Abstract	We formulate language modeling as a matrix factorization problem, and show that the expressiveness of Softmax-based models (including the majority of neural language models) is limited by a Softmax bottleneck. Given that natural language is highly context-dependent, this further implies that in practice Softmax with distributed word embeddings does not have enough capacity to model natural language. We propose a simple and effective method to address this issue, and improve the state-of-the-art perplexities on Penn Treebank and WikiText-2 to 47.69 and 40.68 respectively. The proposed method also excels on the large-scale 1B Word dataset, outperforming the baseline by over 5.6 points in perplexity.
Tasks	Language Modelling, Word Embeddings
Published	2017-11-10
URL	http://arxiv.org/abs/1711.03953v4
PDF	http://arxiv.org/pdf/1711.03953v4.pdf
PWC	https://paperswithcode.com/paper/breaking-the-softmax-bottleneck-a-high-rank
Repo	https://github.com/nunezpaul/MNIST
Framework	tf

Argotario: Computational Argumentation Meets Serious Games


Title	Argotario: Computational Argumentation Meets Serious Games
Authors	Ivan Habernal, Raffael Hannemann, Christian Pollak, Christopher Klamm, Patrick Pauli, Iryna Gurevych
Abstract	An important skill in critical thinking and argumentation is the ability to spot and recognize fallacies. Fallacious arguments, omnipresent in argumentative discourse, can be deceptive, manipulative, or simply leading to `wrong moves’ in a discussion. Despite their importance, argumentation scholars and NLP researchers with focus on argumentation quality have not yet investigated fallacies empirically. The nonexistence of resources dealing with fallacious argumentation calls for scalable approaches to data acquisition and annotation, for which the serious games methodology offers an appealing, yet unexplored, alternative. We present Argotario, a serious game that deals with fallacies in everyday argumentation. Argotario is a multilingual, open-source, platform-independent application with strong educational aspects, accessible at www.argotario.net. \|
Tasks
Published	2017-07-19
URL	http://arxiv.org/abs/1707.06002v1
PDF	http://arxiv.org/pdf/1707.06002v1.pdf
PWC	https://paperswithcode.com/paper/argotario-computational-argumentation-meets
Repo	https://github.com/UKPLab/argotario
Framework	none

RECOD Titans at ISIC Challenge 2017


Title	RECOD Titans at ISIC Challenge 2017
Authors	Afonso Menegola, Julia Tavares, Michel Fornaciali, Lin Tzy Li, Sandra Avila, Eduardo Valle
Abstract	This extended abstract describes the participation of RECOD Titans in parts 1 and 3 of the ISIC Challenge 2017 “Skin Lesion Analysis Towards Melanoma Detection” (ISBI 2017). Although our team has a long experience with melanoma classification, the ISIC Challenge 2017 was the very first time we worked on skin-lesion segmentation. For part 1 (segmentation), our final submission used four of our models: two trained with all 2000 samples, without a validation split, for 250 and for 500 epochs respectively; and other two trained and validated with two different 1600/400 splits, for 220 epochs. Those four models, individually, achieved between 0.780 and 0.783 official validation scores. Our final submission averaged the output of those four models achieved a score of 0.793. For part 3 (classification), the submitted test run as well as our last official validation run were the result from a meta-model that assembled seven base deep-learning models: three based on Inception-V4 trained on our largest dataset; three based on Inception trained on our smallest dataset; and one based on ResNet-101 trained on our smaller dataset. The results of those component models were stacked in a meta-learning layer based on an SVM trained on the validation set of our largest dataset.
Tasks	Lesion Segmentation, Meta-Learning
Published	2017-03-14
URL	http://arxiv.org/abs/1703.04819v1
PDF	http://arxiv.org/pdf/1703.04819v1.pdf
PWC	https://paperswithcode.com/paper/recod-titans-at-isic-challenge-2017
Repo	https://github.com/learningtitans/isbi2017-part1
Framework	none

Biased Importance Sampling for Deep Neural Network Training


Title	Biased Importance Sampling for Deep Neural Network Training
Authors	Angelos Katharopoulos, François Fleuret
Abstract	Importance sampling has been successfully used to accelerate stochastic optimization in many convex problems. However, the lack of an efficient way to calculate the importance still hinders its application to Deep Learning. In this paper, we show that the loss value can be used as an alternative importance metric, and propose a way to efficiently approximate it for a deep model, using a small model trained for that purpose in parallel. This method allows in particular to utilize a biased gradient estimate that implicitly optimizes a soft max-loss, and leads to better generalization performance. While such method suffers from a prohibitively high variance of the gradient estimate when using a standard stochastic optimizer, we show that when it is combined with our sampling mechanism, it results in a reliable procedure. We showcase the generality of our method by testing it on both image classification and language modeling tasks using deep convolutional and recurrent neural networks. In particular, our method results in 30% faster training of a CNN for CIFAR10 than when using uniform sampling.
Tasks	Image Classification, Language Modelling, Stochastic Optimization
Published	2017-05-31
URL	http://arxiv.org/abs/1706.00043v2
PDF	http://arxiv.org/pdf/1706.00043v2.pdf
PWC	https://paperswithcode.com/paper/biased-importance-sampling-for-deep-neural
Repo	https://github.com/idiap/importance-sampling
Framework	tf

A Bayesian Method for Joint Clustering of Vectorial Data and Network Data


Title	A Bayesian Method for Joint Clustering of Vectorial Data and Network Data
Authors	Yunchuan Kong, Xiaodan Fan
Abstract	We present a new model-based integrative method for clustering objects given both vectorial data, which describes the feature of each object, and network data, which indicates the similarity of connected objects. The proposed general model is able to cluster the two types of data simultaneously within one integrative probabilistic model, while traditional methods can only handle one data type or depend on transforming one data type to another. Bayesian inference of the clustering is conducted based on a Markov chain Monte Carlo algorithm. A special case of the general model combining the Gaussian mixture model and the stochastic block model is extensively studied. We used both synthetic data and real data to evaluate this new method and compare it with alternative methods. The results show that our simultaneous clustering method performs much better. This improvement is due to the power of the model-based probabilistic approach for efficiently integrating information.
Tasks	Bayesian Inference
Published	2017-10-24
URL	http://arxiv.org/abs/1710.08846v1
PDF	http://arxiv.org/pdf/1710.08846v1.pdf
PWC	https://paperswithcode.com/paper/a-bayesian-method-for-joint-clustering-of
Repo	https://github.com/yunchuankong/SharedClustering
Framework	none

RobustFill: Neural Program Learning under Noisy I/O


Title	RobustFill: Neural Program Learning under Noisy I/O
Authors	Jacob Devlin, Jonathan Uesato, Surya Bhupatiraju, Rishabh Singh, Abdel-rahman Mohamed, Pushmeet Kohli
Abstract	The problem of automatically generating a computer program from some specification has been studied since the early days of AI. Recently, two competing approaches for automatic program learning have received significant attention: (1) neural program synthesis, where a neural network is conditioned on input/output (I/O) examples and learns to generate a program, and (2) neural program induction, where a neural network generates new outputs directly using a latent program representation. Here, for the first time, we directly compare both approaches on a large-scale, real-world learning task. We additionally contrast to rule-based program synthesis, which uses hand-crafted semantics to guide the program generation. Our neural models use a modified attention RNN to allow encoding of variable-sized sets of I/O pairs. Our best synthesis model achieves 92% accuracy on a real-world test set, compared to the 34% accuracy of the previous best neural synthesis approach. The synthesis model also outperforms a comparable induction model on this task, but we more importantly demonstrate that the strength of each approach is highly dependent on the evaluation metric and end-user application. Finally, we show that we can train our neural models to remain very robust to the type of noise expected in real-world data (e.g., typos), while a highly-engineered rule-based system fails entirely.
Tasks	Program Synthesis
Published	2017-03-21
URL	http://arxiv.org/abs/1703.07469v1
PDF	http://arxiv.org/pdf/1703.07469v1.pdf
PWC	https://paperswithcode.com/paper/robustfill-neural-program-learning-under
Repo	https://github.com/insperatum/pinn
Framework	pytorch

State-of-the-art Speech Recognition With Sequence-to-Sequence Models


Title	State-of-the-art Speech Recognition With Sequence-to-Sequence Models
Authors	Chung-Cheng Chiu, Tara N. Sainath, Yonghui Wu, Rohit Prabhavalkar, Patrick Nguyen, Zhifeng Chen, Anjuli Kannan, Ron J. Weiss, Kanishka Rao, Ekaterina Gonina, Navdeep Jaitly, Bo Li, Jan Chorowski, Michiel Bacchiani
Abstract	Attention-based encoder-decoder architectures such as Listen, Attend, and Spell (LAS), subsume the acoustic, pronunciation and language model components of a traditional automatic speech recognition (ASR) system into a single neural network. In previous work, we have shown that such architectures are comparable to state-of-theart ASR systems on dictation tasks, but it was not clear if such architectures would be practical for more challenging tasks such as voice search. In this work, we explore a variety of structural and optimization improvements to our LAS model which significantly improve performance. On the structural side, we show that word piece models can be used instead of graphemes. We also introduce a multi-head attention architecture, which offers improvements over the commonly-used single-head attention. On the optimization side, we explore synchronous training, scheduled sampling, label smoothing, and minimum word error rate optimization, which are all shown to improve accuracy. We present results with a unidirectional LSTM encoder for streaming recognition. On a 12, 500 hour voice search task, we find that the proposed changes improve the WER from 9.2% to 5.6%, while the best conventional system achieves 6.7%; on a dictation task our model achieves a WER of 4.1% compared to 5% for the conventional system.
Tasks	Language Modelling, Speech Recognition
Published	2017-12-05
URL	http://arxiv.org/abs/1712.01769v6
PDF	http://arxiv.org/pdf/1712.01769v6.pdf
PWC	https://paperswithcode.com/paper/state-of-the-art-speech-recognition-with
Repo	https://github.com/sh951011/Attention-Implementation
Framework	pytorch

Human-like Clustering with Deep Convolutional Neural Networks


Title	Human-like Clustering with Deep Convolutional Neural Networks
Authors	Ali Borji, Aysegul Dundar
Abstract	Classification and clustering have been studied separately in machine learning and computer vision. Inspired by the recent success of deep learning models in solving various vision problems (e.g., object recognition, semantic segmentation) and the fact that humans serve as the gold standard in assessing clustering algorithms, here, we advocate for a unified treatment of the two problems and suggest that hierarchical frameworks that progressively build complex patterns on top of the simpler ones (e.g., convolutional neural networks) offer a promising solution. We do not dwell much on the learning mechanisms in these frameworks as they are still a matter of debate, with respect to biological constraints. Instead, we emphasize on the compositionality of the real world structures and objects. In particular, we show that CNNs, trained end to end using back propagation with noisy labels, are able to cluster data points belonging to several overlapping shapes, and do so much better than the state of the art algorithms. The main takeaway lesson from our study is that mechanisms of human vision, particularly the hierarchal organization of the visual ventral stream should be taken into account in clustering algorithms (e.g., for learning representations in an unsupervised manner or with minimum supervision) to reach human level clustering performance. This, by no means, suggests that other methods do not hold merits. For example, methods relying on pairwise affinities (e.g., spectral clustering) have been very successful in many scenarios but still fail in some cases (e.g., overlapping clusters).
Tasks	Object Recognition, Semantic Segmentation
Published	2017-06-15
URL	http://arxiv.org/abs/1706.05048v2
PDF	http://arxiv.org/pdf/1706.05048v2.pdf
PWC	https://paperswithcode.com/paper/human-like-clustering-with-deep-convolutional
Repo	https://github.com/Naghipourfar/Deep-Clustering
Framework	tf

Personalizing Session-based Recommendations with Hierarchical Recurrent Neural Networks


Title	Personalizing Session-based Recommendations with Hierarchical Recurrent Neural Networks
Authors	Massimo Quadrana, Alexandros Karatzoglou, Balázs Hidasi, Paolo Cremonesi
Abstract	Session-based recommendations are highly relevant in many modern on-line services (e.g. e-commerce, video streaming) and recommendation settings. Recently, Recurrent Neural Networks have been shown to perform very well in session-based settings. While in many session-based recommendation domains user identifiers are hard to come by, there are also domains in which user profiles are readily available. We propose a seamless way to personalize RNN models with cross-session information transfer and devise a Hierarchical RNN model that relays end evolves latent hidden states of the RNNs across user sessions. Results on two industry datasets show large improvements over the session-only RNNs.
Tasks	Session-Based Recommendations
Published	2017-06-13
URL	http://arxiv.org/abs/1706.04148v5
PDF	http://arxiv.org/pdf/1706.04148v5.pdf
PWC	https://paperswithcode.com/paper/personalizing-session-based-recommendations
Repo	https://github.com/mquad/hgru4rec
Framework	none

Visualisation and ‘diagnostic classifiers’ reveal how recurrent and recursive neural networks process hierarchical structure


Title	Visualisation and ‘diagnostic classifiers’ reveal how recurrent and recursive neural networks process hierarchical structure
Authors	Dieuwke Hupkes, Sara Veldhoen, Willem Zuidema
Abstract	We investigate how neural networks can learn and process languages with hierarchical, compositional semantics. To this end, we define the artificial task of processing nested arithmetic expressions, and study whether different types of neural networks can learn to compute their meaning. We find that recursive neural networks can find a generalising solution to this problem, and we visualise this solution by breaking it up in three steps: project, sum and squash. As a next step, we investigate recurrent neural networks, and show that a gated recurrent unit, that processes its input incrementally, also performs very well on this task. To develop an understanding of what the recurrent network encodes, visualisation techniques alone do not suffice. Therefore, we develop an approach where we formulate and test multiple hypotheses on the information encoded and processed by the network. For each hypothesis, we derive predictions about features of the hidden state representations at each time step, and train ‘diagnostic classifiers’ to test those predictions. Our results indicate that the networks follow a strategy similar to our hypothesised ‘cumulative strategy’, which explains the high accuracy of the network on novel expressions, the generalisation to longer expressions than seen in training, and the mild deterioration with increasing length. This is turn shows that diagnostic classifiers can be a useful technique for opening up the black box of neural networks. We argue that diagnostic classification, unlike most visualisation techniques, does scale up from small networks in a toy domain, to larger and deeper recurrent networks dealing with real-life data, and may therefore contribute to a better understanding of the internal dynamics of current state-of-the-art models in natural language processing.
Tasks
Published	2017-11-28
URL	http://arxiv.org/abs/1711.10203v2
PDF	http://arxiv.org/pdf/1711.10203v2.pdf
PWC	https://paperswithcode.com/paper/visualisation-and-diagnostic-classifiers
Repo	https://github.com/i-machine-think/diagnnose
Framework	pytorch

Twin Networks: Matching the Future for Sequence Generation


Title	Twin Networks: Matching the Future for Sequence Generation
Authors	Dmitriy Serdyuk, Nan Rosemary Ke, Alessandro Sordoni, Adam Trischler, Chris Pal, Yoshua Bengio
Abstract	We propose a simple technique for encouraging generative RNNs to plan ahead. We train a “backward” recurrent network to generate a given sequence in reverse order, and we encourage states of the forward model to predict cotemporal states of the backward model. The backward network is used only during training, and plays no role during sampling or inference. We hypothesize that our approach eases modeling of long-term dependencies by implicitly forcing the forward states to hold information about the longer-term future (as contained in the backward states). We show empirically that our approach achieves 9% relative improvement for a speech recognition task, and achieves significant improvement on a COCO caption generation task.
Tasks	Speech Recognition
Published	2017-08-22
URL	http://arxiv.org/abs/1708.06742v3
PDF	http://arxiv.org/pdf/1708.06742v3.pdf
PWC	https://paperswithcode.com/paper/twin-networks-matching-the-future-for
Repo	https://github.com/mravanelli/pytorch-kaldi
Framework	pytorch

Machine Comprehension by Text-to-Text Neural Question Generation


Title	Machine Comprehension by Text-to-Text Neural Question Generation
Authors	Xingdi Yuan, Tong Wang, Caglar Gulcehre, Alessandro Sordoni, Philip Bachman, Sandeep Subramanian, Saizheng Zhang, Adam Trischler
Abstract	We propose a recurrent neural model that generates natural-language questions from documents, conditioned on answers. We show how to train the model using a combination of supervised and reinforcement learning. After teacher forcing for standard maximum likelihood training, we fine-tune the model using policy gradient techniques to maximize several rewards that measure question quality. Most notably, one of these rewards is the performance of a question-answering system. We motivate question generation as a means to improve the performance of question answering systems. Our model is trained and evaluated on the recent question-answering dataset SQuAD.
Tasks	Question Answering, Question Generation, Reading Comprehension
Published	2017-05-04
URL	http://arxiv.org/abs/1705.02012v2
PDF	http://arxiv.org/pdf/1705.02012v2.pdf
PWC	https://paperswithcode.com/paper/machine-comprehension-by-text-to-text-neural
Repo	https://github.com/GauthierDmn/question_generation
Framework	pytorch

Embedding Watermarks into Deep Neural Networks


Title	Embedding Watermarks into Deep Neural Networks
Authors	Yusuke Uchida, Yuki Nagai, Shigeyuki Sakazawa, Shin’ichi Satoh
Abstract	Deep neural networks have recently achieved significant progress. Sharing trained models of these deep neural networks is very important in the rapid progress of researching or developing deep neural network systems. At the same time, it is necessary to protect the rights of shared trained models. To this end, we propose to use a digital watermarking technology to protect intellectual property or detect intellectual property infringement of trained models. Firstly, we formulate a new problem: embedding watermarks into deep neural networks. We also define requirements, embedding situations, and attack types for watermarking to deep neural networks. Secondly, we propose a general framework to embed a watermark into model parameters using a parameter regularizer. Our approach does not hurt the performance of networks into which a watermark is embedded. Finally, we perform comprehensive experiments to reveal the potential of watermarking to deep neural networks as a basis of this new problem. We show that our framework can embed a watermark in the situations of training a network from scratch, fine-tuning, and distilling without hurting the performance of a deep neural network. The embedded watermark does not disappear even after fine-tuning or parameter pruning; the watermark completely remains even after removing 65% of parameters were pruned. The implementation of this research is: https://github.com/yu4u/dnn-watermark
Tasks
Published	2017-01-15
URL	http://arxiv.org/abs/1701.04082v2
PDF	http://arxiv.org/pdf/1701.04082v2.pdf
PWC	https://paperswithcode.com/paper/embedding-watermarks-into-deep-neural
Repo	https://github.com/yu4u/dnn-watermark
Framework	tf

VEEGAN: Reducing Mode Collapse in GANs using Implicit Variational Learning


Title	VEEGAN: Reducing Mode Collapse in GANs using Implicit Variational Learning
Authors	Akash Srivastava, Lazar Valkov, Chris Russell, Michael U. Gutmann, Charles Sutton
Abstract	Deep generative models provide powerful tools for distributions over complicated manifolds, such as those of natural images. But many of these methods, including generative adversarial networks (GANs), can be difficult to train, in part because they are prone to mode collapse, which means that they characterize only a few modes of the true distribution. To address this, we introduce VEEGAN, which features a reconstructor network, reversing the action of the generator by mapping from data to noise. Our training objective retains the original asymptotic consistency guarantee of GANs, and can be interpreted as a novel autoencoder loss over the noise. In sharp contrast to a traditional autoencoder over data points, VEEGAN does not require specifying a loss function over the data, but rather only over the representations, which are standard normal by assumption. On an extensive set of synthetic and real world image datasets, VEEGAN indeed resists mode collapsing to a far greater extent than other recent GAN variants, and produces more realistic samples.
Tasks
Published	2017-05-22
URL	http://arxiv.org/abs/1705.07761v3
PDF	http://arxiv.org/pdf/1705.07761v3.pdf
PWC	https://paperswithcode.com/paper/veegan-reducing-mode-collapse-in-gans-using
Repo	https://github.com/alex98chen/testGAN
Framework	tf