July 30, 2019

2837 words 14 mins read

Paper Group AWR 53

A physical model for efficient ranking in networks. Room for improvement in automatic image description: an error analysis. Autoencoder Node Saliency: Selecting Relevant Latent Representations. Learned Optimizers that Scale and Generalize. DANCin SEQ2SEQ: Fooling Text Classifiers with Adversarial Text Example Generation. Predicting Pairwise Relatio …

A physical model for efficient ranking in networks


Title	A physical model for efficient ranking in networks
Authors	Caterina De Bacco, Daniel B. Larremore, Cristopher Moore
Abstract	We present a physically-inspired model and an efficient algorithm to infer hierarchical rankings of nodes in directed networks. It assigns real-valued ranks to nodes rather than simply ordinal ranks, and it formalizes the assumption that interactions are more likely to occur between individuals with similar ranks. It provides a natural statistical significance test for the inferred hierarchy, and it can be used to perform inference tasks such as predicting the existence or direction of edges. The ranking is obtained by solving a linear system of equations, which is sparse if the network is; thus the resulting algorithm is extremely efficient and scalable. We illustrate these findings by analyzing real and synthetic data, including datasets from animal behavior, faculty hiring, social support networks, and sports tournaments. We show that our method often outperforms a variety of others, in both speed and accuracy, in recovering the underlying ranks and predicting edge directions.
Tasks
Published	2017-09-03
URL	http://arxiv.org/abs/1709.09002v4
PDF	http://arxiv.org/pdf/1709.09002v4.pdf
PWC	https://paperswithcode.com/paper/a-physical-model-for-efficient-ranking-in
Repo	https://github.com/cdebacco/SpringRank
Framework	none

Room for improvement in automatic image description: an error analysis


Title	Room for improvement in automatic image description: an error analysis
Authors	Emiel van Miltenburg, Desmond Elliott
Abstract	In recent years we have seen rapid and significant progress in automatic image description but what are the open problems in this area? Most work has been evaluated using text-based similarity metrics, which only indicate that there have been improvements, without explaining what has improved. In this paper, we present a detailed error analysis of the descriptions generated by a state-of-the-art attention-based model. Our analysis operates on two levels: first we check the descriptions for accuracy, and then we categorize the types of errors we observe in the inaccurate descriptions. We find only 20% of the descriptions are free from errors, and surprisingly that 26% are unrelated to the image. Finally, we manually correct the most frequently occurring error types (e.g. gender identification) to estimate the performance reward for addressing these errors, observing gains of 0.2–1 BLEU point per type.
Tasks
Published	2017-04-13
URL	http://arxiv.org/abs/1704.04198v1
PDF	http://arxiv.org/pdf/1704.04198v1.pdf
PWC	https://paperswithcode.com/paper/room-for-improvement-in-automatic-image
Repo	https://github.com/evanmiltenburg/ErrorAnalysis
Framework	none

Autoencoder Node Saliency: Selecting Relevant Latent Representations


Title	Autoencoder Node Saliency: Selecting Relevant Latent Representations
Authors	Ya Ju Fan
Abstract	The autoencoder is an artificial neural network model that learns hidden representations of unlabeled data. With a linear transfer function it is similar to the principal component analysis (PCA). While both methods use weight vectors for linear transformations, the autoencoder does not come with any indication similar to the eigenvalues in PCA that are paired with the eigenvectors. We propose a novel supervised node saliency (SNS) method that ranks the hidden nodes by comparing class distributions of latent representations against a fixed reference distribution. The latent representations of a hidden node can be described using a one-dimensional histogram. We apply normalized entropy difference (NED) to measure the “interestingness” of the histograms, and conclude a property for NED values to identify a good classifying node. By applying our methods to real data sets, we demonstrate the ability of SNS to explain what the trained autoencoders have learned.
Tasks
Published	2017-11-21
URL	http://arxiv.org/abs/1711.07871v2
PDF	http://arxiv.org/pdf/1711.07871v2.pdf
PWC	https://paperswithcode.com/paper/autoencoder-node-saliency-selecting-relevant
Repo	https://github.com/LLNL/ANS
Framework	none

Learned Optimizers that Scale and Generalize


Title	Learned Optimizers that Scale and Generalize
Authors	Olga Wichrowska, Niru Maheswaranathan, Matthew W. Hoffman, Sergio Gomez Colmenarejo, Misha Denil, Nando de Freitas, Jascha Sohl-Dickstein
Abstract	Learning to learn has emerged as an important direction for achieving artificial intelligence. Two of the primary barriers to its adoption are an inability to scale to larger problems and a limited ability to generalize to new tasks. We introduce a learned gradient descent optimizer that generalizes well to new tasks, and which has significantly reduced memory and computation overhead. We achieve this by introducing a novel hierarchical RNN architecture, with minimal per-parameter overhead, augmented with additional architectural features that mirror the known structure of optimization tasks. We also develop a meta-training ensemble of small, diverse optimization tasks capturing common properties of loss landscapes. The optimizer learns to outperform RMSProp/ADAM on problems in this corpus. More importantly, it performs comparably or better when applied to small convolutional neural networks, despite seeing no neural networks in its meta-training set. Finally, it generalizes to train Inception V3 and ResNet V2 architectures on the ImageNet dataset for thousands of steps, optimization problems that are of a vastly different scale than those it was trained on. We release an open source implementation of the meta-training algorithm.
Tasks
Published	2017-03-14
URL	http://arxiv.org/abs/1703.04813v4
PDF	http://arxiv.org/pdf/1703.04813v4.pdf
PWC	https://paperswithcode.com/paper/learned-optimizers-that-scale-and-generalize
Repo	https://github.com/tensorflow/models
Framework	tf

DANCin SEQ2SEQ: Fooling Text Classifiers with Adversarial Text Example Generation


Title	DANCin SEQ2SEQ: Fooling Text Classifiers with Adversarial Text Example Generation
Authors	Catherine Wong
Abstract	Machine learning models are powerful but fallible. Generating adversarial examples - inputs deliberately crafted to cause model misclassification or other errors - can yield important insight into model assumptions and vulnerabilities. Despite significant recent work on adversarial example generation targeting image classifiers, relatively little work exists exploring adversarial example generation for text classifiers; additionally, many existing adversarial example generation algorithms require full access to target model parameters, rendering them impractical for many real-world attacks. In this work, we introduce DANCin SEQ2SEQ, a GAN-inspired algorithm for adversarial text example generation targeting largely black-box text classifiers. We recast adversarial text example generation as a reinforcement learning problem, and demonstrate that our algorithm offers preliminary but promising steps towards generating semantically meaningful adversarial text examples in a real-world attack scenario.
Tasks	Adversarial Text
Published	2017-12-14
URL	http://arxiv.org/abs/1712.05419v1
PDF	http://arxiv.org/pdf/1712.05419v1.pdf
PWC	https://paperswithcode.com/paper/dancin-seq2seq-fooling-text-classifiers-with
Repo	https://github.com/CatherineWong/dancin_seq2seq
Framework	pytorch

Predicting Pairwise Relations with Neural Similarity Encoders


Title	Predicting Pairwise Relations with Neural Similarity Encoders
Authors	Franziska Horn, Klaus-Robert Müller
Abstract	Matrix factorization is at the heart of many machine learning algorithms, for example, dimensionality reduction (e.g. kernel PCA) or recommender systems relying on collaborative filtering. Understanding a singular value decomposition (SVD) of a matrix as a neural network optimization problem enables us to decompose large matrices efficiently while dealing naturally with missing values in the given matrix. But most importantly, it allows us to learn the connection between data points’ feature vectors and the matrix containing information about their pairwise relations. In this paper we introduce a novel neural network architecture termed Similarity Encoder (SimEc), which is designed to simultaneously factorize a given target matrix while also learning the mapping to project the data points’ feature vectors into a similarity preserving embedding space. This makes it possible to, for example, easily compute out-of-sample solutions for new data points. Additionally, we demonstrate that SimEc can preserve non-metric similarities and even predict multiple pairwise relations between data points at once.
Tasks	Dimensionality Reduction, Recommendation Systems
Published	2017-02-06
URL	http://arxiv.org/abs/1702.01824v2
PDF	http://arxiv.org/pdf/1702.01824v2.pdf
PWC	https://paperswithcode.com/paper/predicting-pairwise-relations-with-neural
Repo	https://github.com/cod3licious/simec
Framework	pytorch

Efficient Vector Representation for Documents through Corruption


Title	Efficient Vector Representation for Documents through Corruption
Authors	Minmin Chen
Abstract	We present an efficient document representation learning framework, Document Vector through Corruption (Doc2VecC). Doc2VecC represents each document as a simple average of word embeddings. It ensures a representation generated as such captures the semantic meanings of the document during learning. A corruption model is included, which introduces a data-dependent regularization that favors informative or rare words while forcing the embeddings of common and non-discriminative ones to be close to zero. Doc2VecC produces significantly better word embeddings than Word2Vec. We compare Doc2VecC with several state-of-the-art document representation learning algorithms. The simple model architecture introduced by Doc2VecC matches or out-performs the state-of-the-art in generating high-quality document representations for sentiment analysis, document classification as well as semantic relatedness tasks. The simplicity of the model enables training on billions of words per hour on a single machine. At the same time, the model is very efficient in generating representations of unseen documents at test time.
Tasks	Document Classification, Representation Learning, Sentiment Analysis, Word Embeddings
Published	2017-07-08
URL	http://arxiv.org/abs/1707.02377v1
PDF	http://arxiv.org/pdf/1707.02377v1.pdf
PWC	https://paperswithcode.com/paper/efficient-vector-representation-for-documents
Repo	https://github.com/mchen24/iclr2017
Framework	none

One-step and Two-step Classification for Abusive Language Detection on Twitter


Title	One-step and Two-step Classification for Abusive Language Detection on Twitter
Authors	Ji Ho Park, Pascale Fung
Abstract	Automatic abusive language detection is a difficult but important task for online social media. Our research explores a two-step approach of performing classification on abusive language and then classifying into specific types and compares it with one-step approach of doing one multi-class classification for detecting sexist and racist languages. With a public English Twitter corpus of 20 thousand tweets in the type of sexism and racism, our approach shows a promising performance of 0.827 F-measure by using HybridCNN in one-step and 0.824 F-measure by using logistic regression in two-steps.
Tasks
Published	2017-06-05
URL	http://arxiv.org/abs/1706.01206v1
PDF	http://arxiv.org/pdf/1706.01206v1.pdf
PWC	https://paperswithcode.com/paper/one-step-and-two-step-classification-for
Repo	https://github.com/younggns/comparative-abusive-lang
Framework	tf

Topological Recurrent Neural Network for Diffusion Prediction


Title	Topological Recurrent Neural Network for Diffusion Prediction
Authors	Jia Wang, Vincent W. Zheng, Zemin Liu, Kevin Chen-Chuan Chang
Abstract	In this paper, we study the problem of using representation learning to assist information diffusion prediction on graphs. In particular, we aim at estimating the probability of an inactive node to be activated next in a cascade. Despite the success of recent deep learning methods for diffusion, we find that they often underexplore the cascade structure. We consider a cascade as not merely a sequence of nodes ordered by their activation time stamps; instead, it has a richer structure indicating the diffusion process over the data graph. As a result, we introduce a new data model, namely diffusion topologies, to fully describe the cascade structure. We find it challenging to model diffusion topologies, which are dynamic directed acyclic graphs (DAGs), with the existing neural networks. Therefore, we propose a novel topological recurrent neural network, namely Topo-LSTM, for modeling dynamic DAGs. We customize Topo-LSTM for the diffusion prediction task, and show it improves the state-of-the-art baselines, by 20.1%–56.6% (MAP) relatively, across multiple real-world data sets. Our code and data sets are available online at https://github.com/vwz/topolstm.
Tasks	Representation Learning
Published	2017-11-28
URL	http://arxiv.org/abs/1711.10162v2
PDF	http://arxiv.org/pdf/1711.10162v2.pdf
PWC	https://paperswithcode.com/paper/topological-recurrent-neural-network-for
Repo	https://github.com/vwz/topolstm
Framework	none

Future Person Localization in First-Person Videos


Title	Future Person Localization in First-Person Videos
Authors	Takuma Yagi, Karttikeya Mangalam, Ryo Yonetani, Yoichi Sato
Abstract	We present a new task that predicts future locations of people observed in first-person videos. Consider a first-person video stream continuously recorded by a wearable camera. Given a short clip of a person that is extracted from the complete stream, we aim to predict that person’s location in future frames. To facilitate this future person localization ability, we make the following three key observations: a) First-person videos typically involve significant ego-motion which greatly affects the location of the target person in future frames; b) Scales of the target person act as a salient cue to estimate a perspective effect in first-person videos; c) First-person videos often capture people up-close, making it easier to leverage target poses (e.g., where they look) for predicting their future locations. We incorporate these three observations into a prediction framework with a multi-stream convolution-deconvolution architecture. Experimental results reveal our method to be effective on our new dataset as well as on a public social interaction dataset.
Tasks
Published	2017-11-30
URL	http://arxiv.org/abs/1711.11217v2
PDF	http://arxiv.org/pdf/1711.11217v2.pdf
PWC	https://paperswithcode.com/paper/future-person-localization-in-first-person
Repo	https://github.com/takumayagi/fpl
Framework	none

Deep Person Re-Identification with Improved Embedding and Efficient Training


Title	Deep Person Re-Identification with Improved Embedding and Efficient Training
Authors	Haibo Jin, Xiaobo Wang, Shengcai Liao, Stan Z. Li
Abstract	Person re-identification task has been greatly boosted by deep convolutional neural networks (CNNs) in recent years. The core of which is to enlarge the inter-class distinction as well as reduce the intra-class variance. However, to achieve this, existing deep models prefer to adopt image pairs or triplets to form verification loss, which is inefficient and unstable since the number of training pairs or triplets grows rapidly as the number of training data grows. Moreover, their performance is limited since they ignore the fact that different dimension of embedding may play different importance. In this paper, we propose to employ identification loss with center loss to train a deep model for person re-identification. The training process is efficient since it does not require image pairs or triplets for training while the inter-class distinction and intra-class variance are well handled. To boost the performance, a new feature reweighting (FRW) layer is designed to explicitly emphasize the importance of each embedding dimension, thus leading to an improved embedding. Experiments on several benchmark datasets have shown the superiority of our method over the state-of-the-art alternatives on both accuracy and speed.
Tasks	Person Re-Identification
Published	2017-05-09
URL	http://arxiv.org/abs/1705.03332v3
PDF	http://arxiv.org/pdf/1705.03332v3.pdf
PWC	https://paperswithcode.com/paper/deep-person-re-identification-with-improved
Repo	https://github.com/jhb86253817/tf-re-id
Framework	tf

Towards Visual Ego-motion Learning in Robots


Title	Towards Visual Ego-motion Learning in Robots
Authors	Sudeep Pillai, John J. Leonard
Abstract	Many model-based Visual Odometry (VO) algorithms have been proposed in the past decade, often restricted to the type of camera optics, or the underlying motion manifold observed. We envision robots to be able to learn and perform these tasks, in a minimally supervised setting, as they gain more experience. To this end, we propose a fully trainable solution to visual ego-motion estimation for varied camera optics. We propose a visual ego-motion learning architecture that maps observed optical flow vectors to an ego-motion density estimate via a Mixture Density Network (MDN). By modeling the architecture as a Conditional Variational Autoencoder (C-VAE), our model is able to provide introspective reasoning and prediction for ego-motion induced scene-flow. Additionally, our proposed model is especially amenable to bootstrapped ego-motion learning in robots where the supervision in ego-motion estimation for a particular camera sensor can be obtained from standard navigation-based sensor fusion strategies (GPS/INS and wheel-odometry fusion). Through experiments, we show the utility of our proposed approach in enabling the concept of self-supervised learning for visual ego-motion estimation in autonomous robots.
Tasks	Motion Estimation, Optical Flow Estimation, Sensor Fusion, Visual Odometry
Published	2017-05-29
URL	http://arxiv.org/abs/1705.10279v1
PDF	http://arxiv.org/pdf/1705.10279v1.pdf
PWC	https://paperswithcode.com/paper/towards-visual-ego-motion-learning-in-robots
Repo	https://github.com/spillai/learning-egomotion
Framework	none

Intrinsic Motivation and Automatic Curricula via Asymmetric Self-Play


Title	Intrinsic Motivation and Automatic Curricula via Asymmetric Self-Play
Authors	Sainbayar Sukhbaatar, Zeming Lin, Ilya Kostrikov, Gabriel Synnaeve, Arthur Szlam, Rob Fergus
Abstract	We describe a simple scheme that allows an agent to learn about its environment in an unsupervised manner. Our scheme pits two versions of the same agent, Alice and Bob, against one another. Alice proposes a task for Bob to complete; and then Bob attempts to complete the task. In this work we will focus on two kinds of environments: (nearly) reversible environments and environments that can be reset. Alice will “propose” the task by doing a sequence of actions and then Bob must undo or repeat them, respectively. Via an appropriate reward structure, Alice and Bob automatically generate a curriculum of exploration, enabling unsupervised training of the agent. When Bob is deployed on an RL task within the environment, this unsupervised training reduces the number of supervised episodes needed to learn, and in some cases converges to a higher reward.
Tasks
Published	2017-03-15
URL	http://arxiv.org/abs/1703.05407v5
PDF	http://arxiv.org/pdf/1703.05407v5.pdf
PWC	https://paperswithcode.com/paper/intrinsic-motivation-and-automatic-curricula
Repo	https://github.com/tesatory/hsp
Framework	pytorch

Deep Reinforcement Learning that Matters


Title	Deep Reinforcement Learning that Matters
Authors	Peter Henderson, Riashat Islam, Philip Bachman, Joelle Pineau, Doina Precup, David Meger
Abstract	In recent years, significant progress has been made in solving challenging problems across various domains using deep reinforcement learning (RL). Reproducing existing work and accurately judging the improvements offered by novel methods is vital to sustaining this progress. Unfortunately, reproducing results for state-of-the-art deep RL methods is seldom straightforward. In particular, non-determinism in standard benchmark environments, combined with variance intrinsic to the methods, can make reported results tough to interpret. Without significance metrics and tighter standardization of experimental reporting, it is difficult to determine whether improvements over the prior state-of-the-art are meaningful. In this paper, we investigate challenges posed by reproducibility, proper experimental techniques, and reporting procedures. We illustrate the variability in reported metrics and results when comparing against common baselines and suggest guidelines to make future results in deep RL more reproducible. We aim to spur discussion about how to ensure continued progress in the field by minimizing wasted effort stemming from results that are non-reproducible and easily misinterpreted.
Tasks
Published	2017-09-19
URL	http://arxiv.org/abs/1709.06560v3
PDF	http://arxiv.org/pdf/1709.06560v3.pdf
PWC	https://paperswithcode.com/paper/deep-reinforcement-learning-that-matters
Repo	https://github.com/chainer/chainerrl
Framework	none

Character-based Neural Embeddings for Tweet Clustering


Title	Character-based Neural Embeddings for Tweet Clustering
Authors	Svitlana Vakulenko, Lyndon Nixon, Mihai Lupu
Abstract	In this paper we show how the performance of tweet clustering can be improved by leveraging character-based neural networks. The proposed approach overcomes the limitations related to the vocabulary explosion in the word-based models and allows for the seamless processing of the multilingual content. Our evaluation results and code are available on-line at https://github.com/vendi12/tweet2vec_clustering
Tasks
Published	2017-03-15
URL	http://arxiv.org/abs/1703.05123v2
PDF	http://arxiv.org/pdf/1703.05123v2.pdf
PWC	https://paperswithcode.com/paper/character-based-neural-embeddings-for-tweet
Repo	https://github.com/vendi12/tweet2vec_clustering
Framework	none