July 29, 2019

2946 words 14 mins read

Paper Group AWR 133

Graph Convolutional Networks for Named Entity Recognition. PasMoQAP: A Parallel Asynchronous Memetic Algorithm for solving the Multi-Objective Quadratic Assignment Problem. SchNet: A continuous-filter convolutional neural network for modeling quantum interactions. Differentially Private Database Release via Kernel Mean Embeddings. Graph learning un …

Graph Convolutional Networks for Named Entity Recognition


Title	Graph Convolutional Networks for Named Entity Recognition
Authors	A. Cetoli, S. Bragaglia, A. D. O’Harney, M. Sloan
Abstract	In this paper we investigate the role of the dependency tree in a named entity recognizer upon using a set of GCN. We perform a comparison among different NER architectures and show that the grammar of a sentence positively influences the results. Experiments on the ontonotes dataset demonstrate consistent performance improvements, without requiring heavy feature engineering nor additional language-specific knowledge.
Tasks	Feature Engineering, Named Entity Recognition
Published	2017-09-28
URL	http://arxiv.org/abs/1709.10053v2
PDF	http://arxiv.org/pdf/1709.10053v2.pdf
PWC	https://paperswithcode.com/paper/graph-convolutional-networks-for-named-entity-1
Repo	https://github.com/contextscout/gcn_ner
Framework	tf

PasMoQAP: A Parallel Asynchronous Memetic Algorithm for solving the Multi-Objective Quadratic Assignment Problem


Title	PasMoQAP: A Parallel Asynchronous Memetic Algorithm for solving the Multi-Objective Quadratic Assignment Problem
Authors	Claudio Sanhueza, Francia Jimenez, Regina Berretta, Pablo Moscato
Abstract	Multi-Objective Optimization Problems (MOPs) have attracted growing attention during the last decades. Multi-Objective Evolutionary Algorithms (MOEAs) have been extensively used to address MOPs because are able to approximate a set of non-dominated high-quality solutions. The Multi-Objective Quadratic Assignment Problem (mQAP) is a MOP. The mQAP is a generalization of the classical QAP which has been extensively studied, and used in several real-life applications. The mQAP is defined as having as input several flows between the facilities which generate multiple cost functions that must be optimized simultaneously. In this study, we propose PasMoQAP, a parallel asynchronous memetic algorithm to solve the Multi-Objective Quadratic Assignment Problem. PasMoQAP is based on an island model that structures the population by creating sub-populations. The memetic algorithm on each island individually evolve a reduced population of solutions, and they asynchronously cooperate by sending selected solutions to the neighboring islands. The experimental results show that our approach significatively outperforms all the island-based variants of the multi-objective evolutionary algorithm NSGA-II. We show that PasMoQAP is a suitable alternative to solve the Multi-Objective Quadratic Assignment Problem.
Tasks
Published	2017-06-27
URL	http://arxiv.org/abs/1706.08700v1
PDF	http://arxiv.org/pdf/1706.08700v1.pdf
PWC	https://paperswithcode.com/paper/pasmoqap-a-parallel-asynchronous-memetic
Repo	https://github.com/csanhuezalobos/gar60
Framework	none

SchNet: A continuous-filter convolutional neural network for modeling quantum interactions


Title	SchNet: A continuous-filter convolutional neural network for modeling quantum interactions
Authors	Kristof T. Schütt, Pieter-Jan Kindermans, Huziel E. Sauceda, Stefan Chmiela, Alexandre Tkatchenko, Klaus-Robert Müller
Abstract	Deep learning has the potential to revolutionize quantum chemistry as it is ideally suited to learn representations for structured data and speed up the exploration of chemical space. While convolutional neural networks have proven to be the first choice for images, audio and video data, the atoms in molecules are not restricted to a grid. Instead, their precise locations contain essential physical information, that would get lost if discretized. Thus, we propose to use continuous-filter convolutional layers to be able to model local correlations without requiring the data to lie on a grid. We apply those layers in SchNet: a novel deep learning architecture modeling quantum interactions in molecules. We obtain a joint model for the total energy and interatomic forces that follows fundamental quantum-chemical principles. This includes rotationally invariant energy predictions and a smooth, differentiable potential energy surface. Our architecture achieves state-of-the-art performance for benchmarks of equilibrium molecules and molecular dynamics trajectories. Finally, we introduce a more challenging benchmark with chemical and structural variations that suggests the path for further work.
Tasks	Formation Energy
Published	2017-06-26
URL	http://arxiv.org/abs/1706.08566v5
PDF	http://arxiv.org/pdf/1706.08566v5.pdf
PWC	https://paperswithcode.com/paper/schnet-a-continuous-filter-convolutional
Repo	https://github.com/atomistic-machine-learning/schnetpack
Framework	pytorch

Differentially Private Database Release via Kernel Mean Embeddings


Title	Differentially Private Database Release via Kernel Mean Embeddings
Authors	Matej Balog, Ilya Tolstikhin, Bernhard Schölkopf
Abstract	We lay theoretical foundations for new database release mechanisms that allow third-parties to construct consistent estimators of population statistics, while ensuring that the privacy of each individual contributing to the database is protected. The proposed framework rests on two main ideas. First, releasing (an estimate of) the kernel mean embedding of the data generating random variable instead of the database itself still allows third-parties to construct consistent estimators of a wide class of population statistics. Second, the algorithm can satisfy the definition of differential privacy by basing the released kernel mean embedding on entirely synthetic data points, while controlling accuracy through the metric available in a Reproducing Kernel Hilbert Space. We describe two instantiations of the proposed framework, suitable under different scenarios, and prove theoretical results guaranteeing differential privacy of the resulting algorithms and the consistency of estimators constructed from their outputs.
Tasks
Published	2017-10-04
URL	http://arxiv.org/abs/1710.01641v2
PDF	http://arxiv.org/pdf/1710.01641v2.pdf
PWC	https://paperswithcode.com/paper/differentially-private-database-release-via
Repo	https://github.com/matejbalog/RKHS-private-database
Framework	pytorch

Graph learning under sparsity priors


Title	Graph learning under sparsity priors
Authors	Hermina Petric Maretic, Dorina Thanou, Pascal Frossard
Abstract	Graph signals offer a very generic and natural representation for data that lives on networks or irregular structures. The actual data structure is however often unknown a priori but can sometimes be estimated from the knowledge of the application domain. If this is not possible, the data structure has to be inferred from the mere signal observations. This is exactly the problem that we address in this paper, under the assumption that the graph signals can be represented as a sparse linear combination of a few atoms of a structured graph dictionary. The dictionary is constructed on polynomials of the graph Laplacian, which can sparsely represent a general class of graph signals composed of localized patterns on the graph. We formulate a graph learning problem, whose solution provides an ideal fit between the signal observations and the sparse graph signal model. As the problem is non-convex, we propose to solve it by alternating between a signal sparse coding and a graph update step. We provide experimental results that outline the good graph recovery performance of our method, which generally compares favourably to other recent network inference algorithms.
Tasks
Published	2017-07-18
URL	http://arxiv.org/abs/1707.05587v1
PDF	http://arxiv.org/pdf/1707.05587v1.pdf
PWC	https://paperswithcode.com/paper/graph-learning-under-sparsity-priors
Repo	https://github.com/Hermina/GraphLearningSparsityPriors
Framework	none

Unsupervised Machine Translation Using Monolingual Corpora Only


Title	Unsupervised Machine Translation Using Monolingual Corpora Only
Authors	Guillaume Lample, Alexis Conneau, Ludovic Denoyer, Marc’Aurelio Ranzato
Abstract	Machine translation has recently achieved impressive performance thanks to recent advances in deep learning and the availability of large-scale parallel corpora. There have been numerous attempts to extend these successes to low-resource language pairs, yet requiring tens of thousands of parallel sentences. In this work, we take this research direction to the extreme and investigate whether it is possible to learn to translate even without any parallel data. We propose a model that takes sentences from monolingual corpora in two different languages and maps them into the same latent space. By learning to reconstruct in both languages from this shared feature space, the model effectively learns to translate without using any labeled data. We demonstrate our model on two widely used datasets and two language pairs, reporting BLEU scores of 32.8 and 15.1 on the Multi30k and WMT English-French datasets, without using even a single parallel sentence at training time.
Tasks	Machine Translation, Unsupervised Machine Translation
Published	2017-10-31
URL	http://arxiv.org/abs/1711.00043v2
PDF	http://arxiv.org/pdf/1711.00043v2.pdf
PWC	https://paperswithcode.com/paper/unsupervised-machine-translation-using
Repo	https://github.com/facebookresearch/MUSE
Framework	pytorch

ObamaNet: Photo-realistic lip-sync from text


Title	ObamaNet: Photo-realistic lip-sync from text
Authors	Rithesh Kumar, Jose Sotelo, Kundan Kumar, Alexandre de Brebisson, Yoshua Bengio
Abstract	We present ObamaNet, the first architecture that generates both audio and synchronized photo-realistic lip-sync videos from any new text. Contrary to other published lip-sync approaches, ours is only composed of fully trainable neural modules and does not rely on any traditional computer graphics methods. More precisely, we use three main modules: a text-to-speech network based on Char2Wav, a time-delayed LSTM to generate mouth-keypoints synced to the audio, and a network based on Pix2Pix to generate the video frames conditioned on the keypoints.
Tasks
Published	2017-12-06
URL	http://arxiv.org/abs/1801.01442v1
PDF	http://arxiv.org/pdf/1801.01442v1.pdf
PWC	https://paperswithcode.com/paper/obamanet-photo-realistic-lip-sync-from-text
Repo	https://github.com/ung200/thats-what-obama-said
Framework	tf

Between-class Learning for Image Classification


Title	Between-class Learning for Image Classification
Authors	Yuji Tokozume, Yoshitaka Ushiku, Tatsuya Harada
Abstract	In this paper, we propose a novel learning method for image classification called Between-Class learning (BC learning). We generate between-class images by mixing two images belonging to different classes with a random ratio. We then input the mixed image to the model and train the model to output the mixing ratio. BC learning has the ability to impose constraints on the shape of the feature distributions, and thus the generalization ability is improved. BC learning is originally a method developed for sounds, which can be digitally mixed. Mixing two image data does not appear to make sense; however, we argue that because convolutional neural networks have an aspect of treating input data as waveforms, what works on sounds must also work on images. First, we propose a simple mixing method using internal divisions, which surprisingly proves to significantly improve performance. Second, we propose a mixing method that treats the images as waveforms, which leads to a further improvement in performance. As a result, we achieved 19.4% and 2.26% top-1 errors on ImageNet-1K and CIFAR-10, respectively.
Tasks	Image Classification
Published	2017-11-28
URL	http://arxiv.org/abs/1711.10284v2
PDF	http://arxiv.org/pdf/1711.10284v2.pdf
PWC	https://paperswithcode.com/paper/between-class-learning-for-image
Repo	https://github.com/mil-tokyo/bc_learning_image
Framework	torch

Application of a Convolutional Neural Network for image classification to the analysis of collisions in High Energy Physics


Title	Application of a Convolutional Neural Network for image classification to the analysis of collisions in High Energy Physics
Authors	Celia Fernández Madrazo, Ignacio Heredia Cacha, Lara Lloret Iglesias, Jesús Marco de Lucas
Abstract	The application of deep learning techniques using convolutional neural networks to the classification of particle collisions in High Energy Physics is explored. An intuitive approach to transform physical variables, like momenta of particles and jets, into a single image that captures the relevant information, is proposed. The idea is tested using a well known deep learning framework on a simulation dataset, including leptonic ttbar events and the corresponding background at 7 TeV from the CMS experiment at LHC, available as Open Data. This initial test shows competitive results when compared to more classical approaches, like those using feedforward neural networks.
Tasks	Image Classification
Published	2017-08-23
URL	http://arxiv.org/abs/1708.07034v1
PDF	http://arxiv.org/pdf/1708.07034v1.pdf
PWC	https://paperswithcode.com/paper/application-of-a-convolutional-neural-network
Repo	https://github.com/jzyee/cms_image_classification
Framework	none

Learning human behaviors from motion capture by adversarial imitation


Title	Learning human behaviors from motion capture by adversarial imitation
Authors	Josh Merel, Yuval Tassa, Dhruva TB, Sriram Srinivasan, Jay Lemmon, Ziyu Wang, Greg Wayne, Nicolas Heess
Abstract	Rapid progress in deep reinforcement learning has made it increasingly feasible to train controllers for high-dimensional humanoid bodies. However, methods that use pure reinforcement learning with simple reward functions tend to produce non-humanlike and overly stereotyped movement behaviors. In this work, we extend generative adversarial imitation learning to enable training of generic neural network policies to produce humanlike movement patterns from limited demonstrations consisting only of partially observed state features, without access to actions, even when the demonstrations come from a body with different and unknown physical parameters. We leverage this approach to build sub-skill policies from motion capture data and show that they can be reused to solve tasks when controlled by a higher level controller.
Tasks	Imitation Learning, Motion Capture
Published	2017-07-07
URL	http://arxiv.org/abs/1707.02201v2
PDF	http://arxiv.org/pdf/1707.02201v2.pdf
PWC	https://paperswithcode.com/paper/learning-human-behaviors-from-motion-capture
Repo	https://github.com/ywchao/merel-mocap-gail
Framework	none

Soft-DTW: a Differentiable Loss Function for Time-Series


Title	Soft-DTW: a Differentiable Loss Function for Time-Series
Authors	Marco Cuturi, Mathieu Blondel
Abstract	We propose in this paper a differentiable learning loss between time series, building upon the celebrated dynamic time warping (DTW) discrepancy. Unlike the Euclidean distance, DTW can compare time series of variable size and is robust to shifts or dilatations across the time dimension. To compute DTW, one typically solves a minimal-cost alignment problem between two time series using dynamic programming. Our work takes advantage of a smoothed formulation of DTW, called soft-DTW, that computes the soft-minimum of all alignment costs. We show in this paper that soft-DTW is a differentiable loss function, and that both its value and gradient can be computed with quadratic time/space complexity (DTW has quadratic time but linear space complexity). We show that this regularization is particularly well suited to average and cluster time series under the DTW geometry, a task for which our proposal significantly outperforms existing baselines. Next, we propose to tune the parameters of a machine that outputs time series by minimizing its fit with ground-truth labels in a soft-DTW sense.
Tasks	Time Series
Published	2017-03-05
URL	http://arxiv.org/abs/1703.01541v2
PDF	http://arxiv.org/pdf/1703.01541v2.pdf
PWC	https://paperswithcode.com/paper/soft-dtw-a-differentiable-loss-function-for
Repo	https://github.com/PeteWe/ts_similarity_tensorflow
Framework	tf

Decomposing Motion and Content for Natural Video Sequence Prediction


Title	Decomposing Motion and Content for Natural Video Sequence Prediction
Authors	Ruben Villegas, Jimei Yang, Seunghoon Hong, Xunyu Lin, Honglak Lee
Abstract	We propose a deep neural network for the prediction of future frames in natural video sequences. To effectively handle complex evolution of pixels in videos, we propose to decompose the motion and content, two key components generating dynamics in videos. Our model is built upon the Encoder-Decoder Convolutional Neural Network and Convolutional LSTM for pixel-level prediction, which independently capture the spatial layout of an image and the corresponding temporal dynamics. By independently modeling motion and content, predicting the next frame reduces to converting the extracted content features into the next frame content by the identified motion features, which simplifies the task of prediction. Our model is end-to-end trainable over multiple time steps, and naturally learns to decompose motion and content without separate training. We evaluate the proposed network architecture on human activity videos using KTH, Weizmann action, and UCF-101 datasets. We show state-of-the-art performance in comparison to recent approaches. To the best of our knowledge, this is the first end-to-end trainable network architecture with motion and content separation to model the spatiotemporal dynamics for pixel-level future prediction in natural videos.
Tasks	Future prediction
Published	2017-06-25
URL	http://arxiv.org/abs/1706.08033v2
PDF	http://arxiv.org/pdf/1706.08033v2.pdf
PWC	https://paperswithcode.com/paper/decomposing-motion-and-content-for-natural
Repo	https://github.com/rubenvillegas/iclr2017mcnet
Framework	tf

Towards Optimally Decentralized Multi-Robot Collision Avoidance via Deep Reinforcement Learning


Title	Towards Optimally Decentralized Multi-Robot Collision Avoidance via Deep Reinforcement Learning
Authors	Pinxin Long, Tingxiang Fan, Xinyi Liao, Wenxi Liu, Hao Zhang, Jia Pan
Abstract	Developing a safe and efficient collision avoidance policy for multiple robots is challenging in the decentralized scenarios where each robot generate its paths without observing other robots’ states and intents. While other distributed multi-robot collision avoidance systems exist, they often require extracting agent-level features to plan a local collision-free action, which can be computationally prohibitive and not robust. More importantly, in practice the performance of these methods are much lower than their centralized counterparts. We present a decentralized sensor-level collision avoidance policy for multi-robot systems, which directly maps raw sensor measurements to an agent’s steering commands in terms of movement velocity. As a first step toward reducing the performance gap between decentralized and centralized methods, we present a multi-scenario multi-stage training framework to find an optimal policy which is trained over a large number of robots on rich, complex environments simultaneously using a policy gradient based reinforcement learning algorithm. We validate the learned sensor-level collision avoidance policy in a variety of simulated scenarios with thorough performance evaluations and show that the final learned policy is able to find time efficient, collision-free paths for a large-scale robot system. We also demonstrate that the learned policy can be well generalized to new scenarios that do not appear in the entire training period, including navigating a heterogeneous group of robots and a large-scale scenario with 100 robots. Videos are available at https://sites.google.com/view/drlmaca
Tasks
Published	2017-09-28
URL	http://arxiv.org/abs/1709.10082v3
PDF	http://arxiv.org/pdf/1709.10082v3.pdf
PWC	https://paperswithcode.com/paper/towards-optimally-decentralized-multi-robot
Repo	https://github.com/mit-acl/gym-collision-avoidance
Framework	none

Visualizing Deep Neural Network Decisions: Prediction Difference Analysis


Title	Visualizing Deep Neural Network Decisions: Prediction Difference Analysis
Authors	Luisa M Zintgraf, Taco S Cohen, Tameem Adel, Max Welling
Abstract	This article presents the prediction difference analysis method for visualizing the response of a deep neural network to a specific input. When classifying images, the method highlights areas in a given input image that provide evidence for or against a certain class. It overcomes several shortcoming of previous methods and provides great additional insight into the decision making process of classifiers. Making neural network decisions interpretable through visualization is important both to improve models and to accelerate the adoption of black-box classifiers in application areas such as medicine. We illustrate the method in experiments on natural images (ImageNet data), as well as medical images (MRI brain scans).
Tasks	Decision Making
Published	2017-02-15
URL	http://arxiv.org/abs/1702.04595v1
PDF	http://arxiv.org/pdf/1702.04595v1.pdf
PWC	https://paperswithcode.com/paper/visualizing-deep-neural-network-decisions
Repo	https://github.com/lmzintgraf/DeepVis-PredDiff
Framework	none

Image2Mesh: A Learning Framework for Single Image 3D Reconstruction


Title	Image2Mesh: A Learning Framework for Single Image 3D Reconstruction
Authors	Jhony K. Pontes, Chen Kong, Sridha Sridharan, Simon Lucey, Anders Eriksson, Clinton Fookes
Abstract	One challenge that remains open in 3D deep learning is how to efficiently represent 3D data to feed deep networks. Recent works have relied on volumetric or point cloud representations, but such approaches suffer from a number of issues such as computational complexity, unordered data, and lack of finer geometry. This paper demonstrates that a mesh representation (i.e. vertices and faces to form polygonal surfaces) is able to capture fine-grained geometry for 3D reconstruction tasks. A mesh however is also unstructured data similar to point clouds. We address this problem by proposing a learning framework to infer the parameters of a compact mesh representation rather than learning from the mesh itself. This compact representation encodes a mesh using free-form deformation and a sparse linear combination of models allowing us to reconstruct 3D meshes from single images. In contrast to prior work, we do not rely on silhouettes and landmarks to perform 3D reconstruction. We evaluate our method on synthetic and real-world datasets with very promising results. Our framework efficiently reconstructs 3D objects in a low-dimensional way while preserving its important geometrical aspects.
Tasks	3D Reconstruction
Published	2017-11-29
URL	http://arxiv.org/abs/1711.10669v1
PDF	http://arxiv.org/pdf/1711.10669v1.pdf
PWC	https://paperswithcode.com/paper/image2mesh-a-learning-framework-for-single
Repo	https://github.com/jhonykaesemodel/image2mesh
Framework	none