July 29, 2019

2931 words 14 mins read

Paper Group AWR 174

Numerical Gaussian Processes for Time-dependent and Non-linear Partial Differential Equations. Dynamic Neural Program Embedding for Program Repair. Effective Neural Solution for Multi-Criteria Word Segmentation. Geometric Affordances from a Single Example via the Interaction Tensor. Searching for Activation Functions. FigureQA: An Annotated Figure …

Numerical Gaussian Processes for Time-dependent and Non-linear Partial Differential Equations


Title	Numerical Gaussian Processes for Time-dependent and Non-linear Partial Differential Equations
Authors	Maziar Raissi, Paris Perdikaris, George Em Karniadakis
Abstract	We introduce the concept of numerical Gaussian processes, which we define as Gaussian processes with covariance functions resulting from temporal discretization of time-dependent partial differential equations. Numerical Gaussian processes, by construction, are designed to deal with cases where: (1) all we observe are noisy data on black-box initial conditions, and (2) we are interested in quantifying the uncertainty associated with such noisy data in our solutions to time-dependent partial differential equations. Our method circumvents the need for spatial discretization of the differential operators by proper placement of Gaussian process priors. This is an attempt to construct structured and data-efficient learning machines, which are explicitly informed by the underlying physics that possibly generated the observed data. The effectiveness of the proposed approach is demonstrated through several benchmark problems involving linear and nonlinear time-dependent operators. In all examples, we are able to recover accurate approximations of the latent solutions, and consistently propagate uncertainty, even in cases involving very long time integration.
Tasks	Gaussian Processes
Published	2017-03-29
URL	http://arxiv.org/abs/1703.10230v1
PDF	http://arxiv.org/pdf/1703.10230v1.pdf
PWC	https://paperswithcode.com/paper/numerical-gaussian-processes-for-time
Repo	https://github.com/maziarraissi/NumericalGP
Framework	none

Dynamic Neural Program Embedding for Program Repair


Title	Dynamic Neural Program Embedding for Program Repair
Authors	Ke Wang, Rishabh Singh, Zhendong Su
Abstract	Neural program embeddings have shown much promise recently for a variety of program analysis tasks, including program synthesis, program repair, fault localization, etc. However, most existing program embeddings are based on syntactic features of programs, such as raw token sequences or abstract syntax trees. Unlike images and text, a program has an unambiguous semantic meaning that can be difficult to capture by only considering its syntax (i.e. syntactically similar pro- grams can exhibit vastly different run-time behavior), which makes syntax-based program embeddings fundamentally limited. This paper proposes a novel semantic program embedding that is learned from program execution traces. Our key insight is that program states expressed as sequential tuples of live variable values not only captures program semantics more precisely, but also offer a more natural fit for Recurrent Neural Networks to model. We evaluate different syntactic and semantic program embeddings on predicting the types of errors that students make in their submissions to an introductory programming class and two exercises on the CodeHunt education platform. Evaluation results show that our new semantic program embedding significantly outperforms the syntactic program embeddings based on token sequences and abstract syntax trees. In addition, we augment a search-based program repair system with the predictions obtained from our se- mantic embedding, and show that search efficiency is also significantly improved.
Tasks	Program Synthesis
Published	2017-11-20
URL	http://arxiv.org/abs/1711.07163v4
PDF	http://arxiv.org/pdf/1711.07163v4.pdf
PWC	https://paperswithcode.com/paper/dynamic-neural-program-embedding-for-program
Repo	https://github.com/keowang/dynamic-program-embedding
Framework	tf

Effective Neural Solution for Multi-Criteria Word Segmentation


Title	Effective Neural Solution for Multi-Criteria Word Segmentation
Authors	Han He, Lei Wu, Hua Yan, Zhimin Gao, Yi Feng, George Townsend
Abstract	We present a simple yet elegant solution to train a single joint model on multi-criteria corpora for Chinese Word Segmentation (CWS). Our novel design requires no private layers in model architecture, instead, introduces two artificial tokens at the beginning and ending of input sentence to specify the required target criteria. The rest of the model including Long Short-Term Memory (LSTM) layer and Conditional Random Fields (CRFs) layer remains unchanged and is shared across all datasets, keeping the size of parameter collection minimal and constant. On Bakeoff 2005 and Bakeoff 2008 datasets, our innovative design has surpassed both single-criterion and multi-criteria state-of-the-art learning results. To the best knowledge, our design is the first one that has achieved the latest high performance on such large scale datasets. Source codes and corpora of this paper are available on GitHub.
Tasks	Chinese Word Segmentation
Published	2017-12-07
URL	http://arxiv.org/abs/1712.02856v2
PDF	http://arxiv.org/pdf/1712.02856v2.pdf
PWC	https://paperswithcode.com/paper/effective-neural-solution-for-multi-criteria
Repo	https://github.com/hankcs/multi-criteria-cws
Framework	none

Geometric Affordances from a Single Example via the Interaction Tensor


Title	Geometric Affordances from a Single Example via the Interaction Tensor
Authors	Eduardo Ruiz, Walterio Mayol-Cuevas
Abstract	This paper develops and evaluates a new tensor field representation to express the geometric affordance of one object over another. We expand the well known bisector surface representation to one that is weight-driven and that retains the provenance of surface points with directional vectors. We also incorporate the notion of affordance keypoints which allow for faster decisions at a point of query and with a compact and straightforward descriptor. Using a single interaction example, we are able to generalize to previously-unseen scenarios; both synthetic and also real scenes captured with RGBD sensors. We show how our interaction tensor allows for significantly better performance over alternative formulations. Evaluations also include crowdsourcing comparisons that confirm the validity of our affordance proposals, which agree on average 84% of the time with human judgments, and which is 20-40% better than the baseline methods.
Tasks
Published	2017-03-30
URL	http://arxiv.org/abs/1703.10584v1
PDF	http://arxiv.org/pdf/1703.10584v1.pdf
PWC	https://paperswithcode.com/paper/geometric-affordances-from-a-single-example
Repo	https://github.com/eduard626/interaction-tensor
Framework	none

Searching for Activation Functions


Title	Searching for Activation Functions
Authors	Prajit Ramachandran, Barret Zoph, Quoc V. Le
Abstract	The choice of activation functions in deep networks has a significant effect on the training dynamics and task performance. Currently, the most successful and widely-used activation function is the Rectified Linear Unit (ReLU). Although various hand-designed alternatives to ReLU have been proposed, none have managed to replace it due to inconsistent gains. In this work, we propose to leverage automatic search techniques to discover new activation functions. Using a combination of exhaustive and reinforcement learning-based search, we discover multiple novel activation functions. We verify the effectiveness of the searches by conducting an empirical evaluation with the best discovered activation function. Our experiments show that the best discovered activation function, $f(x) = x \cdot \text{sigmoid}(\beta x)$, which we name Swish, tends to work better than ReLU on deeper models across a number of challenging datasets. For example, simply replacing ReLUs with Swish units improves top-1 classification accuracy on ImageNet by 0.9% for Mobile NASNet-A and 0.6% for Inception-ResNet-v2. The simplicity of Swish and its similarity to ReLU make it easy for practitioners to replace ReLUs with Swish units in any neural network.
Tasks	Image Classification
Published	2017-10-16
URL	http://arxiv.org/abs/1710.05941v2
PDF	http://arxiv.org/pdf/1710.05941v2.pdf
PWC	https://paperswithcode.com/paper/searching-for-activation-functions
Repo	https://github.com/swordgeek/SR
Framework	pytorch

FigureQA: An Annotated Figure Dataset for Visual Reasoning


Title	FigureQA: An Annotated Figure Dataset for Visual Reasoning
Authors	Samira Ebrahimi Kahou, Vincent Michalski, Adam Atkinson, Akos Kadar, Adam Trischler, Yoshua Bengio
Abstract	We introduce FigureQA, a visual reasoning corpus of over one million question-answer pairs grounded in over 100,000 images. The images are synthetic, scientific-style figures from five classes: line plots, dot-line plots, vertical and horizontal bar graphs, and pie charts. We formulate our reasoning task by generating questions from 15 templates; questions concern various relationships between plot elements and examine characteristics like the maximum, the minimum, area-under-the-curve, smoothness, and intersection. To resolve, such questions often require reference to multiple plot elements and synthesis of information distributed spatially throughout a figure. To facilitate the training of machine learning systems, the corpus also includes side data that can be used to formulate auxiliary objectives. In particular, we provide the numerical data used to generate each figure as well as bounding-box annotations for all plot elements. We study the proposed visual reasoning task by training several models, including the recently proposed Relation Network as a strong baseline. Preliminary results indicate that the task poses a significant machine learning challenge. We envision FigureQA as a first step towards developing models that can intuitively recognize patterns from visual representations of data.
Tasks	Visual Reasoning
Published	2017-10-19
URL	http://arxiv.org/abs/1710.07300v2
PDF	http://arxiv.org/pdf/1710.07300v2.pdf
PWC	https://paperswithcode.com/paper/figureqa-an-annotated-figure-dataset-for
Repo	https://github.com/vmichals/FigureQA-baseline
Framework	tf

Learning to Reason: End-to-End Module Networks for Visual Question Answering


Title	Learning to Reason: End-to-End Module Networks for Visual Question Answering
Authors	Ronghang Hu, Jacob Andreas, Marcus Rohrbach, Trevor Darrell, Kate Saenko
Abstract	Natural language questions are inherently compositional, and many are most easily answered by reasoning about their decomposition into modular sub-problems. For example, to answer “is there an equal number of balls and boxes?” we can look for balls, look for boxes, count them, and compare the results. The recently proposed Neural Module Network (NMN) architecture implements this approach to question answering by parsing questions into linguistic substructures and assembling question-specific deep networks from smaller modules that each solve one subtask. However, existing NMN implementations rely on brittle off-the-shelf parsers, and are restricted to the module configurations proposed by these parsers rather than learning them from data. In this paper, we propose End-to-End Module Networks (N2NMNs), which learn to reason by directly predicting instance-specific network layouts without the aid of a parser. Our model learns to generate network structures (by imitating expert demonstrations) while simultaneously learning network parameters (using the downstream task loss). Experimental results on the new CLEVR dataset targeted at compositional question answering show that N2NMNs achieve an error reduction of nearly 50% relative to state-of-the-art attentional approaches, while discovering interpretable network architectures specialized for each question.
Tasks	Visual Question Answering
Published	2017-04-18
URL	http://arxiv.org/abs/1704.05526v3
PDF	http://arxiv.org/pdf/1704.05526v3.pdf
PWC	https://paperswithcode.com/paper/learning-to-reason-end-to-end-module-networks
Repo	https://github.com/ronghanghu/n2nmn
Framework	tf

A Fast Method For Computing Principal Curvatures From Range Images


Title	A Fast Method For Computing Principal Curvatures From Range Images
Authors	Andrew Spek, Wai Ho Li, Tom Drummond
Abstract	Estimation of surface curvature from range data is important for a range of tasks in computer vision and robotics, object segmentation, object recognition and robotic grasping estimation. This work presents a fast method of robustly computing accurate metric principal curvature values from noisy point clouds which was implemented on GPU. In contrast to existing readily available solutions which first differentiate the surface to estimate surface normals and then differentiate these to obtain curvature, amplifying noise, our method iteratively fits parabolic quadric surface patches to the data. Additionally previous methods with a similar formulation use less robust techniques less applicable to a high noise sensor. We demonstrate that our method is fast and provides better curvature estimates than existing techniques. In particular we compare our method to several alternatives to demonstrate the improvement.
Tasks	Object Recognition, Robotic Grasping, Semantic Segmentation
Published	2017-07-03
URL	http://arxiv.org/abs/1707.00385v2
PDF	http://arxiv.org/pdf/1707.00385v2.pdf
PWC	https://paperswithcode.com/paper/a-fast-method-for-computing-principal
Repo	https://github.com/aspek1/QuadricCurvature
Framework	none

Distributional Adversarial Networks


Title	Distributional Adversarial Networks
Authors	Chengtao Li, David Alvarez-Melis, Keyulu Xu, Stefanie Jegelka, Suvrit Sra
Abstract	We propose a framework for adversarial training that relies on a sample rather than a single sample point as the fundamental unit of discrimination. Inspired by discrepancy measures and two-sample tests between probability distributions, we propose two such distributional adversaries that operate and predict on samples, and show how they can be easily implemented on top of existing models. Various experimental results show that generators trained with our distributional adversaries are much more stable and are remarkably less prone to mode collapse than traditional models trained with pointwise prediction discriminators. The application of our framework to domain adaptation also results in considerable improvement over recent state-of-the-art.
Tasks	Domain Adaptation
Published	2017-06-29
URL	http://arxiv.org/abs/1706.09549v3
PDF	http://arxiv.org/pdf/1706.09549v3.pdf
PWC	https://paperswithcode.com/paper/distributional-adversarial-networks
Repo	https://github.com/ChengtaoLi/dan
Framework	tf

Deep Occlusion Reasoning for Multi-Camera Multi-Target Detection


Title	Deep Occlusion Reasoning for Multi-Camera Multi-Target Detection
Authors	Pierre Baqué, François Fleuret, Pascal Fua
Abstract	People detection in single 2D images has improved greatly in recent years. However, comparatively little of this progress has percolated into multi-camera multi-people tracking algorithms, whose performance still degrades severely when scenes become very crowded. In this work, we introduce a new architecture that combines Convolutional Neural Nets and Conditional Random Fields to explicitly model those ambiguities. One of its key ingredients are high-order CRF terms that model potential occlusions and give our approach its robustness even when many people are present. Our model is trained end-to-end and we show that it outperforms several state-of-art algorithms on challenging scenes.
Tasks
Published	2017-04-19
URL	http://arxiv.org/abs/1704.05775v2
PDF	http://arxiv.org/pdf/1704.05775v2.pdf
PWC	https://paperswithcode.com/paper/deep-occlusion-reasoning-for-multi-camera
Repo	https://github.com/rickyHong/DeepOcclustion-repl
Framework	none

Sentiment Analysis on Financial News Headlines using Training Dataset Augmentation


Title	Sentiment Analysis on Financial News Headlines using Training Dataset Augmentation
Authors	Vineet John, Olga Vechtomova
Abstract	This paper discusses the approach taken by the UWaterloo team to arrive at a solution for the Fine-Grained Sentiment Analysis problem posed by Task 5 of SemEval 2017. The paper describes the document vectorization and sentiment score prediction techniques used, as well as the design and implementation decisions taken while building the system for this task. The system uses text vectorization models, such as N-gram, TF-IDF and paragraph embeddings, coupled with regression model variants to predict the sentiment scores. Amongst the methods examined, unigrams and bigrams coupled with simple linear regression obtained the best baseline accuracy. The paper also explores data augmentation methods to supplement the training dataset. This system was designed for Subtask 2 (News Statements and Headlines).
Tasks	Data Augmentation, Sentiment Analysis
Published	2017-07-29
URL	http://arxiv.org/abs/1707.09448v1
PDF	http://arxiv.org/pdf/1707.09448v1.pdf
PWC	https://paperswithcode.com/paper/sentiment-analysis-on-financial-news
Repo	https://github.com/v1n337/semeval2017-task5
Framework	tf

Temporally Efficient Deep Learning with Spikes


Title	Temporally Efficient Deep Learning with Spikes
Authors	Peter O’Connor, Efstratios Gavves, Max Welling
Abstract	The vast majority of natural sensory data is temporally redundant. Video frames or audio samples which are sampled at nearby points in time tend to have similar values. Typically, deep learning algorithms take no advantage of this redundancy to reduce computation. This can be an obscene waste of energy. We present a variant on backpropagation for neural networks in which computation scales with the rate of change of the data - not the rate at which we process the data. We do this by having neurons communicate a combination of their state, and their temporal change in state. Intriguingly, this simple communication rule give rise to units that resemble biologically-inspired leaky integrate-and-fire neurons, and to a weight-update rule that is equivalent to a form of Spike-Timing Dependent Plasticity (STDP), a synaptic learning rule observed in the brain. We demonstrate that on MNIST and a temporal variant of MNIST, our algorithm performs about as well as a Multilayer Perceptron trained with backpropagation, despite only communicating discrete values between layers.
Tasks
Published	2017-06-13
URL	http://arxiv.org/abs/1706.04159v1
PDF	http://arxiv.org/pdf/1706.04159v1.pdf
PWC	https://paperswithcode.com/paper/temporally-efficient-deep-learning-with
Repo	https://github.com/petered/pdnn
Framework	none

User Modelling for Avoiding Overfitting in Interactive Knowledge Elicitation for Prediction


Title	User Modelling for Avoiding Overfitting in Interactive Knowledge Elicitation for Prediction
Authors	Pedram Daee, Tomi Peltola, Aki Vehtari, Samuel Kaski
Abstract	In human-in-the-loop machine learning, the user provides information beyond that in the training data. Many algorithms and user interfaces have been designed to optimize and facilitate this human–machine interaction; however, fewer studies have addressed the potential defects the designs can cause. Effective interaction often requires exposing the user to the training data or its statistics. The design of the system is then critical, as this can lead to double use of data and overfitting, if the user reinforces noisy patterns in the data. We propose a user modelling methodology, by assuming simple rational behaviour, to correct the problem. We show, in a user study with 48 participants, that the method improves predictive performance in a sparse linear regression sentiment analysis task, where graded user knowledge on feature relevance is elicited. We believe that the key idea of inferring user knowledge with probabilistic user models has general applicability in guarding against overfitting and improving interactive machine learning.
Tasks	Sentiment Analysis
Published	2017-10-13
URL	http://arxiv.org/abs/1710.04881v2
PDF	http://arxiv.org/pdf/1710.04881v2.pdf
PWC	https://paperswithcode.com/paper/user-modelling-for-avoiding-overfitting-in
Repo	https://github.com/HIIT/human-overfitting-in-IML
Framework	none

Synthetic and Natural Noise Both Break Neural Machine Translation


Title	Synthetic and Natural Noise Both Break Neural Machine Translation
Authors	Yonatan Belinkov, Yonatan Bisk
Abstract	Character-based neural machine translation (NMT) models alleviate out-of-vocabulary issues, learn morphology, and move us closer to completely end-to-end translation systems. Unfortunately, they are also very brittle and easily falter when presented with noisy data. In this paper, we confront NMT models with synthetic and natural sources of noise. We find that state-of-the-art models fail to translate even moderately noisy texts that humans have no trouble comprehending. We explore two approaches to increase model robustness: structure-invariant word representations and robust training on noisy texts. We find that a model based on a character convolutional neural network is able to simultaneously learn representations robust to multiple kinds of noise.
Tasks	Machine Translation
Published	2017-11-06
URL	http://arxiv.org/abs/1711.02173v2
PDF	http://arxiv.org/pdf/1711.02173v2.pdf
PWC	https://paperswithcode.com/paper/synthetic-and-natural-noise-both-break-neural
Repo	https://github.com/ybisk/charNMT-noise
Framework	none

DocEmul: a Toolkit to Generate Structured Historical Documents


Title	DocEmul: a Toolkit to Generate Structured Historical Documents
Authors	Samuele Capobianco, Simone Marinai
Abstract	We propose a toolkit to generate structured synthetic documents emulating the actual document production process. Synthetic documents can be used to train systems to perform document analysis tasks. In our case we address the record counting task on handwritten structured collections containing a limited number of examples. Using the DocEmul toolkit we can generate a larger dataset to train a deep architecture to predict the number of records for each page. The toolkit is able to generate synthetic collections and also perform data augmentation to create a larger trainable dataset. It includes one method to extract the page background from real pages which can be used as a substrate where records can be written on the basis of variable structures and using cursive fonts. Moreover, it is possible to extend the synthetic collection by adding random noise, page rotations, and other visual variations. We performed some experiments on two different handwritten collections using the toolkit to generate synthetic data to train a Convolutional Neural Network able to count the number of records in the real collections.
Tasks	Data Augmentation
Published	2017-10-10
URL	http://arxiv.org/abs/1710.03474v1
PDF	http://arxiv.org/pdf/1710.03474v1.pdf
PWC	https://paperswithcode.com/paper/docemul-a-toolkit-to-generate-structured
Repo	https://github.com/scstech85/DocEmul
Framework	none