January 31, 2020

3057 words 15 mins read

Paper Group ANR 159

Universal Approximation with Deep Narrow Networks. VGG Fine-tuning for Cooking State Recognition. Leveraging Crowdsourced GPS Data for Road Extraction from Aerial Imagery. Universal, transferable and targeted adversarial attacks. Analysis of Automatic Annotation Suggestions for Hard Discourse-Level Tasks in Expert Domains. EILearn: Learning Increme …

Universal Approximation with Deep Narrow Networks


Title	Universal Approximation with Deep Narrow Networks
Authors	Patrick Kidger, Terry Lyons
Abstract	The classical Universal Approximation Theorem certifies that the universal approximation property holds for the class of neural networks of arbitrary width. Here we consider the natural dual' theorem for width-bounded networks of arbitrary depth. Precisely, let $n$ be the number of inputs neurons, $m$ be the number of output neurons, and let $\rho$ be any nonaffine continuous function, with a continuous nonzero derivative at some point. Then we show that the class of neural networks of arbitrary depth, width $n + m + 2$, and activation function $\rho$, exhibits the universal approximation property with respect to the uniform norm on compact subsets of $\mathbb{R}^n$. This covers every activation function possible to use in practice; in particular this includes polynomial activation functions, making this genuinely different to the classical case. We go on to establish some natural extensions of this result. Firstly, we show an analogous result for a certain class of nowhere differentiable activation functions. Secondly, we establish an analogous result for noncompact domains, by showing that deep narrow networks with the ReLU activation function exhibit the universal approximation property with respect to the $p$-norm on $\mathbb{R}^n$. Finally, we show that width of only $n + m + 1$ suffices for most’ activation functions (whilst it is known that width of $n + m - 1$ does not suffice in general).
Tasks
Published	2019-05-21
URL	https://arxiv.org/abs/1905.08539v1
PDF	https://arxiv.org/pdf/1905.08539v1.pdf
PWC	https://paperswithcode.com/paper/universal-approximation-with-deep-narrow
Repo
Framework

VGG Fine-tuning for Cooking State Recognition


Title	VGG Fine-tuning for Cooking State Recognition
Authors	Juan Wilches
Abstract	An important task that domestic robots need to achieve is the recognition of states of food ingredients so they can continue their cooking actions. This project focuses on a fine-tuning algorithm for the VGG (Visual Geometry Group) architecture of deep convolutional neural networks (CNN) for object recognition. The algorithm aims to identify eleven different ingredient cooking states for an image dataset. The original VGG model was adjusted and trained to properly classify the food states. The model was initialized with Imagenet weights. Different experiments were carried out in order to find the model parameters that provided the best performance. The accuracy achieved for the validation set was 76.7% and for the test set 76.6% after changing several parameters of the VGG model.
Tasks	Object Recognition
Published	2019-05-13
URL	https://arxiv.org/abs/1905.08606v1
PDF	https://arxiv.org/pdf/1905.08606v1.pdf
PWC	https://paperswithcode.com/paper/190508606
Repo
Framework

Leveraging Crowdsourced GPS Data for Road Extraction from Aerial Imagery


Title	Leveraging Crowdsourced GPS Data for Road Extraction from Aerial Imagery
Authors	Tao Sun, Zonglin Di, Pengyu Che, Chun Liu, Yin Wang
Abstract	Deep learning is revolutionizing the mapping industry. Under lightweight human curation, computer has generated almost half of the roads in Thailand on OpenStreetMap (OSM) using high-resolution aerial imagery. Bing maps are displaying 125 million computer-generated building polygons in the U.S. While tremendously more efficient than manual mapping, one cannot map out everything from the air. Especially for roads, a small prediction gap by image occlusion renders the entire road useless for routing. Misconnections can be more dangerous. Therefore computer-based mapping often requires local verifications, which is still labor intensive. In this paper, we propose to leverage crowdsourced GPS data to improve and support road extraction from aerial imagery. Through novel data augmentation, GPS rendering, and 1D transpose convolution techniques, we show almost 5% improvements over previous competition winning models, and much better robustness when predicting new areas without any new training data or domain adaptation.
Tasks	Data Augmentation, Domain Adaptation
Published	2019-05-04
URL	https://arxiv.org/abs/1905.01447v1
PDF	https://arxiv.org/pdf/1905.01447v1.pdf
PWC	https://paperswithcode.com/paper/leveraging-crowdsourced-gps-data-for-road
Repo
Framework

Universal, transferable and targeted adversarial attacks


Title	Universal, transferable and targeted adversarial attacks
Authors	Junde Wu, Rao Fu
Abstract	Deep Neural Network has been found vulnerable recently. A kind of well-designed inputs, which called adversarial examples, can lead the networks to make incorrect predictions. Depending on the different scenarios, goals and capabilities, the difficulty to generate the attack is different. For example, generating a targeted attack is more difficult than a non-targeted attack, a universal attack is more difficult than a non-universal attack, a transferable attack is more difficult than a nontransferable one. The question is: Is there exist an attack that can survival in the most harsh adversity to meet all these requirements. Although many cheap and effective attacks have been proposed, this question is still not completely solved over large models and large scale dataset. In this paper, we learn a universal mapping from the sources to the adversarial examples. These examples can fool classification networks into classifying all of them to one targeted class. Besides, they are also transferable between different models.
Tasks
Published	2019-08-29
URL	https://arxiv.org/abs/1908.11332v2
PDF	https://arxiv.org/pdf/1908.11332v2.pdf
PWC	https://paperswithcode.com/paper/universal-transferable-and-targeted
Repo
Framework

Analysis of Automatic Annotation Suggestions for Hard Discourse-Level Tasks in Expert Domains


Title	Analysis of Automatic Annotation Suggestions for Hard Discourse-Level Tasks in Expert Domains
Authors	Claudia Schulz, Christian M. Meyer, Jan Kiesewetter, Michael Sailer, Elisabeth Bauer, Martin R. Fischer, Frank Fischer, Iryna Gurevych
Abstract	Many complex discourse-level tasks can aid domain experts in their work but require costly expert annotations for data creation. To speed up and ease annotations, we investigate the viability of automatically generated annotation suggestions for such tasks. As an example, we choose a task that is particularly hard for both humans and machines: the segmentation and classification of epistemic activities in diagnostic reasoning texts. We create and publish a new dataset covering two domains and carefully analyse the suggested annotations. We find that suggestions have positive effects on annotation speed and performance, while not introducing noteworthy biases. Envisioning suggestion models that improve with newly annotated texts, we contrast methods for continuous model adjustment and suggest the most effective setup for suggestions in future expert tasks.
Tasks
Published	2019-06-06
URL	https://arxiv.org/abs/1906.02564v1
PDF	https://arxiv.org/pdf/1906.02564v1.pdf
PWC	https://paperswithcode.com/paper/analysis-of-automatic-annotation-suggestions
Repo
Framework

EILearn: Learning Incrementally Using Previous Knowledge Obtained From an Ensemble of Classifiers


Title	EILearn: Learning Incrementally Using Previous Knowledge Obtained From an Ensemble of Classifiers
Authors	Shivang Agarwal, C. Ravindranath Chowdary, Shripriya Maheshwari
Abstract	We propose an algorithm for incremental learning of classifiers. The proposed method enables an ensemble of classifiers to learn incrementally by accommodating new training data. We use an effective mechanism to overcome the stability-plasticity dilemma. In incremental learning, the general convention is to use only the knowledge acquired in the previous phase but not the previously seen data. We follow this convention by retaining the previously acquired knowledge which is relevant and using it along with the current data. The performance of each classifier is monitored to eliminate the poorly performing classifiers in the subsequent phases. Experimental results show that the proposed approach outperforms the existing incremental learning approaches.
Tasks
Published	2019-02-08
URL	http://arxiv.org/abs/1902.02948v1
PDF	http://arxiv.org/pdf/1902.02948v1.pdf
PWC	https://paperswithcode.com/paper/eilearn-learning-incrementally-using-previous
Repo
Framework

SentiLR: Linguistic Knowledge Enhanced Language Representation for Sentiment Analysis


Title	SentiLR: Linguistic Knowledge Enhanced Language Representation for Sentiment Analysis
Authors	Pei Ke, Haozhe Ji, Siyang Liu, Xiaoyan Zhu, Minlie Huang
Abstract	Most of the existing pre-trained language representation models neglect to consider the linguistic knowledge of texts, whereas we argue that such knowledge can promote language understanding in various NLP tasks. In this paper, we propose a novel language representation model called SentiLR, which introduces word-level linguistic knowledge including part-of-speech tag and prior sentiment polarity from SentiWordNet to benefit the downstream tasks in sentiment analysis. During pre-training, we first acquire the prior sentiment polarity of each word by querying the SentiWordNet dictionary with its part-of-speech tag. Then, we devise a new pre-training task called label-aware masked language model (LA-MLM) consisting of two subtasks: 1) word knowledge recovering given the sentence-level label; 2) sentence-level label prediction with linguistic knowledge enhanced context. Experiments show that SentiLR achieves state-of-the-art performance on several sentence-level / aspect-level sentiment analysis tasks by fine-tuning, and also obtain comparative results on general language understanding tasks.
Tasks	Language Modelling, Sentiment Analysis
Published	2019-11-06
URL	https://arxiv.org/abs/1911.02493v1
PDF	https://arxiv.org/pdf/1911.02493v1.pdf
PWC	https://paperswithcode.com/paper/sentilr-linguistic-knowledge-enhanced
Repo
Framework


Title	Toward the Evaluation of Written Proficiency on a Collaborative Social Network for Learning Languages: Yask
Authors	Fabio N. Silva, Sergio Jimenez, George Dueñas
Abstract	Yask is an online social collaborative network for practicing languages in a framework that includes requests, answers, and votes. Since measuring linguistic competence using current approaches is difficult, expensive and in many cases imprecise, we present a new alternative approach based on social networks. Our method, called Proficiency Rank, extends the well-known Page Rank algorithm to measure the reputation of users in a collaborative social graph. First, we extended Page Rank so that it not only considers positive links (votes) but also negative links. Second, in addition to using explicit links, we also incorporate other 4 types of signals implicit in the social graph. These extensions allow Proficiency Rank to produce proficiency rankings for almost all users in the data set used, where only a minority contributes by answering, while the majority contributes only by voting. This overcomes the intrinsic limitation of Page Rank of only being able to rank the nodes that have incoming links. Our experimental validation showed that the reputation/importance of the users in Yask is significantly correlated with their language proficiency. In contrast, their written production was poorly correlated with the vocabulary profiles of the Common European Framework of Reference. In addition, we found that negative signals (votes) are considerably more informative than positive ones. We concluded that the use of this technology is a promising tool for measuring second language proficiency, even for relatively small groups of people.
Tasks
Published	2019-03-23
URL	http://arxiv.org/abs/1903.09846v1
PDF	http://arxiv.org/pdf/1903.09846v1.pdf
PWC	https://paperswithcode.com/paper/toward-the-evaluation-of-written-proficiency
Repo
Framework

Very high resolution Airborne PolSAR Image Classification using Convolutional Neural Networks


Title	Very high resolution Airborne PolSAR Image Classification using Convolutional Neural Networks
Authors	Minh-Tan Pham, Sébastien Lefèvre
Abstract	In this work, we exploit convolutional neural networks (CNNs) for the classification of very high resolution (VHR) polarimetric SAR (PolSAR) data. Due to the significant appearance of heterogeneous textures within these data, not only polarimetric features but also structural tensors are exploited to feed CNN models. For deep networks, we use the SegNet model for semantic segmentation, which corresponds to pixelwise classification in remote sensing. Our experiments on the airborne F-SAR data show that for VHR PolSAR images, SegNet could provide high accuracy for the classification task; and introducing structural tensors with polarimetric features as inputs could help the network to focus more on geometrical information to significantly improve the classification performance.
Tasks	Image Classification, Semantic Segmentation
Published	2019-10-31
URL	https://arxiv.org/abs/1910.14578v1
PDF	https://arxiv.org/pdf/1910.14578v1.pdf
PWC	https://paperswithcode.com/paper/very-high-resolution-airborne-polsar-image
Repo
Framework

A Novel Topology Optimization Approach using Conditional Deep Learning


Title	A Novel Topology Optimization Approach using Conditional Deep Learning
Authors	Sharad Rawat, M. -H. Herman Shen
Abstract	In this study, a novel topology optimization approach based on conditional Wasserstein generative adversarial networks (CWGAN) is developed to replicate the conventional topology optimization algorithms in an extremely computationally inexpensive way. CWGAN consists of a generator and a discriminator, both of which are deep convolutional neural networks (CNN). The limited samples of data, quasi-optimal planar structures, needed for training purposes are generated using the conventional topology optimization algorithms. With CWGANs, the topology optimization conditions can be set to a required value before generating samples. CWGAN truncates the global design space by introducing an equality constraint by the designer. The results are validated by generating an optimized planar structure using the conventional algorithms with the same settings. A proof of concept is presented which is known to be the first such illustration of fusion of CWGANs and topology optimization.
Tasks
Published	2019-01-14
URL	http://arxiv.org/abs/1901.04859v1
PDF	http://arxiv.org/pdf/1901.04859v1.pdf
PWC	https://paperswithcode.com/paper/a-novel-topology-optimization-approach-using
Repo
Framework

Word Embedding Algorithms as Generalized Low Rank Models and their Canonical Form


Title	Word Embedding Algorithms as Generalized Low Rank Models and their Canonical Form
Authors	Kian Kenyon-Dean
Abstract	Word embedding algorithms produce very reliable feature representations of words that are used by neural network models across a constantly growing multitude of NLP tasks. As such, it is imperative for NLP practitioners to understand how their word representations are produced, and why they are so impactful. The present work presents the Simple Embedder framework, generalizing the state-of-the-art existing word embedding algorithms (including Word2vec (SGNS) and GloVe) under the umbrella of generalized low rank models. We derive that both of these algorithms attempt to produce embedding inner products that approximate pointwise mutual information (PMI) statistics in the corpus. Once cast as Simple Embedders, comparison of these models reveals that these successful embedders all resemble a straightforward maximum likelihood estimate (MLE) of the PMI parametrized by the inner product (between embeddings). This MLE induces our proposed novel word embedding model, Hilbert-MLE, as the canonical representative of the Simple Embedder framework. We empirically compare these algorithms with evaluations on 17 different datasets. Hilbert-MLE consistently observes second-best performance on every extrinsic evaluation (news classification, sentiment analysis, POS-tagging, and supersense tagging), while the first-best model depends varying on the task. Moreover, Hilbert-MLE consistently observes the least variance in results with respect to the random initialization of the weights in bidirectional LSTMs. Our empirical results demonstrate that Hilbert-MLE is a very consistent word embedding algorithm that can be reliably integrated into existing NLP systems to obtain high-quality results.
Tasks	Sentiment Analysis
Published	2019-11-06
URL	https://arxiv.org/abs/1911.02639v1
PDF	https://arxiv.org/pdf/1911.02639v1.pdf
PWC	https://paperswithcode.com/paper/word-embedding-algorithms-as-generalized-low
Repo
Framework

Coverage Guided Testing for Recurrent Neural Networks


Title	Coverage Guided Testing for Recurrent Neural Networks
Authors	Wei Huang, Youcheng Sun, James Sharp, Wenjie Ruan, Jie Meng, Xiaowei Huang
Abstract	Recurrent neural networks (RNNs) have been applied to a broad range of applications such as natural language processing, drug discovery, and video recognition. This paper develops a coverage-guided testing approach for a major class of RNNs – long short-term memory networks (LSTMs). We start from defining a family of three test metrics that are designed to quantify not only the values but also the temporal relations (including both step-wise and bounded-length) learned through LSTM’s internal structures. While testing, random mutation enhanced with the coverage knowledge, i.e., targeted mutation, is designed to generate test cases. Based on these, we develop the coverage-guided testing tool testRNN. To our knowledge, this is the first time structural coverage metrics are used to test LSTMs. We extensively evaluate testRNN with a variety of LSTM benchmarks. Experiments confirm that there is a positive correlation between adversary rate and coverage rate, evidence showing that the test metrics are valid indicators of robustness evaluation. Also, we show that testRNN effectively captures erroneous behaviours in RNNs. Furthermore, meaningful information can be collected from testRNN for users to understand what the testing results represent. This is in contrast to most neural network testing works, and we believe testRNN is an important step towards interpretable neural network testing.
Tasks	Drug Discovery, Image Classification, Sentiment Analysis, Video Recognition
Published	2019-11-05
URL	https://arxiv.org/abs/1911.01952v2
PDF	https://arxiv.org/pdf/1911.01952v2.pdf
PWC	https://paperswithcode.com/paper/test-metrics-for-recurrent-neural-networks
Repo
Framework

Learning Densities in Feature Space for Reliable Segmentation of Indoor Scenes


Title	Learning Densities in Feature Space for Reliable Segmentation of Indoor Scenes
Authors	Nicolas Marchal, Charlotte Moraldo, Roland Siegwart, Hermann Blum, Cesar Cadena, Abel Gawel
Abstract	Deep learning has enabled remarkable advances in scene understanding, particularly in semantic segmentation tasks. Yet, current state of the art approaches are limited to a closed set of classes, and fail when facing novel elements, also known as out of distribution (OoD) data. This is a problem as autonomous agents will inevitably come across a wide range of objects, all of which cannot be included during training. We propose a novel method to distinguish any object (foreground) from empty building structure (background) in indoor environments. We use normalizing flow to estimate the probability distribution of high-dimensional background descriptors. Foreground objects are therefore detected as areas in an image for which the descriptors are unlikely given the background distribution. As our method does not explicitly learn the representation of individual objects, its performance generalizes well outside of the training examples. Our model results in an innovative solution to reliably segment foreground from background in indoor scenes, which opens the way to a safer deployment of robots in human environments.
Tasks	Scene Understanding, Semantic Segmentation
Published	2019-08-01
URL	https://arxiv.org/abs/1908.00448v4
PDF	https://arxiv.org/pdf/1908.00448v4.pdf
PWC	https://paperswithcode.com/paper/learning-densities-in-feature-space-for
Repo
Framework

Learning from Explanations with Neural Execution Tree


Title	Learning from Explanations with Neural Execution Tree
Authors	Ziqi Wang, Yujia Qin, Wenxuan Zhou, Jun Yan, Qinyuan Ye, Leonardo Neves, Zhiyuan Liu, Xiang Ren
Abstract	While deep neural networks have achieved impressive performance on a range of NLP tasks, these data-hungry models heavily rely on labeled data, which restricts their applications in scenarios where data annotation is expensive. Natural language (NL) explanations have been demonstrated very useful additional supervision, which can provide sufficient domain knowledge for generating more labeled data over new instances, while the annotation time only doubles. However, directly applying them for augmenting model learning encounters two challenges: (1) NL explanations are unstructured and inherently compositional, which asks for a modularized model to represent their semantics, (2) NL explanations often have large numbers of linguistic variants, resulting in low recall and limited generalization ability. In this paper, we propose a novel Neural Execution Tree (NExT) framework to augment training data for text classification using NL explanations. After transforming NL explanations into executable logical forms by semantic parsing, NExT generalizes different types of actions specified by the logical forms for labeling data instances, which substantially increases the coverage of each NL explanation. Experiments on two NLP tasks (relation extraction and sentiment analysis) demonstrate its superiority over baseline methods. Its extension to multi-hop question answering achieves performance gain with light annotation effort.
Tasks	Data Augmentation, Question Answering, Relation Extraction, Semantic Parsing, Sentiment Analysis, Text Classification
Published	2019-11-04
URL	https://arxiv.org/abs/1911.01352v3
PDF	https://arxiv.org/pdf/1911.01352v3.pdf
PWC	https://paperswithcode.com/paper/learning-to-annotate-modularizing-data
Repo
Framework

Modern Deep Reinforcement Learning Algorithms


Title	Modern Deep Reinforcement Learning Algorithms
Authors	Sergey Ivanov, Alexander D’yakonov
Abstract	Recent advances in Reinforcement Learning, grounded on combining classical theoretical results with Deep Learning paradigm, led to breakthroughs in many artificial intelligence tasks and gave birth to Deep Reinforcement Learning (DRL) as a field of research. In this work latest DRL algorithms are reviewed with a focus on their theoretical justification, practical limitations and observed empirical properties.
Tasks
Published	2019-06-24
URL	https://arxiv.org/abs/1906.10025v2
PDF	https://arxiv.org/pdf/1906.10025v2.pdf
PWC	https://paperswithcode.com/paper/modern-deep-reinforcement-learning-algorithms
Repo
Framework