January 27, 2020

3083 words 15 mins read

Paper Group ANR 1336

Transforming Wikipedia into Augmented Data for Query-Focused Summarization. Learning Driving Decisions by Imitating Drivers’ Control Behaviors. Secrets of the Brain: An Introduction to the Brain Anatomical Structure and Biological Function. A Cost Effective Solution for Road Crack Inspection using Cameras and Deep Neural Networks. Model Based Plann …

Transforming Wikipedia into Augmented Data for Query-Focused Summarization


Title	Transforming Wikipedia into Augmented Data for Query-Focused Summarization
Authors	Haichao Zhu, Li Dong, Furu Wei, Bing Qin, Ting Liu
Abstract	The manual construction of a query-focused summarization corpus is costly and timeconsuming. The limited size of existing datasets renders training data-driven summarization models challenging. In this paper, we use Wikipedia to automatically collect a large query-focused summarization dataset (named as WIKIREF) of more than 280,000 examples, which can serve as a means of data augmentation. Moreover, we develop a query-focused summarization model based on BERT to extract summaries from the documents. Experimental results on three DUC benchmarks show that the model pre-trained on WIKIREF has already achieved reasonable performance. After fine-tuning on the specific datasets, the model with data augmentation outperforms the state of the art on the benchmarks.
Tasks	Data Augmentation
Published	2019-11-08
URL	https://arxiv.org/abs/1911.03324v1
PDF	https://arxiv.org/pdf/1911.03324v1.pdf
PWC	https://paperswithcode.com/paper/transforming-wikipedia-into-augmented-data
Repo
Framework

Learning Driving Decisions by Imitating Drivers’ Control Behaviors


Title	Learning Driving Decisions by Imitating Drivers’ Control Behaviors
Authors	Junning Huang, Sirui Xie, Jiankai Sun, Qiurui Ma, Chunxiao Liu, Jianping Shi, Dahua Lin, Bolei Zhou
Abstract	Classical autonomous driving systems are modularized as a pipeline of perception, decision, planning, and control. The driving decision plays a central role in processing the observation from the perception as well as directing the execution of downstream planning and control modules. Commonly the decision module is designed to be rule-based and is difficult to learn from data. Recently end-to-end neural control policy has been proposed to replace this pipeline, given its generalization ability. However, it remains challenging to enforce physical or logical constraints on the decision to ensure driving safety and stability. In this work, we propose a hybrid framework for learning a decision module, which is agnostic to the mechanisms of perception, planning, and control modules. By imitating the low-level control behavior, it learns the high-level driving decisions while bypasses the ambiguous annotation of high-level driving decisions. We demonstrate that the simulation agents with a learned decision module can be generalized to various complex driving scenarios where the rule-based approach fails. Furthermore, it can generate driving behaviors that are smoother and safer than end-to-end neural policies.
Tasks	Autonomous Driving
Published	2019-11-30
URL	https://arxiv.org/abs/1912.00191v1
PDF	https://arxiv.org/pdf/1912.00191v1.pdf
PWC	https://paperswithcode.com/paper/learning-driving-decisions-by-imitating
Repo
Framework

Secrets of the Brain: An Introduction to the Brain Anatomical Structure and Biological Function


Title	Secrets of the Brain: An Introduction to the Brain Anatomical Structure and Biological Function
Authors	Jiawei Zhang
Abstract	In this paper, we will provide an introduction to the brain structure and function. Brain is an astonishing living organ inside our heads, weighing about 1.5kg, consisting of billions of tiny cells. The brain enables us to sense the world around us (to touch, to smell, to see and to hear, etc.), to think and to respond to the world as well. The main obstacles that prevent us from creating a machine which can behavior like real-world creatures are due to our limited knowledge about the brain in both its structure and its function. In this paper, we will focus introducing the brain anatomical structure and biological function, as well as its surrounding sensory systems. Many of the materials used in this paper are from wikipedia and several other neuroscience introductory articles, which will be properly cited in this article. This is the first of the three tutorial articles about the brain (the other two are [26] and [27]). In the follow-up two articles, we will further introduce the low-level composition basis structures (e.g., neuron, synapse and action potential) and the high-level cognitive functions (e.g., consciousness, attention, learning and memory) of the brain, respectively.
Tasks
Published	2019-05-31
URL	https://arxiv.org/abs/1906.03314v1
PDF	https://arxiv.org/pdf/1906.03314v1.pdf
PWC	https://paperswithcode.com/paper/secrets-of-the-brain-an-introduction-to-the
Repo
Framework

A Cost Effective Solution for Road Crack Inspection using Cameras and Deep Neural Networks


Title	A Cost Effective Solution for Road Crack Inspection using Cameras and Deep Neural Networks
Authors	Qipei Mei, Mustafa Gül
Abstract	Automatic crack detection on pavement surfaces is an important research field in the scope of developing an intelligent transportation infrastructure system. In this paper, a cost effective solution for road crack inspection by mounting commercial grade sport camera, GoPro, on the rear of the moving vehicle is introduced. Also, a novel method called ConnCrack combining conditional Wasserstein generative adversarial network and connectivity maps is proposed for road crack detection. In this method, a 121-layer densely connected neural network with deconvolution layers for multi-level feature fusion is used as generator, and a 5-layer fully convolutional network is used as discriminator. To overcome the scattered output issue related to deconvolution layers, connectivity maps are introduced to represent the crack information within the proposed ConnCrack. The proposed method is tested on a publicly available dataset as well our collected data. The results show that the proposed method achieves state-of-the-art performance compared with other existing methods in terms of precision, recall and F1 score.
Tasks
Published	2019-07-13
URL	https://arxiv.org/abs/1907.06014v2
PDF	https://arxiv.org/pdf/1907.06014v2.pdf
PWC	https://paperswithcode.com/paper/a-conditional-wasserstein-generative
Repo
Framework

Model Based Planning with Energy Based Models


Title	Model Based Planning with Energy Based Models
Authors	Yilun Du, Toru Lin, Igor Mordatch
Abstract	Model-based planning holds great promise for improving both sample efficiency and generalization in reinforcement learning (RL). We show that energy-based models (EBMs) are a promising class of models to use for model-based planning. EBMs naturally support inference of intermediate states given start and goal state distributions. We provide an online algorithm to train EBMs while interacting with the environment, and show that EBMs allow for significantly better online learning than corresponding feed-forward networks. We further show that EBMs support maximum entropy state inference and are able to generate diverse state space plans. We show that inference purely in state space - without planning actions - allows for better generalization to previously unseen obstacles in the environment and prevents the planner from exploiting the dynamics model by applying uncharacteristic action sequences. Finally, we show that online EBM training naturally leads to intentionally planned state exploration which performs significantly better than random exploration.
Tasks
Published	2019-09-15
URL	https://arxiv.org/abs/1909.06878v1
PDF	https://arxiv.org/pdf/1909.06878v1.pdf
PWC	https://paperswithcode.com/paper/model-based-planning-with-energy-based-models
Repo
Framework

Efficient Subsampled Gauss-Newton and Natural Gradient Methods for Training Neural Networks


Title	Efficient Subsampled Gauss-Newton and Natural Gradient Methods for Training Neural Networks
Authors	Yi Ren, Donald Goldfarb
Abstract	We present practical Levenberg-Marquardt variants of Gauss-Newton and natural gradient methods for solving non-convex optimization problems that arise in training deep neural networks involving enormous numbers of variables and huge data sets. Our methods use subsampled Gauss-Newton or Fisher information matrices and either subsampled gradient estimates (fully stochastic) or full gradients (semi-stochastic), which, in the latter case, we prove convergent to a stationary point. By using the Sherman-Morrison-Woodbury formula with automatic differentiation (backpropagation) we show how our methods can be implemented to perform efficiently. Finally, numerical results are presented to demonstrate the effectiveness of our proposed methods.
Tasks
Published	2019-06-05
URL	https://arxiv.org/abs/1906.02353v1
PDF	https://arxiv.org/pdf/1906.02353v1.pdf
PWC	https://paperswithcode.com/paper/efficient-subsampled-gauss-newton-and-natural
Repo
Framework

FickleNet: Weakly and Semi-supervised Semantic Image Segmentation using Stochastic Inference


Title	FickleNet: Weakly and Semi-supervised Semantic Image Segmentation using Stochastic Inference
Authors	Jungbeom Lee, Eunji Kim, Sungmin Lee, Jangho Lee, Sungroh Yoon
Abstract	The main obstacle to weakly supervised semantic image segmentation is the difficulty of obtaining pixel-level information from coarse image-level annotations. Most methods based on image-level annotations use localization maps obtained from the classifier, but these only focus on the small discriminative parts of objects and do not capture precise boundaries. FickleNet explores diverse combinations of locations on feature maps created by generic deep neural networks. It selects hidden units randomly and then uses them to obtain activation scores for image classification. FickleNet implicitly learns the coherence of each location in the feature maps, resulting in a localization map which identifies both discriminative and other parts of objects. The ensemble effects are obtained from a single network by selecting random hidden unit pairs, which means that a variety of localization maps are generated from a single image. Our approach does not require any additional training steps and only adds a simple layer to a standard convolutional neural network; nevertheless it outperforms recent comparable techniques on the Pascal VOC 2012 benchmark in both weakly and semi-supervised settings.
Tasks	Image Classification, Semantic Segmentation, Weakly-Supervised Semantic Segmentation
Published	2019-02-27
URL	http://arxiv.org/abs/1902.10421v2
PDF	http://arxiv.org/pdf/1902.10421v2.pdf
PWC	https://paperswithcode.com/paper/ficklenet-weakly-and-semi-supervised-semantic
Repo
Framework

Deep Evidential Regression


Title	Deep Evidential Regression
Authors	Alexander Amini, Wilko Schwarting, Ava Soleimany, Daniela Rus
Abstract	Deterministic neural networks (NNs) are increasingly being deployed in safety critical domains, where calibrated, robust and efficient measures of uncertainty are crucial. While it is possible to train regression networks to output the parameters of a probability distribution by maximizing a Gaussian likelihood function, the resulting model remains oblivious to the underlying confidence of its predictions. In this paper, we propose a novel method for training deterministic NNs to not only estimate the desired target but also the associated evidence in support of that target. We accomplish this by placing evidential priors over our original Gaussian likelihood function and training our NN to infer the hyperparameters of our evidential distribution. We impose priors during training such that the model is penalized when its predicted evidence is not aligned with the correct output. Thus the model estimates not only the probabilistic mean and variance of our target but also the underlying uncertainty associated with each of those parameters. We observe that our evidential regression method learns well-calibrated measures of uncertainty on various benchmarks, scales to complex computer vision tasks, and is robust to adversarial input perturbations.
Tasks
Published	2019-10-07
URL	https://arxiv.org/abs/1910.02600v1
PDF	https://arxiv.org/pdf/1910.02600v1.pdf
PWC	https://paperswithcode.com/paper/deep-evidential-regression
Repo
Framework

Influence-Based Multi-Agent Exploration


Title	Influence-Based Multi-Agent Exploration
Authors	Tonghan Wang, Jianhao Wang, Yi Wu, Chongjie Zhang
Abstract	Intrinsically motivated reinforcement learning aims to address the exploration challenge for sparse-reward tasks. However, the study of exploration methods in transition-dependent multi-agent settings is largely absent from the literature. We aim to take a step towards solving this problem. We present two exploration methods: exploration via information-theoretic influence (EITI) and exploration via decision-theoretic influence (EDTI), by exploiting the role of interaction in coordinated behaviors of agents. EITI uses mutual information to capture influence transition dynamics. EDTI uses a novel intrinsic reward, called Value of Interaction (VoI), to characterize and quantify the influence of one agent’s behavior on expected returns of other agents. By optimizing EITI or EDTI objective as a regularizer, agents are encouraged to coordinate their exploration and learn policies to optimize team performance. We show how to optimize these regularizers so that they can be easily integrated with policy gradient reinforcement learning. The resulting update rule draws a connection between coordinated exploration and intrinsic reward distribution. Finally, we empirically demonstrate the significant strength of our method in a variety of multi-agent scenarios.
Tasks
Published	2019-10-12
URL	https://arxiv.org/abs/1910.05512v1
PDF	https://arxiv.org/pdf/1910.05512v1.pdf
PWC	https://paperswithcode.com/paper/influence-based-multi-agent-exploration
Repo
Framework

Multi-Paragraph Reasoning with Knowledge-enhanced Graph Neural Network


Title	Multi-Paragraph Reasoning with Knowledge-enhanced Graph Neural Network
Authors	Deming Ye, Yankai Lin, Zhenghao Liu, Zhiyuan Liu, Maosong Sun
Abstract	Multi-paragraph reasoning is indispensable for open-domain question answering (OpenQA), which receives less attention in the current OpenQA systems. In this work, we propose a knowledge-enhanced graph neural network (KGNN), which performs reasoning over multiple paragraphs with entities. To explicitly capture the entities’ relatedness, KGNN utilizes relational facts in knowledge graph to build the entity graph. The experimental results show that KGNN outperforms in both distractor and full wiki settings than baselines methods on HotpotQA dataset. And our further analysis illustrates KGNN is effective and robust with more retrieved paragraphs.
Tasks	Open-Domain Question Answering, Question Answering
Published	2019-11-06
URL	https://arxiv.org/abs/1911.02170v1
PDF	https://arxiv.org/pdf/1911.02170v1.pdf
PWC	https://paperswithcode.com/paper/multi-paragraph-reasoning-with-knowledge
Repo
Framework

Deep Automodulators


Title	Deep Automodulators
Authors	Ari Heljakka, Yuxin Hou, Juho Kannala, Arno Solin
Abstract	We introduce a new family of generative neural network models called automodulators. These autoencoder-like networks can faithfully reproduce individual real-world input images like autoencoders, and also generate a fused sample from an arbitrary combination of several such images, allowing “style-mixing” and other new applications. An automodulator decouples the data flow of decoder operations from statistical properties thereof and uses the latent vector to modulate the former by the latter, with a principled approach for mutual disentanglement of decoder layers. This is the first general-purpose model to successfully apply this principle on existing input images, whereas prior work has focused on random sampling in GANs. We introduce novel techniques for stable unsupervised training of the model on four high-resolution data sets. Besides style-mixing, we show state-of-the-art results in autoencoder comparison, and visual image quality nearly indistinguishable from state-of-the-art GANs. We expect the automodulator variants to become a useful building block for image applications and other data domains.
Tasks
Published	2019-12-21
URL	https://arxiv.org/abs/1912.10321v2
PDF	https://arxiv.org/pdf/1912.10321v2.pdf
PWC	https://paperswithcode.com/paper/deep-automodulators-1
Repo
Framework

Towards Understanding the Importance of Noise in Training Neural Networks


Title	Towards Understanding the Importance of Noise in Training Neural Networks
Authors	Mo Zhou, Tianyi Liu, Yan Li, Dachao Lin, Enlu Zhou, Tuo Zhao
Abstract	Numerous empirical evidence has corroborated that the noise plays a crucial rule in effective and efficient training of neural networks. The theory behind, however, is still largely unknown. This paper studies this fundamental problem through training a simple two-layer convolutional neural network model. Although training such a network requires solving a nonconvex optimization problem with a spurious local optimum and a global optimum, we prove that perturbed gradient descent and perturbed mini-batch stochastic gradient algorithms in conjunction with noise annealing is guaranteed to converge to a global optimum in polynomial time with arbitrary initialization. This implies that the noise enables the algorithm to efficiently escape from the spurious local optimum. Numerical experiments are provided to support our theory.
Tasks
Published	2019-09-07
URL	https://arxiv.org/abs/1909.03172v1
PDF	https://arxiv.org/pdf/1909.03172v1.pdf
PWC	https://paperswithcode.com/paper/towards-understanding-the-importance-of-noise
Repo
Framework

Point2SpatialCapsule: Aggregating Features and Spatial Relationships of Local Regions on Point Clouds using Spatial-aware Capsules


Title	Point2SpatialCapsule: Aggregating Features and Spatial Relationships of Local Regions on Point Clouds using Spatial-aware Capsules
Authors	Xin Wen, Zhizhong Han, Xinhai Liu, Yu-Shen Liu
Abstract	Learning discriminative shape representation directly on point clouds is still challenging in 3D shape analysis and understanding. Recent studies usually involve three steps: first splitting a point cloud into some local regions, then extracting corresponding feature of each local region, and finally aggregating all individual local region features into a global feature as shape representation using simple max pooling. However, such pooling-based feature aggregation methods do not adequately take the spatial relationships between local regions into account, which greatly limits the ability to learn discriminative shape representation. To address this issue, we propose a novel deep learning network, named Point2SpatialCapsule, for aggregating features and spatial relationships of local regions on point clouds, which aims to learn more discriminative shape representation. Compared with traditional max-pooling based feature aggregation networks, Point2SpatialCapsule can explicitly learn not only geometric features of local regions but also spatial relationships among them. It consists of two modules. To resolve the disorder problem of local regions, the first module, named geometric feature aggregation, is designed to aggregate the local region features into the learnable cluster centers, which explicitly encodes the spatial locations from the original 3D space. The second module, named spatial relationship aggregation, is proposed for further aggregating clustered features and the spatial relationships among them in the feature space using the spatial-aware capsules developed in this paper. Compared to the previous capsule network based methods, the feature routing on the spatial-aware capsules can learn more discriminative spatial relationships among local regions for point clouds, which establishes a direct mapping between log priors and the spatial locations through feature clusters.
Tasks	3D Shape Analysis
Published	2019-08-29
URL	https://arxiv.org/abs/1908.11026v1
PDF	https://arxiv.org/pdf/1908.11026v1.pdf
PWC	https://paperswithcode.com/paper/point2spatialcapsule-aggregating-features-and
Repo
Framework

General Purpose (GenP) Bioimage Ensemble of Handcrafted and Learned Features with Data Augmentation


Title	General Purpose (GenP) Bioimage Ensemble of Handcrafted and Learned Features with Data Augmentation
Authors	L. Nanni, S. Brahnam, S. Ghidoni, G. Maguolo
Abstract	Bioimage classification plays a crucial role in many biological problems. Here we present a new General Purpose (GenP) ensemble that boosts performance by combining local features, dense sampling features, and deep learning approaches. We propose an ensemble of deep learning methods built using different criteria (different batch sizes, learning rates, topologies, and data augmentation methods). One of the contributions of this paper is the proposal of new methods of data augmentation based on feature transforms (principal component analysis/discrete cosine transform) that boost performance of Convolutional Neural Networks (CNNs). Each handcrafted descriptor is used to train a different Support Vector Machine (SVM), and the different SVMs are combined with the ensemble of CNNs. Our method is evaluated on a diverse set of bioimage classification problems. Results demonstrate that the proposed GenP bioimage ensemble obtains state-of-the-art performance without any ad-hoc dataset tuning of parameters (avoiding the risk of overfitting/overtraining).
Tasks	Data Augmentation
Published	2019-04-17
URL	http://arxiv.org/abs/1904.08084v1
PDF	http://arxiv.org/pdf/1904.08084v1.pdf
PWC	https://paperswithcode.com/paper/general-purpose-genp-bioimage-ensemble-of
Repo
Framework

Automated design of error-resilient and hardware-efficient deep neural networks


Title	Automated design of error-resilient and hardware-efficient deep neural networks
Authors	Christoph Schorn, Thomas Elsken, Sebastian Vogel, Armin Runge, Andre Guntoro, Gerd Ascheid
Abstract	Applying deep neural networks (DNNs) in mobile and safety-critical systems, such as autonomous vehicles, demands a reliable and efficient execution on hardware. Optimized dedicated hardware accelerators are being developed to achieve this. However, the design of efficient and reliable hardware has become increasingly difficult, due to the increased complexity of modern integrated circuit technology and its sensitivity against hardware faults, such as random bit-flips. It is thus desirable to exploit optimization potential for error resilience and efficiency also at the algorithmic side, e.g., by optimizing the architecture of the DNN. Since there are numerous design choices for the architecture of DNNs, with partially opposing effects on the preferred characteristics (such as small error rates at low latency), multi-objective optimization strategies are necessary. In this paper, we develop an evolutionary optimization technique for the automated design of hardware-optimized DNN architectures. For this purpose, we derive a set of easily computable objective functions, which enable the fast evaluation of DNN architectures with respect to their hardware efficiency and error resilience solely based on the network topology. We observe a strong correlation between predicted error resilience and actual measurements obtained from fault injection simulations. Furthermore, we analyze two different quantization schemes for efficient DNN computation and find significant differences regarding their effect on error resilience.
Tasks	Autonomous Vehicles, Quantization
Published	2019-09-30
URL	https://arxiv.org/abs/1909.13844v1
PDF	https://arxiv.org/pdf/1909.13844v1.pdf
PWC	https://paperswithcode.com/paper/automated-design-of-error-resilient-and
Repo
Framework