July 29, 2019

2848 words 14 mins read

Paper Group AWR 128

An Online Algorithm for Nonparametric Correlations. False Positive and Cross-relation Signals in Distant Supervision Data. Adversarial Attacks on Neural Network Policies. Deep Quaternion Networks. Neural Task Programming: Learning to Generalize Across Hierarchical Tasks. 3D Semantic Segmentation with Submanifold Sparse Convolutional Networks. Refle …

An Online Algorithm for Nonparametric Correlations


Title	An Online Algorithm for Nonparametric Correlations
Authors	Wei Xiao
Abstract	Nonparametric correlations such as Spearman’s rank correlation and Kendall’s tau correlation are widely applied in scientific and engineering fields. This paper investigates the problem of computing nonparametric correlations on the fly for streaming data. Standard batch algorithms are generally too slow to handle real-world big data applications. They also require too much memory because all the data need to be stored in the memory before processing. This paper proposes a novel online algorithm for computing nonparametric correlations. The algorithm has O(1) time complexity and O(1) memory cost and is quite suitable for edge devices, where only limited memory and processing power are available. You can seek a balance between speed and accuracy by changing the number of cutpoints specified in the algorithm. The online algorithm can compute the nonparametric correlations 10 to 1,000 times faster than the corresponding batch algorithm, and it can compute them based either on all past observations or on fixed-size sliding windows.
Tasks
Published	2017-12-05
URL	http://arxiv.org/abs/1712.01521v1
PDF	http://arxiv.org/pdf/1712.01521v1.pdf
PWC	https://paperswithcode.com/paper/an-online-algorithm-for-nonparametric
Repo	https://github.com/wxiao0421/onlineNPCORR
Framework	none

False Positive and Cross-relation Signals in Distant Supervision Data


Title	False Positive and Cross-relation Signals in Distant Supervision Data
Authors	Anca Dumitrache, Lora Aroyo, Chris Welty
Abstract	Distant supervision (DS) is a well-established method for relation extraction from text, based on the assumption that when a knowledge-base contains a relation between a term pair, then sentences that contain that pair are likely to express the relation. In this paper, we use the results of a crowdsourcing relation extraction task to identify two problems with DS data quality: the widely varying degree of false positives across different relations, and the observed causal connection between relations that are not considered by the DS method. The crowdsourcing data aggregation is performed using ambiguity-aware CrowdTruth metrics, that are used to capture and interpret inter-annotator disagreement. We also present preliminary results of using the crowd to enhance DS training data for a relation classification model, without requiring the crowd to annotate the entire set.
Tasks	Relation Classification, Relation Extraction
Published	2017-11-14
URL	http://arxiv.org/abs/1711.05186v2
PDF	http://arxiv.org/pdf/1711.05186v2.pdf
PWC	https://paperswithcode.com/paper/false-positive-and-cross-relation-signals-in
Repo	https://github.com/CrowdTruth/Open-Domain-Relation-Extraction
Framework	none

Adversarial Attacks on Neural Network Policies


Title	Adversarial Attacks on Neural Network Policies
Authors	Sandy Huang, Nicolas Papernot, Ian Goodfellow, Yan Duan, Pieter Abbeel
Abstract	Machine learning classifiers are known to be vulnerable to inputs maliciously constructed by adversaries to force misclassification. Such adversarial examples have been extensively studied in the context of computer vision applications. In this work, we show adversarial attacks are also effective when targeting neural network policies in reinforcement learning. Specifically, we show existing adversarial example crafting techniques can be used to significantly degrade test-time performance of trained policies. Our threat model considers adversaries capable of introducing small perturbations to the raw input of the policy. We characterize the degree of vulnerability across tasks and training algorithms, for a subclass of adversarial-example attacks in white-box and black-box settings. Regardless of the learned task or training algorithm, we observe a significant drop in performance, even with small adversarial perturbations that do not interfere with human perception. Videos are available at http://rll.berkeley.edu/adversarial.
Tasks
Published	2017-02-08
URL	http://arxiv.org/abs/1702.02284v1
PDF	http://arxiv.org/pdf/1702.02284v1.pdf
PWC	https://paperswithcode.com/paper/adversarial-attacks-on-neural-network
Repo	https://github.com/BPDanek/learning_resources
Framework	none

Deep Quaternion Networks


Title	Deep Quaternion Networks
Authors	Chase Gaudet, Anthony Maida
Abstract	The field of deep learning has seen significant advancement in recent years. However, much of the existing work has been focused on real-valued numbers. Recent work has shown that a deep learning system using the complex numbers can be deeper for a fixed parameter budget compared to its real-valued counterpart. In this work, we explore the benefits of generalizing one step further into the hyper-complex numbers, quaternions specifically, and provide the architecture components needed to build deep quaternion networks. We develop the theoretical basis by reviewing quaternion convolutions, developing a novel quaternion weight initialization scheme, and developing novel algorithms for quaternion batch-normalization. These pieces are tested in a classification model by end-to-end training on the CIFAR-10 and CIFAR-100 data sets and a segmentation model by end-to-end training on the KITTI Road Segmentation data set. These quaternion networks show improved convergence compared to real-valued and complex-valued networks, especially on the segmentation task, while having fewer parameters
Tasks
Published	2017-12-13
URL	http://arxiv.org/abs/1712.04604v3
PDF	http://arxiv.org/pdf/1712.04604v3.pdf
PWC	https://paperswithcode.com/paper/deep-quaternion-networks
Repo	https://github.com/heheqianqian/DeepQuaternionNetworks
Framework	none

Neural Task Programming: Learning to Generalize Across Hierarchical Tasks


Title	Neural Task Programming: Learning to Generalize Across Hierarchical Tasks
Authors	Danfei Xu, Suraj Nair, Yuke Zhu, Julian Gao, Animesh Garg, Li Fei-Fei, Silvio Savarese
Abstract	In this work, we propose a novel robot learning framework called Neural Task Programming (NTP), which bridges the idea of few-shot learning from demonstration and neural program induction. NTP takes as input a task specification (e.g., video demonstration of a task) and recursively decomposes it into finer sub-task specifications. These specifications are fed to a hierarchical neural program, where bottom-level programs are callable subroutines that interact with the environment. We validate our method in three robot manipulation tasks. NTP achieves strong generalization across sequential tasks that exhibit hierarchal and compositional structures. The experimental results show that NTP learns to generalize well to- wards unseen tasks with increasing lengths, variable topologies, and changing objectives.
Tasks	Few-Shot Learning
Published	2017-10-04
URL	http://arxiv.org/abs/1710.01813v2
PDF	http://arxiv.org/pdf/1710.01813v2.pdf
PWC	https://paperswithcode.com/paper/neural-task-programming-learning-to
Repo	https://github.com/StanfordVL/arxivbot
Framework	none

3D Semantic Segmentation with Submanifold Sparse Convolutional Networks


Title	3D Semantic Segmentation with Submanifold Sparse Convolutional Networks
Authors	Benjamin Graham, Martin Engelcke, Laurens van der Maaten
Abstract	Submanifold sparse convolutional networks
Tasks	3D Semantic Segmentation, Semantic Segmentation
Published	2017-11-28
URL	http://arxiv.org/abs/1711.10275v1
PDF	http://arxiv.org/pdf/1711.10275v1.pdf
PWC	https://paperswithcode.com/paper/3d-semantic-segmentation-with-submanifold
Repo	https://github.com/edraizen/molmimic
Framework	pytorch

Reflectance Intensity Assisted Automatic and Accurate Extrinsic Calibration of 3D LiDAR and Panoramic Camera Using a Printed Chessboard


Title	Reflectance Intensity Assisted Automatic and Accurate Extrinsic Calibration of 3D LiDAR and Panoramic Camera Using a Printed Chessboard
Authors	Weimin Wang, Ken Sakurada, Nobuo Kawaguchi
Abstract	This paper presents a novel method for fully automatic and convenient extrinsic calibration of a 3D LiDAR and a panoramic camera with a normally printed chessboard. The proposed method is based on the 3D corner estimation of the chessboard from the sparse point cloud generated by one frame scan of the LiDAR. To estimate the corners, we formulate a full-scale model of the chessboard and fit it to the segmented 3D points of the chessboard. The model is fitted by optimizing the cost function under constraints of correlation between the reflectance intensity of laser and the color of the chessboard’s patterns. Powell’s method is introduced for resolving the discontinuity problem in optimization. The corners of the fitted model are considered as the 3D corners of the chessboard. Once the corners of the chessboard in the 3D point cloud are estimated, the extrinsic calibration of the two sensors is converted to a 3D-2D matching problem. The corresponding 3D-2D points are used to calculate the absolute pose of the two sensors with Unified Perspective-n-Point (UPnP). Further, the calculated parameters are regarded as initial values and are refined using the Levenberg-Marquardt method. The performance of the proposed corner detection method from the 3D point cloud is evaluated using simulations. The results of experiments, conducted on a Velodyne HDL-32e LiDAR and a Ladybug3 camera under the proposed re-projection error metric, qualitatively and quantitatively demonstrate the accuracy and stability of the final extrinsic calibration parameters.
Tasks	Calibration
Published	2017-08-18
URL	http://arxiv.org/abs/1708.05514v1
PDF	http://arxiv.org/pdf/1708.05514v1.pdf
PWC	https://paperswithcode.com/paper/reflectance-intensity-assisted-automatic-and
Repo	https://github.com/mfxox/ILCC
Framework	none

FSRNet: End-to-End Learning Face Super-Resolution with Facial Priors


Title	FSRNet: End-to-End Learning Face Super-Resolution with Facial Priors
Authors	Yu Chen, Ying Tai, Xiaoming Liu, Chunhua Shen, Jian Yang
Abstract	Face Super-Resolution (SR) is a domain-specific super-resolution problem. The specific facial prior knowledge could be leveraged for better super-resolving face images. We present a novel deep end-to-end trainable Face Super-Resolution Network (FSRNet), which makes full use of the geometry prior, i.e., facial landmark heatmaps and parsing maps, to super-resolve very low-resolution (LR) face images without well-aligned requirement. Specifically, we first construct a coarse SR network to recover a coarse high-resolution (HR) image. Then, the coarse HR image is sent to two branches: a fine SR encoder and a prior information estimation network, which extracts the image features, and estimates landmark heatmaps/parsing maps respectively. Both image features and prior information are sent to a fine SR decoder to recover the HR image. To further generate realistic faces, we propose the Face Super-Resolution Generative Adversarial Network (FSRGAN) to incorporate the adversarial loss into FSRNet. Moreover, we introduce two related tasks, face alignment and parsing, as the new evaluation metrics for face SR, which address the inconsistency of classic metrics w.r.t. visual perception. Extensive benchmark experiments show that FSRNet and FSRGAN significantly outperforms state of the arts for very LR face SR, both quantitatively and qualitatively. Code will be made available upon publication.
Tasks	Face Alignment, Super-Resolution
Published	2017-11-29
URL	http://arxiv.org/abs/1711.10703v1
PDF	http://arxiv.org/pdf/1711.10703v1.pdf
PWC	https://paperswithcode.com/paper/fsrnet-end-to-end-learning-face-super
Repo	https://github.com/ZoieMo/Multi-task
Framework	none

Data Augmentation Generative Adversarial Networks


Title	Data Augmentation Generative Adversarial Networks
Authors	Antreas Antoniou, Amos Storkey, Harrison Edwards
Abstract	Effective training of neural networks requires much data. In the low-data regime, parameters are underdetermined, and learnt networks generalise poorly. Data Augmentation alleviates this by using existing data more effectively. However standard data augmentation produces only limited plausible alternative data. Given there is potential to generate a much broader set of augmentations, we design and train a generative model to do data augmentation. The model, based on image conditional Generative Adversarial Networks, takes data from a source domain and learns to take any data item and generalise it to generate other within-class data items. As this generative process does not depend on the classes themselves, it can be applied to novel unseen classes of data. We show that a Data Augmentation Generative Adversarial Network (DAGAN) augments standard vanilla classifiers well. We also show a DAGAN can enhance few-shot learning systems such as Matching Networks. We demonstrate these approaches on Omniglot, on EMNIST having learnt the DAGAN on Omniglot, and VGG-Face data. In our experiments we can see over 13% increase in accuracy in the low-data regime experiments in Omniglot (from 69% to 82%), EMNIST (73.9% to 76%) and VGG-Face (4.5% to 12%); in Matching Networks for Omniglot we observe an increase of 0.5% (from 96.9% to 97.4%) and an increase of 1.8% in EMNIST (from 59.5% to 61.3%).
Tasks	Data Augmentation, Few-Shot Learning, Omniglot
Published	2017-11-12
URL	http://arxiv.org/abs/1711.04340v3
PDF	http://arxiv.org/pdf/1711.04340v3.pdf
PWC	https://paperswithcode.com/paper/data-augmentation-generative-adversarial
Repo	https://github.com/AntreasAntoniou/DAGAN
Framework	none

Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks


Title	Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks
Authors	Shiyu Liang, Yixuan Li, R. Srikant
Abstract	We consider the problem of detecting out-of-distribution images in neural networks. We propose ODIN, a simple and effective method that does not require any change to a pre-trained neural network. Our method is based on the observation that using temperature scaling and adding small perturbations to the input can separate the softmax score distributions between in- and out-of-distribution images, allowing for more effective detection. We show in a series of experiments that ODIN is compatible with diverse network architectures and datasets. It consistently outperforms the baseline approach by a large margin, establishing a new state-of-the-art performance on this task. For example, ODIN reduces the false positive rate from the baseline 34.7% to 4.3% on the DenseNet (applied to CIFAR-10) when the true positive rate is 95%.
Tasks
Published	2017-06-08
URL	http://arxiv.org/abs/1706.02690v4
PDF	http://arxiv.org/pdf/1706.02690v4.pdf
PWC	https://paperswithcode.com/paper/enhancing-the-reliability-of-out-of
Repo	https://github.com/ShiyuLiang/odin-pytorch
Framework	pytorch

Neural Map: Structured Memory for Deep Reinforcement Learning


Title	Neural Map: Structured Memory for Deep Reinforcement Learning
Authors	Emilio Parisotto, Ruslan Salakhutdinov
Abstract	A critical component to enabling intelligent reasoning in partially observable environments is memory. Despite this importance, Deep Reinforcement Learning (DRL) agents have so far used relatively simple memory architectures, with the main methods to overcome partial observability being either a temporal convolution over the past k frames or an LSTM layer. More recent work (Oh et al., 2016) has went beyond these architectures by using memory networks which can allow more sophisticated addressing schemes over the past k frames. But even these architectures are unsatisfactory due to the reason that they are limited to only remembering information from the last k frames. In this paper, we develop a memory system with an adaptable write operator that is customized to the sorts of 3D environments that DRL agents typically interact with. This architecture, called the Neural Map, uses a spatially structured 2D memory image to learn to store arbitrary information about the environment over long time lags. We demonstrate empirically that the Neural Map surpasses previous DRL memories on a set of challenging 2D and 3D maze environments and show that it is capable of generalizing to environments that were not seen during training.
Tasks
Published	2017-02-27
URL	http://arxiv.org/abs/1702.08360v1
PDF	http://arxiv.org/pdf/1702.08360v1.pdf
PWC	https://paperswithcode.com/paper/neural-map-structured-memory-for-deep
Repo	https://github.com/zuoxingdong/VIN_PyTorch_Visdom
Framework	pytorch

Style Transfer from Non-Parallel Text by Cross-Alignment


Title	Style Transfer from Non-Parallel Text by Cross-Alignment
Authors	Tianxiao Shen, Tao Lei, Regina Barzilay, Tommi Jaakkola
Abstract	This paper focuses on style transfer on the basis of non-parallel text. This is an instance of a broad family of problems including machine translation, decipherment, and sentiment modification. The key challenge is to separate the content from other aspects such as style. We assume a shared latent content distribution across different text corpora, and propose a method that leverages refined alignment of latent representations to perform style transfer. The transferred sentences from one style should match example sentences from the other style as a population. We demonstrate the effectiveness of this cross-alignment method on three tasks: sentiment modification, decipherment of word substitution ciphers, and recovery of word order.
Tasks	Machine Translation, Style Transfer, Text Style Transfer
Published	2017-05-26
URL	http://arxiv.org/abs/1705.09655v2
PDF	http://arxiv.org/pdf/1705.09655v2.pdf
PWC	https://paperswithcode.com/paper/style-transfer-from-non-parallel-text-by
Repo	https://github.com/sy-sunmoon/Clever-Commenter-Let-s-Try-More-Apps
Framework	pytorch

Neural Network Dynamics for Model-Based Deep Reinforcement Learning with Model-Free Fine-Tuning


Title	Neural Network Dynamics for Model-Based Deep Reinforcement Learning with Model-Free Fine-Tuning
Authors	Anusha Nagabandi, Gregory Kahn, Ronald S. Fearing, Sergey Levine
Abstract	Model-free deep reinforcement learning algorithms have been shown to be capable of learning a wide range of robotic skills, but typically require a very large number of samples to achieve good performance. Model-based algorithms, in principle, can provide for much more efficient learning, but have proven difficult to extend to expressive, high-capacity models such as deep neural networks. In this work, we demonstrate that medium-sized neural network models can in fact be combined with model predictive control (MPC) to achieve excellent sample complexity in a model-based reinforcement learning algorithm, producing stable and plausible gaits to accomplish various complex locomotion tasks. We also propose using deep neural network dynamics models to initialize a model-free learner, in order to combine the sample efficiency of model-based approaches with the high task-specific performance of model-free methods. We empirically demonstrate on MuJoCo locomotion tasks that our pure model-based approach trained on just random action data can follow arbitrary trajectories with excellent sample efficiency, and that our hybrid algorithm can accelerate model-free learning on high-speed benchmark tasks, achieving sample efficiency gains of 3-5x on swimmer, cheetah, hopper, and ant agents. Videos can be found at https://sites.google.com/view/mbmf
Tasks
Published	2017-08-08
URL	http://arxiv.org/abs/1708.02596v2
PDF	http://arxiv.org/pdf/1708.02596v2.pdf
PWC	https://paperswithcode.com/paper/neural-network-dynamics-for-model-based-deep
Repo	https://github.com/nagaban2/nn_dynamics
Framework	tf

Real-time On-Demand Crowd-powered Entity Extraction


Title	Real-time On-Demand Crowd-powered Entity Extraction
Authors	Ting-Hao ‘Kenneth’ Huang, Yun-Nung Chen, Jeffrey P. Bigham
Abstract	Output-agreement mechanisms such as ESP Game have been widely used in human computation to obtain reliable human-generated labels. In this paper, we argue that a “time-limited” output-agreement mechanism can be used to create a fast and robust crowd-powered component in interactive systems, particularly dialogue systems, to extract key information from user utterances on the fly. Our experiments on Amazon Mechanical Turk using the Airline Travel Information System (ATIS) dataset showed that the proposed approach achieves high-quality results with an average response time shorter than 9 seconds.
Tasks	Entity Extraction
Published	2017-04-12
URL	http://arxiv.org/abs/1704.03627v2
PDF	http://arxiv.org/pdf/1704.03627v2.pdf
PWC	https://paperswithcode.com/paper/real-time-on-demand-crowd-powered-entity
Repo	https://github.com/windx0303/dialogue-esp-game
Framework	none

Universal Dependencies Parsing for Colloquial Singaporean English


Title	Universal Dependencies Parsing for Colloquial Singaporean English
Authors	Hongmin Wang, Yue Zhang, GuangYong Leonard Chan, Jie Yang, Hai Leong Chieu
Abstract	Singlish can be interesting to the ACL community both linguistically as a major creole based on English, and computationally for information extraction and sentiment analysis of regional social media. We investigate dependency parsing of Singlish by constructing a dependency treebank under the Universal Dependencies scheme, and then training a neural network model by integrating English syntactic knowledge into a state-of-the-art parser trained on the Singlish treebank. Results show that English knowledge can lead to 25% relative error reduction, resulting in a parser of 84.47% accuracies. To the best of our knowledge, we are the first to use neural stacking to improve cross-lingual dependency parsing on low-resource languages. We make both our annotation and parser available for further research.
Tasks	Dependency Parsing, Sentiment Analysis
Published	2017-05-18
URL	http://arxiv.org/abs/1705.06463v1
PDF	http://arxiv.org/pdf/1705.06463v1.pdf
PWC	https://paperswithcode.com/paper/universal-dependencies-parsing-for-colloquial
Repo	https://github.com/wanghm92/Sing_Par
Framework	tf