January 27, 2020

3096 words 15 mins read

Paper Group ANR 1288

Don’t Forget the Long Tail! A Comprehensive Analysis of Morphological Generalization in Bilingual Lexicon Induction. Road Context-aware Intrusion Detection System for Autonomous Cars. ART: Abstraction Refinement-Guided Training for Provably Correct Neural Networks. Predictive Uncertainty Quantification with Compound Density Networks. Challenges in …

Don’t Forget the Long Tail! A Comprehensive Analysis of Morphological Generalization in Bilingual Lexicon Induction


Title	Don’t Forget the Long Tail! A Comprehensive Analysis of Morphological Generalization in Bilingual Lexicon Induction
Authors	Paula Czarnowska, Sebastian Ruder, Edouard Grave, Ryan Cotterell, Ann Copestake
Abstract	Human translators routinely have to translate rare inflections of words - due to the Zipfian distribution of words in a language. When translating from Spanish, a good translator would have no problem identifying the proper translation of a statistically rare inflection such as habl'aramos. Note the lexeme itself, hablar, is relatively common. In this work, we investigate whether state-of-the-art bilingual lexicon inducers are capable of learning this kind of generalization. We introduce 40 morphologically complete dictionaries in 10 languages and evaluate three of the state-of-the-art models on the task of translation of less frequent morphological forms. We demonstrate that the performance of state-of-the-art models drops considerably when evaluated on infrequent morphological inflections and then show that adding a simple morphological constraint at training time improves the performance, proving that the bilingual lexicon inducers can benefit from better encoding of morphology.
Tasks
Published	2019-09-06
URL	https://arxiv.org/abs/1909.02855v2
PDF	https://arxiv.org/pdf/1909.02855v2.pdf
PWC	https://paperswithcode.com/paper/dont-forget-the-long-tail-a-comprehensive
Repo
Framework

Road Context-aware Intrusion Detection System for Autonomous Cars


Title	Road Context-aware Intrusion Detection System for Autonomous Cars
Authors	Jingxuan Jiang, Chundong Wang, Sudipta Chattopadhyay, Wei Zhang
Abstract	Security is of primary importance to vehicles. The viability of performing remote intrusions onto the in-vehicle network has been manifested. In regard to unmanned autonomous cars, limited work has been done to detect intrusions for them while existing intrusion detection systems (IDSs) embrace limitations against strong adversaries. In this paper, we consider the very nature of autonomous car and leverage the road context to build a novel IDS, named Road context-aware IDS (RAIDS). When a computer-controlled car is driving through continuous roads, road contexts and genuine frames transmitted on the car’s in-vehicle network should resemble a regular and intelligible pattern. RAIDS hence employs a lightweight machine learning model to extract road contexts from sensory information (e.g., camera images and distance sensor values) that are used to generate control signals for maneuvering the car. With such ongoing road context, RAIDS validates corresponding frames observed on the in-vehicle network. Anomalous frames that substantially deviate from road context will be discerned as intrusions. We have implemented a prototype of RAIDS with neural networks, and conducted experiments on a Raspberry Pi with extensive datasets and meaningful intrusion cases. Evaluations show that RAIDS significantly outperforms state-of-the-art IDS without using road context by up to 99.9% accuracy and short response time.
Tasks	Intrusion Detection
Published	2019-08-02
URL	https://arxiv.org/abs/1908.00732v1
PDF	https://arxiv.org/pdf/1908.00732v1.pdf
PWC	https://paperswithcode.com/paper/road-context-aware-intrusion-detection-system
Repo
Framework

ART: Abstraction Refinement-Guided Training for Provably Correct Neural Networks


Title	ART: Abstraction Refinement-Guided Training for Provably Correct Neural Networks
Authors	Xuankang Lin, He Zhu, Roopsha Samanta, Suresh Jagannathan
Abstract	Artificial neural networks (ANNs) have demonstrated remarkable utility in a variety of challenging machine learning applications. However, their complex architecture makes asserting any formal guarantees about their behavior difficult. Existing approaches to this problem typically consider verification as a post facto white-box process, one that reasons about the safety of an existing network through exploration of its internal structure, rather than via a methodology that ensures the network is correct-by-construction. In this paper, we present a novel learning framework that takes an important first step towards realizing such a methodology. Our technique enables the construction of provably correct networks with respect to a broad class of safety properties, a capability that goes well-beyond existing approaches. Overcoming the challenge of general safety property enforcement within the network training process in a supervised learning pipeline, however, requires a fundamental shift in how we architect and build ANNs. Our key insight is that we can integrate an optimization-based abstraction refinement loop into the learning process that iteratively splits the input space from which training data is drawn, based on the efficacy with which such a partition enables safety verification. To do so, our approach enables training to take place over an abstraction of a concrete network that operates over dynamically constructed partitions of the input space. We provide theoretical results that show that classical gradient descent methods used to optimize these networks can be seamlessly adopted to this framework to ensure soundness of our approach. Moreover, we empirically demonstrate that realizing soundness does not come at the price of accuracy, giving us a meaningful pathway for building both precise and correct networks.
Tasks
Published	2019-07-17
URL	https://arxiv.org/abs/1907.10662v2
PDF	https://arxiv.org/pdf/1907.10662v2.pdf
PWC	https://paperswithcode.com/paper/art-abstraction-refinement-guided-training
Repo
Framework

Predictive Uncertainty Quantification with Compound Density Networks


Title	Predictive Uncertainty Quantification with Compound Density Networks
Authors	Agustinus Kristiadi, Sina Däubener, Asja Fischer
Abstract	Despite the huge success of deep neural networks (NNs), finding good mechanisms for quantifying their prediction uncertainty is still an open problem. Bayesian neural networks are one of the most popular approaches to uncertainty quantification. On the other hand, it was recently shown that ensembles of NNs, which belong to the class of mixture models, can be used to quantify prediction uncertainty. In this paper, we build upon these two approaches. First, we increase the mixture model’s flexibility by replacing the fixed mixing weights by an adaptive, input-dependent distribution (specifying the probability of each component) represented by NNs, and by considering uncountably many mixture components. The resulting class of models can be seen as the continuous counterpart to mixture density networks and is therefore referred to as compound density networks (CDNs). We employ both maximum likelihood and variational Bayesian inference to train CDNs, and empirically show that they yield better uncertainty estimates on out-of-distribution data and are more robust to adversarial examples than the previous approaches.
Tasks	Bayesian Inference
Published	2019-02-04
URL	https://arxiv.org/abs/1902.01080v2
PDF	https://arxiv.org/pdf/1902.01080v2.pdf
PWC	https://paperswithcode.com/paper/predictive-uncertainty-quantification-with
Repo
Framework

Challenges in Bayesian inference via Markov chain Monte Carlo for neural networks


Title	Challenges in Bayesian inference via Markov chain Monte Carlo for neural networks
Authors	Theodore Papamarkou, Jacob Hinkle, M. Todd Young, David Womble
Abstract	Markov chain Monte Carlo (MCMC) methods and neural networks are instrumental in tackling inferential and prediction problems. However, Bayesian inference based on joint use of MCMC methods and of neural networks is limited. This paper reviews the main challenges posed by neural networks to MCMC developments, including lack of parameter identifiability due to weight symmetries, prior specification effects, and consequently high computational cost and convergence failure. Population and manifold MCMC algorithms are combined to demonstrate these challenges via multilayer perceptron (MLP) examples and to develop case studies for assessing the capacity of approximate inference methods to uncover the posterior covariance of neural network parameters. Some of these challenges, such as high computational cost arising from the application of neural networks to big data and parameter identifiability arising from weight symmetries, stimulate research towards more scalable approximate MCMC methods or towards MCMC methods in reduced parameter spaces.
Tasks	Bayesian Inference
Published	2019-10-15
URL	https://arxiv.org/abs/1910.06539v3
PDF	https://arxiv.org/pdf/1910.06539v3.pdf
PWC	https://paperswithcode.com/paper/challenges-in-bayesian-inference-via-markov
Repo
Framework

Structured Binary Neural Networks for Image Recognition


Title	Structured Binary Neural Networks for Image Recognition
Authors	Bohan Zhuang, Chunhua Shen, Mingkui Tan, Lingqiao Liu, Ian Reid
Abstract	We propose methods to train convolutional neural networks (CNNs) with both binarized weights and activations, leading to quantized models that are specifically friendly to mobile devices with limited power capacity and computation resources. Previous works on quantizing CNNs often seek to approximate the floating-point information using a set of discrete values, which we call value approximation, typically assuming the same architecture as the full-precision networks. Here we take a novel “structure approximation” view of quantization—it is very likely that different architectures designed for low-bit networks may be better for achieving good performance. In particular, we propose a “network decomposition” strategy, termed Group-Net, in which we divide the network into groups. Thus, each full-precision group can be effectively reconstructed by aggregating a set of homogeneous binary branches. In addition, we learn effective connections among groups to improve the representation capability. Moreover, the proposed Group-Net shows strong generalization to other tasks. For instance, we extend Group-Net for accurate semantic segmentation by embedding rich context into the binary structure. Furthermore, for the first time, we apply binary neural networks to object detection. Experiments on both classification, semantic segmentation and object detection tasks demonstrate the superior performance of the proposed methods over various quantized networks in the literature. Our methods outperform the previous best binary neural networks in terms of accuracy and computation efficiency.
Tasks	Object Detection, Quantization, Semantic Segmentation
Published	2019-09-22
URL	https://arxiv.org/abs/1909.09934v1
PDF	https://arxiv.org/pdf/1909.09934v1.pdf
PWC	https://paperswithcode.com/paper/190909934
Repo
Framework

ShadingNet: Image Intrinsics by Fine-Grained Shading Decomposition


Title	ShadingNet: Image Intrinsics by Fine-Grained Shading Decomposition
Authors	Anil S. Baslamisli, Partha Das, Hoang-An Le, Sezer Karaoglu, Theo Gevers
Abstract	In general, intrinsic image decomposition algorithms interpret shading as one unified component including all photometric effects. As shading transitions are generally smoother than albedo changes, these methods may fail in distinguishing strong (cast) shadows from albedo variations. That in return may leak into albedo map predictions. Therefore, in this paper, we propose to decompose the shading component into direct (illumination) and indirect shading (ambient light and shadows). The aim is to distinguish strong cast shadows from reflectance variations. Two end-to-end supervised CNN models (ShadingNets) are proposed exploiting the fine-grained shading model. Furthermore, surface normal features are jointly learned by the proposed CNN networks. Surface normals are expected to assist the decomposition task. A large-scale dataset of scene-level synthetic images of outdoor natural environments is provided with intrinsic image ground-truths. Large scale experiments show that our CNN approach using fine-grained shading decomposition outperforms state-of-the-art methods using unified shading.
Tasks	Intrinsic Image Decomposition
Published	2019-12-09
URL	https://arxiv.org/abs/1912.04023v1
PDF	https://arxiv.org/pdf/1912.04023v1.pdf
PWC	https://paperswithcode.com/paper/shadingnet-image-intrinsics-by-fine-grained
Repo
Framework

Towards Time-Aware Distant Supervision for Relation Extraction


Title	Towards Time-Aware Distant Supervision for Relation Extraction
Authors	Tianwen Jiang, Sendong Zhao, Jing Liu, Jin-Ge Yao, Ming Liu, Bing Qin, Ting Liu, Chin-Yew Lin
Abstract	Distant supervision for relation extraction heavily suffers from the wrong labeling problem. To alleviate this issue in news data with the timestamp, we take a new factor time into consideration and propose a novel time-aware distant supervision framework (Time-DS). Time-DS is composed of a time series instance-popularity and two strategies. Instance-popularity is to encode the strong relevance of time and true relation mention. Therefore, instance-popularity would be an effective clue to reduce the noises generated through distant supervision labeling. The two strategies, i.e., hard filter and curriculum learning are both ways to implement instance-popularity for better relation extraction in the manner of Time-DS. The curriculum learning is a more sophisticated and flexible way to exploit instance-popularity to eliminate the bad effects of noises, thus get better relation extraction performance. Experiments on our collected multi-source news corpus show that Time-DS achieves significant improvements for relation extraction.
Tasks	Relation Extraction, Time Series
Published	2019-03-08
URL	http://arxiv.org/abs/1903.03289v1
PDF	http://arxiv.org/pdf/1903.03289v1.pdf
PWC	https://paperswithcode.com/paper/towards-time-aware-distant-supervision-for
Repo
Framework

The Algorithmic Automation Problem: Prediction, Triage, and Human Effort


Title	The Algorithmic Automation Problem: Prediction, Triage, and Human Effort
Authors	Maithra Raghu, Katy Blumer, Greg Corrado, Jon Kleinberg, Ziad Obermeyer, Sendhil Mullainathan
Abstract	In a wide array of areas, algorithms are matching and surpassing the performance of human experts, leading to consideration of the roles of human judgment and algorithmic prediction in these domains. The discussion around these developments, however, has implicitly equated the specific task of prediction with the general task of automation. We argue here that automation is broader than just a comparison of human versus algorithmic performance on a task; it also involves the decision of which instances of the task to give to the algorithm in the first place. We develop a general framework that poses this latter decision as an optimization problem, and we show how basic heuristics for this optimization problem can lead to performance gains even on heavily-studied applications of AI in medicine. Our framework also serves to highlight how effective automation depends crucially on estimating both algorithmic and human error on an instance-by-instance basis, and our results show how improvements in these error estimation problems can yield significant gains for automation as well.
Tasks
Published	2019-03-28
URL	http://arxiv.org/abs/1903.12220v1
PDF	http://arxiv.org/pdf/1903.12220v1.pdf
PWC	https://paperswithcode.com/paper/the-algorithmic-automation-problem-prediction
Repo
Framework

Duality in RKHSs with Infinite Dimensional Outputs: Application to Robust Losses


Title	Duality in RKHSs with Infinite Dimensional Outputs: Application to Robust Losses
Authors	Pierre Laforgue, Alex Lambert, Luc Brogat-Motte, Florence d’Alché-Buc
Abstract	Operator-Valued Kernels (OVKs) and associated vector-valued Reproducing Kernel Hilbert Spaces provide an elegant way to extend scalar kernel methods when the output space is a Hilbert space. Although primarily used in finite dimension for problems like multi-task regression, the ability of this framework to deal with infinite dimensional output spaces unlocks many more applications, such as functional regression, structured output prediction, and structured data representation. However, these sophisticated schemes crucially rely on the kernel trick in the output space, so that most of previous works have focused on the square norm loss function, completely neglecting robustness issues that may arise in such surrogate problems. To overcome this limitation, this paper develops a duality approach that allows to solve OVK machines for a wide range of loss functions. The infinite dimensional Lagrange multipliers are handled through a Double Representer Theorem, and algorithms for $\epsilon$-insensitive losses and the Huber loss are thoroughly detailed. Robustness benefits are emphasized by a theoretical stability analysis, as well as empirical improvements on structured data applications.
Tasks	Representation Learning, Structured Prediction
Published	2019-10-10
URL	https://arxiv.org/abs/1910.04621v2
PDF	https://arxiv.org/pdf/1910.04621v2.pdf
PWC	https://paperswithcode.com/paper/on-the-dualization-of-operator-valued-kernel
Repo
Framework

Towards Improved Testing For Deep Learning


Title	Towards Improved Testing For Deep Learning
Authors	Jasmine Sekhon, Cody Fleming
Abstract	The growing use of deep neural networks in safety-critical applications makes it necessary to carry out adequate testing to detect and correct any incorrect behavior for corner case inputs before they can be actually used. Deep neural networks lack an explicit control-flow structure, making it impossible to apply to them traditional software testing criteria such as code coverage. In this paper, we examine existing testing methods for deep neural networks, the opportunities for improvement and the need for a fast, scalable, generalizable end-to-end testing method. We also propose a coverage criterion for deep neural networks that tries to capture all possible parts of the deep neural network’s logic.
Tasks
Published	2019-02-17
URL	http://arxiv.org/abs/1902.06320v1
PDF	http://arxiv.org/pdf/1902.06320v1.pdf
PWC	https://paperswithcode.com/paper/towards-improved-testing-for-deep-learning
Repo
Framework

Unsupervised Learning for Intrinsic Image Decomposition from a Single Image


Title	Unsupervised Learning for Intrinsic Image Decomposition from a Single Image
Authors	Yunfei Liu, Yu Li, Shaodi You, Feng Lu
Abstract	Intrinsic image decomposition, which is an essential task in computer vision, aims to infer the reflectance and shading of the scene. It is challenging since it needs to separate one image into two components. To tackle this, conventional methods introduce various priors to constrain the solution, yet with limited performance. Meanwhile, the problem is typically solved by supervised learning methods, which is actually not an ideal solution since obtaining ground truth reflectance and shading for massive general natural scenes is challenging and even impossible. In this paper, we propose a novel unsupervised intrinsic image decomposition framework, which relies on neither labeled training data nor hand-crafted priors. Instead, it directly learns the latent feature of reflectance and shading from unsupervised and uncorrelated data. To enable this, we explore the independence between reflectance and shading, the domain invariant content constraint and the physical constraint. Extensive experiments on both synthetic and real image datasets demonstrate consistently superior performance of the proposed method.
Tasks	Intrinsic Image Decomposition
Published	2019-11-22
URL	https://arxiv.org/abs/1911.09930v1
PDF	https://arxiv.org/pdf/1911.09930v1.pdf
PWC	https://paperswithcode.com/paper/unsupervised-learning-for-intrinsic-image
Repo
Framework

Generalized Matrix Means for Semi-Supervised Learning with Multilayer Graphs


Title	Generalized Matrix Means for Semi-Supervised Learning with Multilayer Graphs
Authors	Pedro Mercado, Francesco Tudisco, Matthias Hein
Abstract	We study the task of semi-supervised learning on multilayer graphs by taking into account both labeled and unlabeled observations together with the information encoded by each individual graph layer. We propose a regularizer based on the generalized matrix mean, which is a one-parameter family of matrix means that includes the arithmetic, geometric and harmonic means as particular cases. We analyze it in expectation under a Multilayer Stochastic Block Model and verify numerically that it outperforms state of the art methods. Moreover, we introduce a matrix-free numerical scheme based on contour integral quadratures and Krylov subspace solvers that scales to large sparse multilayer graphs.
Tasks
Published	2019-10-30
URL	https://arxiv.org/abs/1910.13951v1
PDF	https://arxiv.org/pdf/1910.13951v1.pdf
PWC	https://paperswithcode.com/paper/generalized-matrix-means-for-semi-supervised
Repo
Framework

Relation Extraction using Explicit Context Conditioning


Title	Relation Extraction using Explicit Context Conditioning
Authors	Gaurav Singh, Parminder Bhatia
Abstract	Relation Extraction (RE) aims to label relations between groups of marked entities in raw text. Most current RE models learn context-aware representations of the target entities that are then used to establish relation between them. This works well for intra-sentence RE and we call them first-order relations. However, this methodology can sometimes fail to capture complex and long dependencies. To address this, we hypothesize that at times two target entities can be explicitly connected via a context token. We refer to such indirect relations as second-order relations and describe an efficient implementation for computing them. These second-order relation scores are then combined with first-order relation scores. Our empirical results show that the proposed method leads to state-of-the-art performance over two biomedical datasets.
Tasks	Relation Extraction
Published	2019-02-25
URL	http://arxiv.org/abs/1902.09271v1
PDF	http://arxiv.org/pdf/1902.09271v1.pdf
PWC	https://paperswithcode.com/paper/relation-extraction-using-explicit-context
Repo
Framework

Compression based bound for non-compressed network: unified generalization error analysis of large compressible deep neural network


Title	Compression based bound for non-compressed network: unified generalization error analysis of large compressible deep neural network
Authors	Taiji Suzuki
Abstract	One of biggest issues in deep learning theory is its generalization ability despite the huge model size. The classical learning theory suggests that overparameterized models cause overfitting. However, practically used large deep models avoid overfitting, which is not well explained by the classical approaches. To resolve this issue, several attempts have been made. Among them, the compression based bound is one of the promising approaches. However, the compression based bound can be applied only to a compressed network, and it is not applicable to the non-compressed original network. In this paper, we give a unified frame-work that can convert compression based bounds to those for non-compressed original networks. The bound gives even better rate than the one for the compressed network by improving the bias term. By establishing the unified frame-work, we can obtain a data dependent generalization error bound which gives a tighter evaluation than the data independent ones.
Tasks
Published	2019-09-25
URL	https://arxiv.org/abs/1909.11274v2
PDF	https://arxiv.org/pdf/1909.11274v2.pdf
PWC	https://paperswithcode.com/paper/compression-based-bound-for-non-compressed-1
Repo
Framework