July 30, 2019

3348 words 16 mins read

Paper Group AWR 24

DeepTest: Automated Testing of Deep-Neural-Network-driven Autonomous Cars. A Distributed Frank-Wolfe Framework for Learning Low-Rank Matrices with the Trace Norm. Smooth Neighbors on Teacher Graphs for Semi-supervised Learning. Bayesian Nonlinear Support Vector Machines for Big Data. The biglasso Package: A Memory- and Computation-Efficient Solver …

DeepTest: Automated Testing of Deep-Neural-Network-driven Autonomous Cars


Title	DeepTest: Automated Testing of Deep-Neural-Network-driven Autonomous Cars
Authors	Yuchi Tian, Kexin Pei, Suman Jana, Baishakhi Ray
Abstract	Recent advances in Deep Neural Networks (DNNs) have led to the development of DNN-driven autonomous cars that, using sensors like camera, LiDAR, etc., can drive without any human intervention. Most major manufacturers including Tesla, GM, Ford, BMW, and Waymo/Google are working on building and testing different types of autonomous vehicles. The lawmakers of several US states including California, Texas, and New York have passed new legislation to fast-track the process of testing and deployment of autonomous vehicles on their roads. However, despite their spectacular progress, DNNs, just like traditional software, often demonstrate incorrect or unexpected corner case behaviors that can lead to potentially fatal collisions. Several such real-world accidents involving autonomous cars have already happened including one which resulted in a fatality. Most existing testing techniques for DNN-driven vehicles are heavily dependent on the manual collection of test data under different driving conditions which become prohibitively expensive as the number of test conditions increases. In this paper, we design, implement and evaluate DeepTest, a systematic testing tool for automatically detecting erroneous behaviors of DNN-driven vehicles that can potentially lead to fatal crashes. First, our tool is designed to automatically generated test cases leveraging real-world changes in driving conditions like rain, fog, lighting conditions, etc. DeepTest systematically explores different parts of the DNN logic by generating test inputs that maximize the numbers of activated neurons. DeepTest found thousands of erroneous behaviors under different realistic driving conditions (e.g., blurring, rain, fog, etc.) many of which lead to potentially fatal crashes in three top performing DNNs in the Udacity self-driving car challenge.
Tasks	Autonomous Vehicles
Published	2017-08-28
URL	http://arxiv.org/abs/1708.08559v2
PDF	http://arxiv.org/pdf/1708.08559v2.pdf
PWC	https://paperswithcode.com/paper/deeptest-automated-testing-of-deep-neural
Repo	https://github.com/ARiSE-Lab/deepTest
Framework	none

A Distributed Frank-Wolfe Framework for Learning Low-Rank Matrices with the Trace Norm


Title	A Distributed Frank-Wolfe Framework for Learning Low-Rank Matrices with the Trace Norm
Authors	Wenjie Zheng, Aurélien Bellet, Patrick Gallinari
Abstract	We consider the problem of learning a high-dimensional but low-rank matrix from a large-scale dataset distributed over several machines, where low-rankness is enforced by a convex trace norm constraint. We propose DFW-Trace, a distributed Frank-Wolfe algorithm which leverages the low-rank structure of its updates to achieve efficiency in time, memory and communication usage. The step at the heart of DFW-Trace is solved approximately using a distributed version of the power method. We provide a theoretical analysis of the convergence of DFW-Trace, showing that we can ensure sublinear convergence in expectation to an optimal solution with few power iterations per epoch. We implement DFW-Trace in the Apache Spark distributed programming framework and validate the usefulness of our approach on synthetic and real data, including the ImageNet dataset with high-dimensional features extracted from a deep neural network.
Tasks
Published	2017-12-20
URL	http://arxiv.org/abs/1712.07495v2
PDF	http://arxiv.org/pdf/1712.07495v2.pdf
PWC	https://paperswithcode.com/paper/a-distributed-frank-wolfe-framework-for
Repo	https://github.com/WenjieZ/distributed-frank-wolfe
Framework	none

Smooth Neighbors on Teacher Graphs for Semi-supervised Learning


Title	Smooth Neighbors on Teacher Graphs for Semi-supervised Learning
Authors	Yucen Luo, Jun Zhu, Mengxi Li, Yong Ren, Bo Zhang
Abstract	The recently proposed self-ensembling methods have achieved promising results in deep semi-supervised learning, which penalize inconsistent predictions of unlabeled data under different perturbations. However, they only consider adding perturbations to each single data point, while ignoring the connections between data samples. In this paper, we propose a novel method, called Smooth Neighbors on Teacher Graphs (SNTG). In SNTG, a graph is constructed based on the predictions of the teacher model, i.e., the implicit self-ensemble of models. Then the graph serves as a similarity measure with respect to which the representations of “similar” neighboring points are learned to be smooth on the low-dimensional manifold. We achieve state-of-the-art results on semi-supervised learning benchmarks. The error rates are 9.89%, 3.99% for CIFAR-10 with 4000 labels, SVHN with 500 labels, respectively. In particular, the improvements are significant when the labels are fewer. For the non-augmented MNIST with only 20 labels, the error rate is reduced from previous 4.81% to 1.36%. Our method also shows robustness to noisy labels.
Tasks
Published	2017-11-01
URL	http://arxiv.org/abs/1711.00258v2
PDF	http://arxiv.org/pdf/1711.00258v2.pdf
PWC	https://paperswithcode.com/paper/smooth-neighbors-on-teacher-graphs-for-semi
Repo	https://github.com/xinmei9322/SNTG
Framework	none

Bayesian Nonlinear Support Vector Machines for Big Data


Title	Bayesian Nonlinear Support Vector Machines for Big Data
Authors	Florian Wenzel, Theo Galy-Fajou, Matthaeus Deutsch, Marius Kloft
Abstract	We propose a fast inference method for Bayesian nonlinear support vector machines that leverages stochastic variational inference and inducing points. Our experiments show that the proposed method is faster than competing Bayesian approaches and scales easily to millions of data points. It provides additional features over frequentist competitors such as accurate predictive uncertainty estimates and automatic hyperparameter search.
Tasks
Published	2017-07-18
URL	http://arxiv.org/abs/1707.05532v1
PDF	http://arxiv.org/pdf/1707.05532v1.pdf
PWC	https://paperswithcode.com/paper/bayesian-nonlinear-support-vector-machines
Repo	https://github.com/theogf/BayesianSVM
Framework	tf

The biglasso Package: A Memory- and Computation-Efficient Solver for Lasso Model Fitting with Big Data in R


Title	The biglasso Package: A Memory- and Computation-Efficient Solver for Lasso Model Fitting with Big Data in R
Authors	Yaohui Zeng, Patrick Breheny
Abstract	Penalized regression models such as the lasso have been extensively applied to analyzing high-dimensional data sets. However, due to memory limitations, existing R packages like glmnet and ncvreg are not capable of fitting lasso-type models for ultrahigh-dimensional, multi-gigabyte data sets that are increasingly seen in many areas such as genetics, genomics, biomedical imaging, and high-frequency finance. In this research, we implement an R package called biglasso that tackles this challenge. biglasso utilizes memory-mapped files to store the massive data on the disk, only reading data into memory when necessary during model fitting, and is thus able to handle out-of-core computation seamlessly. Moreover, it’s equipped with newly proposed, more efficient feature screening rules, which substantially accelerate the computation. Benchmarking experiments show that our biglasso package, as compared to existing popular ones like glmnet, is much more memory- and computation-efficient. We further analyze a 31 GB real data set on a laptop with only 16 GB RAM to demonstrate the out-of-core computation capability of biglasso in analyzing massive data sets that cannot be accommodated by existing R packages.
Tasks
Published	2017-01-20
URL	http://arxiv.org/abs/1701.05936v2
PDF	http://arxiv.org/pdf/1701.05936v2.pdf
PWC	https://paperswithcode.com/paper/the-biglasso-package-a-memory-and-computation
Repo	https://github.com/YaohuiZeng/biglasso
Framework	none

DeepXplore: Automated Whitebox Testing of Deep Learning Systems


Title	DeepXplore: Automated Whitebox Testing of Deep Learning Systems
Authors	Kexin Pei, Yinzhi Cao, Junfeng Yang, Suman Jana
Abstract	Deep learning (DL) systems are increasingly deployed in safety- and security-critical domains including self-driving cars and malware detection, where the correctness and predictability of a system’s behavior for corner case inputs are of great importance. Existing DL testing depends heavily on manually labeled data and therefore often fails to expose erroneous behaviors for rare inputs. We design, implement, and evaluate DeepXplore, the first whitebox framework for systematically testing real-world DL systems. First, we introduce neuron coverage for systematically measuring the parts of a DL system exercised by test inputs. Next, we leverage multiple DL systems with similar functionality as cross-referencing oracles to avoid manual checking. Finally, we demonstrate how finding inputs for DL systems that both trigger many differential behaviors and achieve high neuron coverage can be represented as a joint optimization problem and solved efficiently using gradient-based search techniques. DeepXplore efficiently finds thousands of incorrect corner case behaviors (e.g., self-driving cars crashing into guard rails and malware masquerading as benign software) in state-of-the-art DL models with thousands of neurons trained on five popular datasets including ImageNet and Udacity self-driving challenge data. For all tested DL models, on average, DeepXplore generated one test input demonstrating incorrect behavior within one second while running only on a commodity laptop. We further show that the test inputs generated by DeepXplore can also be used to retrain the corresponding DL model to improve the model’s accuracy by up to 3%.
Tasks	Malware Detection, Self-Driving Cars
Published	2017-05-18
URL	http://arxiv.org/abs/1705.06640v4
PDF	http://arxiv.org/pdf/1705.06640v4.pdf
PWC	https://paperswithcode.com/paper/deepxplore-automated-whitebox-testing-of-deep
Repo	https://github.com/peikexin9/deepxplore
Framework	tf

Semantic Specialisation of Distributional Word Vector Spaces using Monolingual and Cross-Lingual Constraints


Title	Semantic Specialisation of Distributional Word Vector Spaces using Monolingual and Cross-Lingual Constraints
Authors	Nikola Mrkšić, Ivan Vulić, Diarmuid Ó Séaghdha, Ira Leviant, Roi Reichart, Milica Gašić, Anna Korhonen, Steve Young
Abstract	We present Attract-Repel, an algorithm for improving the semantic quality of word vectors by injecting constraints extracted from lexical resources. Attract-Repel facilitates the use of constraints from mono- and cross-lingual resources, yielding semantically specialised cross-lingual vector spaces. Our evaluation shows that the method can make use of existing cross-lingual lexicons to construct high-quality vector spaces for a plethora of different languages, facilitating semantic transfer from high- to lower-resource ones. The effectiveness of our approach is demonstrated with state-of-the-art results on semantic similarity datasets in six languages. We next show that Attract-Repel-specialised vectors boost performance in the downstream task of dialogue state tracking (DST) across multiple languages. Finally, we show that cross-lingual vector spaces produced by our algorithm facilitate the training of multilingual DST models, which brings further performance improvements.
Tasks	Dialogue State Tracking, Semantic Similarity, Semantic Textual Similarity
Published	2017-06-01
URL	http://arxiv.org/abs/1706.00374v1
PDF	http://arxiv.org/pdf/1706.00374v1.pdf
PWC	https://paperswithcode.com/paper/semantic-specialisation-of-distributional
Repo	https://github.com/zliucr/mixed-language-training
Framework	pytorch

Stable and Controllable Neural Texture Synthesis and Style Transfer Using Histogram Losses


Title	Stable and Controllable Neural Texture Synthesis and Style Transfer Using Histogram Losses
Authors	Eric Risser, Pierre Wilmot, Connelly Barnes
Abstract	Recently, methods have been proposed that perform texture synthesis and style transfer by using convolutional neural networks (e.g. Gatys et al. [2015,2016]). These methods are exciting because they can in some cases create results with state-of-the-art quality. However, in this paper, we show these methods also have limitations in texture quality, stability, requisite parameter tuning, and lack of user controls. This paper presents a multiscale synthesis pipeline based on convolutional neural networks that ameliorates these issues. We first give a mathematical explanation of the source of instabilities in many previous approaches. We then improve these instabilities by using histogram losses to synthesize textures that better statistically match the exemplar. We also show how to integrate localized style losses in our multiscale framework. These losses can improve the quality of large features, improve the separation of content and style, and offer artistic controls such as paint by numbers. We demonstrate that our approach offers improved quality, convergence in fewer iterations, and more stability over the optimization.
Tasks	Style Transfer, Texture Synthesis
Published	2017-01-31
URL	http://arxiv.org/abs/1701.08893v2
PDF	http://arxiv.org/pdf/1701.08893v2.pdf
PWC	https://paperswithcode.com/paper/stable-and-controllable-neural-texture
Repo	https://github.com/Asteur/paint_harmonization
Framework	torch

Comparing deep neural networks against humans: object recognition when the signal gets weaker


Title	Comparing deep neural networks against humans: object recognition when the signal gets weaker
Authors	Robert Geirhos, David H. J. Janssen, Heiko H. Schütt, Jonas Rauber, Matthias Bethge, Felix A. Wichmann
Abstract	Human visual object recognition is typically rapid and seemingly effortless, as well as largely independent of viewpoint and object orientation. Until very recently, animate visual systems were the only ones capable of this remarkable computational feat. This has changed with the rise of a class of computer vision algorithms called deep neural networks (DNNs) that achieve human-level classification performance on object recognition tasks. Furthermore, a growing number of studies report similarities in the way DNNs and the human visual system process objects, suggesting that current DNNs may be good models of human visual object recognition. Yet there clearly exist important architectural and processing differences between state-of-the-art DNNs and the primate visual system. The potential behavioural consequences of these differences are not well understood. We aim to address this issue by comparing human and DNN generalisation abilities towards image degradations. We find the human visual system to be more robust to image manipulations like contrast reduction, additive noise or novel eidolon-distortions. In addition, we find progressively diverging classification error-patterns between humans and DNNs when the signal gets weaker, indicating that there may still be marked differences in the way humans and current DNNs perform visual object recognition. We envision that our findings as well as our carefully measured and freely available behavioural datasets provide a new useful benchmark for the computer vision community to improve the robustness of DNNs and a motivation for neuroscientists to search for mechanisms in the brain that could facilitate this robustness.
Tasks	Object Recognition
Published	2017-06-21
URL	http://arxiv.org/abs/1706.06969v2
PDF	http://arxiv.org/pdf/1706.06969v2.pdf
PWC	https://paperswithcode.com/paper/comparing-deep-neural-networks-against-humans
Repo	https://github.com/rgeirhos/object-recognition
Framework	none

Insights into Analogy Completion from the Biomedical Domain


Title	Insights into Analogy Completion from the Biomedical Domain
Authors	Denis Newman-Griffis, Albert M Lai, Eric Fosler-Lussier
Abstract	Analogy completion has been a popular task in recent years for evaluating the semantic properties of word embeddings, but the standard methodology makes a number of assumptions about analogies that do not always hold, either in recent benchmark datasets or when expanding into other domains. Through an analysis of analogies in the biomedical domain, we identify three assumptions: that of a Single Answer for any given analogy, that the pairs involved describe the Same Relationship, and that each pair is Informative with respect to the other. We propose modifying the standard methodology to relax these assumptions by allowing for multiple correct answers, reporting MAP and MRR in addition to accuracy, and using multiple example pairs. We further present BMASS, a novel dataset for evaluating linguistic regularities in biomedical embeddings, and demonstrate that the relationships described in the dataset pose significant semantic challenges to current word embedding methods.
Tasks	Word Embeddings
Published	2017-06-07
URL	http://arxiv.org/abs/1706.02241v1
PDF	http://arxiv.org/pdf/1706.02241v1.pdf
PWC	https://paperswithcode.com/paper/insights-into-analogy-completion-from-the
Repo	https://github.com/OSU-slatelab/BMASS
Framework	tf

Long-term Blood Pressure Prediction with Deep Recurrent Neural Networks


Title	Long-term Blood Pressure Prediction with Deep Recurrent Neural Networks
Authors	Peng Su, Xiao-Rong Ding, Yuan-Ting Zhang, Jing Liu, Fen Miao, Ni Zhao
Abstract	Existing methods for arterial blood pressure (BP) estimation directly map the input physiological signals to output BP values without explicitly modeling the underlying temporal dependencies in BP dynamics. As a result, these models suffer from accuracy decay over a long time and thus require frequent calibration. In this work, we address this issue by formulating BP estimation as a sequence prediction problem in which both the input and target are temporal sequences. We propose a novel deep recurrent neural network (RNN) consisting of multilayered Long Short-Term Memory (LSTM) networks, which are incorporated with (1) a bidirectional structure to access larger-scale context information of input sequence, and (2) residual connections to allow gradients in deep RNN to propagate more effectively. The proposed deep RNN model was tested on a static BP dataset, and it achieved root mean square error (RMSE) of 3.90 and 2.66 mmHg for systolic BP (SBP) and diastolic BP (DBP) prediction respectively, surpassing the accuracy of traditional BP prediction models. On a multi-day BP dataset, the deep RNN achieved RMSE of 3.84, 5.25, 5.80 and 5.81 mmHg for the 1st day, 2nd day, 4th day and 6th month after the 1st day SBP prediction, and 1.80, 4.78, 5.0, 5.21 mmHg for corresponding DBP prediction, respectively, which outperforms all previous models with notable improvement. The experimental results suggest that modeling the temporal dependencies in BP dynamics significantly improves the long-term BP prediction accuracy.
Tasks	Blood pressure estimation, Electrocardiography (ECG), Photoplethysmography (PPG)
Published	2017-05-12
URL	http://arxiv.org/abs/1705.04524v3
PDF	http://arxiv.org/pdf/1705.04524v3.pdf
PWC	https://paperswithcode.com/paper/long-term-blood-pressure-prediction-with-deep
Repo	https://github.com/psu1/DeepRNN
Framework	tf

ChainerCV: a Library for Deep Learning in Computer Vision


Title	ChainerCV: a Library for Deep Learning in Computer Vision
Authors	Yusuke Niitani, Toru Ogawa, Shunta Saito, Masaki Saito
Abstract	Despite significant progress of deep learning in the field of computer vision, there has not been a software library that covers these methods in a unifying manner. We introduce ChainerCV, a software library that is intended to fill this gap. ChainerCV supports numerous neural network models as well as software components needed to conduct research in computer vision. These implementations emphasize simplicity, flexibility and good software engineering practices. The library is designed to perform on par with the results reported in published papers and its tools can be used as a baseline for future research in computer vision. Our implementation includes sophisticated models like Faster R-CNN and SSD, and covers tasks such as object detection and semantic segmentation.
Tasks	Object Detection, Semantic Segmentation
Published	2017-08-28
URL	http://arxiv.org/abs/1708.08169v1
PDF	http://arxiv.org/pdf/1708.08169v1.pdf
PWC	https://paperswithcode.com/paper/chainercv-a-library-for-deep-learning-in
Repo	https://github.com/chainer/chainercv
Framework	none

Probabilistic RGB-D Odometry based on Points, Lines and Planes Under Depth Uncertainty


Title	Probabilistic RGB-D Odometry based on Points, Lines and Planes Under Depth Uncertainty
Authors	Pedro F. Proenca, Yang Gao
Abstract	This work proposes a robust visual odometry method for structured environments that combines point features with line and plane segments, extracted through an RGB-D camera. Noisy depth maps are processed by a probabilistic depth fusion framework based on Mixtures of Gaussians to denoise and derive the depth uncertainty, which is then propagated throughout the visual odometry pipeline. Probabilistic 3D plane and line fitting solutions are used to model the uncertainties of the feature parameters and pose is estimated by combining the three types of primitives based on their uncertainties. Performance evaluation on RGB-D sequences collected in this work and two public RGB-D datasets: TUM and ICL-NUIM show the benefit of using the proposed depth fusion framework and combining the three feature-types, particularly in scenes with low-textured surfaces, dynamic objects and missing depth measurements.
Tasks	Visual Odometry
Published	2017-06-13
URL	http://arxiv.org/abs/1706.04034v3
PDF	http://arxiv.org/pdf/1706.04034v3.pdf
PWC	https://paperswithcode.com/paper/probabilistic-rgb-d-odometry-based-on-points
Repo	https://github.com/pedropro/OMG_Depth_Fusion
Framework	none

Multi-View Data Generation Without View Supervision


Title	Multi-View Data Generation Without View Supervision
Authors	Mickaël Chen, Ludovic Denoyer, Thierry Artières
Abstract	The development of high-dimensional generative models has recently gained a great surge of interest with the introduction of variational auto-encoders and generative adversarial neural networks. Different variants have been proposed where the underlying latent space is structured, for example, based on attributes describing the data to generate. We focus on a particular problem where one aims at generating samples corresponding to a number of objects under various views. We assume that the distribution of the data is driven by two independent latent factors: the content, which represents the intrinsic features of an object, and the view, which stands for the settings of a particular observation of that object. Therefore, we propose a generative model and a conditional variant built on such a disentangled latent space. This approach allows us to generate realistic samples corresponding to various objects in a high variety of views. Unlike many multi-view approaches, our model doesn’t need any supervision on the views but only on the content. Compared to other conditional generation approaches that are mostly based on binary or categorical attributes, we make no such assumption about the factors of variations. Our model can be used on problems with a huge, potentially infinite, number of categories. We experiment it on four image datasets on which we demonstrate the effectiveness of the model and its ability to generalize.
Tasks
Published	2017-11-01
URL	http://arxiv.org/abs/1711.00305v2
PDF	http://arxiv.org/pdf/1711.00305v2.pdf
PWC	https://paperswithcode.com/paper/multi-view-data-generation-without-view
Repo	https://github.com/mickaelChen/GMV
Framework	pytorch

A Large Dimensional Study of Regularized Discriminant Analysis Classifiers


Title	A Large Dimensional Study of Regularized Discriminant Analysis Classifiers
Authors	Khalil Elkhalil, Abla Kammoun, Romain Couillet, Tareq Y. Al-Naffouri, Mohamed-Slim Alouini
Abstract	This article carries out a large dimensional analysis of standard regularized discriminant analysis classifiers designed on the assumption that data arise from a Gaussian mixture model with different means and covariances. The analysis relies on fundamental results from random matrix theory (RMT) when both the number of features and the cardinality of the training data within each class grow large at the same pace. Under mild assumptions, we show that the asymptotic classification error approaches a deterministic quantity that depends only on the means and covariances associated with each class as well as the problem dimensions. Such a result permits a better understanding of the performance of regularized discriminant analsysis, in practical large but finite dimensions, and can be used to determine and pre-estimate the optimal regularization parameter that minimizes the misclassification error probability. Despite being theoretically valid only for Gaussian data, our findings are shown to yield a high accuracy in predicting the performances achieved with real data sets drawn from the popular USPS data base, thereby making an interesting connection between theory and practice.
Tasks
Published	2017-11-01
URL	https://arxiv.org/abs/1711.00382v4
PDF	https://arxiv.org/pdf/1711.00382v4.pdf
PWC	https://paperswithcode.com/paper/a-large-dimensional-study-of-regularized
Repo	https://github.com/KhalilElkhalil/Large-Dimensional-Discriminant-Analysis-Classifiers-with-Random-Matrix-Theory
Framework	none