January 26, 2020

3046 words 15 mins read

Paper Group ANR 1440

Compressing Representations for Embedded Deep Learning. Incremental Learning of Motion Primitives for Pedestrian Trajectory Prediction at Intersections. DScribe: Library of Descriptors for Machine Learning in Materials Science. Adaptive Probabilistic Vehicle Trajectory Prediction Through Physically Feasible Bayesian Recurrent Neural Network. Harmon …

Compressing Representations for Embedded Deep Learning


Title	Compressing Representations for Embedded Deep Learning
Authors	Juliano S. Assine, Alan Godoy, Eduardo Valle
Abstract	Despite recent advances in architectures for mobile devices, deep learning computational requirements remains prohibitive for most embedded devices. To address that issue, we envision sharing the computational costs of inference between local devices and the cloud, taking advantage of the compression performed by the first layers of the networks to reduce communication costs. Inference in such distributed setting would allow new applications, but requires balancing a triple trade-off between computation cost, communication bandwidth, and model accuracy. We explore that trade-off by studying the compressibility of representations at different stages of MobileNetV2, showing those results agree with theoretical intuitions about deep learning, and that an optimal splitting layer for network can be found with a simple PCA-based compression scheme.
Tasks
Published	2019-11-23
URL	https://arxiv.org/abs/1911.10321v1
PDF	https://arxiv.org/pdf/1911.10321v1.pdf
PWC	https://paperswithcode.com/paper/compressing-representations-for-embedded-deep
Repo
Framework

Incremental Learning of Motion Primitives for Pedestrian Trajectory Prediction at Intersections


Title	Incremental Learning of Motion Primitives for Pedestrian Trajectory Prediction at Intersections
Authors	Golnaz Habibi, Nikita Japuria, Jonathan P. How
Abstract	This paper presents a novel incremental learning algorithm for pedestrian motion prediction, with the ability to improve the learned model over time when data is incrementally available. In this setup, trajectories are modeled as simple segments called motion primitives. Transitions between motion primitives are modeled as Gaussian Processes. When new data is available, the motion primitives learned from the new data are compared with the previous ones by measuring the inner product of the motion primitive vectors. Similar motion primitives and transitions are fused and novel motion primitives are added to capture newly observed behaviors. The proposed approach is tested and compared with other baselines in intersection scenarios where the data is incrementally available either from a single intersection or from multiple intersections with different geometries. In both cases, our method incrementally learns motion patterns and outperforms the offline learning approach in terms of prediction errors. The results also show that the model size in our algorithm grows at a much lower rate than standard incremental learning, where newly learned motion primitives and transitions are simply accumulated over time.
Tasks	Gaussian Processes, motion prediction, Trajectory Prediction
Published	2019-11-21
URL	https://arxiv.org/abs/1911.09476v1
PDF	https://arxiv.org/pdf/1911.09476v1.pdf
PWC	https://paperswithcode.com/paper/incremental-learning-of-motion-primitives-for
Repo
Framework

DScribe: Library of Descriptors for Machine Learning in Materials Science


Title	DScribe: Library of Descriptors for Machine Learning in Materials Science
Authors	Lauri Himanen, Marc O. J. Jäger, Eiaki V. Morooka, Filippo Federici Canova, Yashasvi S. Ranawat, David Z. Gao, Patrick Rinke, Adam S. Foster
Abstract	DScribe is a software package for machine learning that provides popular feature transformations (“descriptors”) for atomistic materials simulations. DScribe accelerates the application of machine learning for atomistic property prediction by providing user-friendly, off-the-shelf descriptor implementations. The package currently contains implementations for Coulomb matrix, Ewald sum matrix, sine matrix, Many-body Tensor Representation (MBTR), Atom-centered Symmetry Function (ACSF) and Smooth Overlap of Atomic Positions (SOAP). Usage of the package is illustrated for two different applications: formation energy prediction for solids and ionic charge prediction for atoms in organic molecules. The package is freely available under the open-source Apache License 2.0.
Tasks	Formation Energy
Published	2019-04-18
URL	http://arxiv.org/abs/1904.08875v1
PDF	http://arxiv.org/pdf/1904.08875v1.pdf
PWC	https://paperswithcode.com/paper/dscribe-library-of-descriptors-for-machine
Repo
Framework

Adaptive Probabilistic Vehicle Trajectory Prediction Through Physically Feasible Bayesian Recurrent Neural Network


Title	Adaptive Probabilistic Vehicle Trajectory Prediction Through Physically Feasible Bayesian Recurrent Neural Network
Authors	Chen Tang, Jianyu Chen, Masayoshi Tomizuka
Abstract	Probabilistic vehicle trajectory prediction is essential for robust safety of autonomous driving. Current methods for long-term trajectory prediction cannot guarantee the physical feasibility of predicted distribution. Moreover, their models cannot adapt to the driving policy of the predicted target human driver. In this work, we propose to overcome these two shortcomings by a Bayesian recurrent neural network model consisting of Bayesian-neural-network-based policy model and known physical model of the scenario. Bayesian neural network can ensemble complicated output distribution, enabling rich family of trajectory distribution. The embedded physical model ensures feasibility of the distribution. Moreover, the adopted gradient-based training method allows direct optimization for better performance in long prediction horizon. Furthermore, a particle-filter-based parameter adaptation algorithm is designed to adapt the policy Bayesian neural network to the predicted target online. Effectiveness of the proposed methods is verified with a toy example with multi-modal stochastic feedback gain and naturalistic car following data.
Tasks	Autonomous Driving, Trajectory Prediction
Published	2019-11-11
URL	https://arxiv.org/abs/1911.04597v1
PDF	https://arxiv.org/pdf/1911.04597v1.pdf
PWC	https://paperswithcode.com/paper/adaptive-probabilistic-vehicle-trajectory
Repo
Framework

Harmonic Mean Point Processes: Proportional Rate Error Minimization for Obtundation Prediction


Title	Harmonic Mean Point Processes: Proportional Rate Error Minimization for Obtundation Prediction
Authors	Yoonjung Kim, Jeremy C. Weiss
Abstract	In healthcare, the highest risk individuals for morbidity and mortality are rarely those with the greatest modifiable risk. By contrast, many machine learning formulations implicitly attend to the highest risk individuals. We focus on this problem in point processes, a popular modeling technique for the analysis of the temporal event sequences in electronic health records (EHR) data with applications in risk stratification and risk score systems. We show that optimization of the log-likelihood function also gives disproportionate attention to high risk individuals and leads to poor prediction results for low risk individuals compared to ones at high risk. We characterize the problem and propose an adjusted log-likelihood formulation as a new objective for point processes. We demonstrate the benefits of our method in simulations and in EHR data of patients admitted to the critical care unit for intracerebral hemorrhage.
Tasks	Point Processes
Published	2019-11-12
URL	https://arxiv.org/abs/1911.05109v2
PDF	https://arxiv.org/pdf/1911.05109v2.pdf
PWC	https://paperswithcode.com/paper/harmonic-mean-point-processes-proportional
Repo
Framework

swTVM: Exploring the Automated Compilation for Deep Learning on Sunway Architecture


Title	swTVM: Exploring the Automated Compilation for Deep Learning on Sunway Architecture
Authors	Changxi Liu, Hailong Yang, Rujun Sun, Zhongzhi Luan, Lin Gan, Guangwen Yang, Depei Qian
Abstract	The flourish of deep learning frameworks and hardware platforms has been demanding an efficient compiler that can shield the diversity in both software and hardware in order to provide application portability. Among the exiting deep learning compilers, TVM is well known for its efficiency in code generation and optimization across diverse hardware devices. In the meanwhile, the Sunway many-core processor renders itself as a competitive candidate for its attractive computational power in both scientific and deep learning applications. This paper combines the trends in these two directions. Specifically, we propose swTVM that extends the original TVM to support ahead-of-time compilation for architecture requiring cross-compilation such as Sunway. In addition, we leverage the architecture features during the compilation such as core group for massive parallelism, DMA for high bandwidth memory transfer and local device memory for data locality, in order to generate efficient code for deep learning application on Sunway. The experimental results show the ability of swTVM to automatically generate code for various deep neural network models on Sunway. The performance of automatically generated code for AlexNet and VGG-19 by swTVM achieves 6.71x and 2.45x speedup on average than hand-optimized OpenACC implementations on convolution and fully connected layers respectively. This work is the first attempt from the compiler perspective to bridge the gap of deep learning and high performance architecture particularly with productivity and efficiency in mind. We would like to open source the implementation so that more people can embrace the power of deep learning compiler and Sunway many-core processor.
Tasks	Code Generation
Published	2019-04-16
URL	http://arxiv.org/abs/1904.07404v2
PDF	http://arxiv.org/pdf/1904.07404v2.pdf
PWC	https://paperswithcode.com/paper/swtvm-exploring-the-automated-compilation-for
Repo
Framework

Human Driver Behavior Prediction based on UrbanFlow


Title	Human Driver Behavior Prediction based on UrbanFlow
Authors	Zhiqian Qiao, Jing Zhao, Zachariah Tyree, Priyantha Mudalige, Jeff Schneider, John M. Dolan
Abstract	How autonomous vehicles and human drivers share public transportation systems is an important problem, as fully automatic transportation environments are still a long way off. Understanding human drivers’ behavior can be beneficial for autonomous vehicle decision making and planning, especially when the autonomous vehicle is surrounded by human drivers who have various driving behaviors and patterns of interaction with other vehicles. In this paper, we propose an LSTM-based trajectory prediction method for human drivers which can help the autonomous vehicle make better decisions, especially in urban intersection scenarios. Meanwhile, in order to collect human drivers’ driving behavior data in the urban scenario, we describe a system called UrbanFlow which includes the whole procedure from raw bird’s-eye view data collection via drone to the final processed trajectories. The system is mainly intended for urban scenarios but can be extended to be used for any traffic scenarios.
Tasks	Autonomous Vehicles, Decision Making, Trajectory Prediction
Published	2019-11-09
URL	https://arxiv.org/abs/1911.03801v1
PDF	https://arxiv.org/pdf/1911.03801v1.pdf
PWC	https://paperswithcode.com/paper/human-driver-behavior-prediction-based-on
Repo
Framework

Personality-Aware Probabilistic Map for Trajectory Prediction of Pedestrians


Title	Personality-Aware Probabilistic Map for Trajectory Prediction of Pedestrians
Authors	Chaochao Li, Pei Lv, Mingliang Xu, Xinyu Wang, Dinesh Manocha, Bing Zhou, Meng Wang
Abstract	We present a novel trajectory prediction algorithm for pedestrians based on a personality-aware probabilistic feature map. This map is computed using a spatial query structure and each value represents the probability of the predicted pedestrian passing through various positions in the crowd space. We update this map dynamically based on the agents in the environment and prior trajectory of a pedestrian. Furthermore, we estimate the personality characteristics of each pedestrian and use them to improve the prediction by estimating the shortest path in this map. Our approach is general and works well on crowd videos with low and high pedestrian density. We evaluate our model on standard human-trajectory datasets. In practice, our prediction algorithm improves the accuracy by 5-9% over prior algorithms.
Tasks	Trajectory Prediction
Published	2019-11-01
URL	https://arxiv.org/abs/1911.00193v1
PDF	https://arxiv.org/pdf/1911.00193v1.pdf
PWC	https://paperswithcode.com/paper/personality-aware-probabilistic-map-for
Repo
Framework

Testing the robustness of attribution methods for convolutional neural networks in MRI-based Alzheimer’s disease classification


Title	Testing the robustness of attribution methods for convolutional neural networks in MRI-based Alzheimer’s disease classification
Authors	Fabian Eitel, Kerstin Ritter
Abstract	Attribution methods are an easy to use tool for investigating and validating machine learning models. Multiple methods have been suggested in the literature and it is not yet clear which method is most suitable for a given task. In this study, we tested the robustness of four attribution methods, namely gradient*input, guided backpropagation, layer-wise relevance propagation and occlusion, for the task of Alzheimer’s disease classification. We have repeatedly trained a convolutional neural network (CNN) with identical training settings in order to separate structural MRI data of patients with Alzheimer’s disease and healthy controls. Afterwards, we produced attribution maps for each subject in the test data and quantitatively compared them across models and attribution methods. We show that visual comparison is not sufficient and that some widely used attribution methods produce highly inconsistent outcomes.
Tasks
Published	2019-09-19
URL	https://arxiv.org/abs/1909.08856v1
PDF	https://arxiv.org/pdf/1909.08856v1.pdf
PWC	https://paperswithcode.com/paper/testing-the-robustness-of-attribution-methods
Repo
Framework

Versatile linkage: a family of space-conserving strategies for agglomerative hierarchical clustering


Title	Versatile linkage: a family of space-conserving strategies for agglomerative hierarchical clustering
Authors	Alberto Fernández, Sergio Gómez
Abstract	Agglomerative hierarchical clustering can be implemented with several strategies that differ in the way elements of a collection are grouped together to build a hierarchy of clusters. Here we introduce versatile linkage, a new infinite system of agglomerative hierarchical clustering strategies based on generalized means, which go from single linkage to complete linkage, passing through arithmetic average linkage and other clustering methods yet unexplored such as geometric linkage and harmonic linkage. We compare the different clustering strategies in terms of cophenetic correlation, mean absolute error, and also tree balance and space distortion, two new measures proposed to describe hierarchical trees. Unlike the $\beta$-flexible clustering system, we show that the versatile linkage family is space-conserving.
Tasks
Published	2019-06-21
URL	https://arxiv.org/abs/1906.09222v1
PDF	https://arxiv.org/pdf/1906.09222v1.pdf
PWC	https://paperswithcode.com/paper/versatile-linkage-a-family-of-space
Repo
Framework

Probabilistic Trajectory Prediction for Autonomous Vehicles with Attentive Recurrent Neural Process


Title	Probabilistic Trajectory Prediction for Autonomous Vehicles with Attentive Recurrent Neural Process
Authors	Jiacheng Zhu, Shenghao Qin, Wenshuo Wang, Ding Zhao
Abstract	Predicting surrounding vehicle behaviors are critical to autonomous vehicles when negotiating in multi-vehicle interaction scenarios. Most existing approaches require tedious training process with large amounts of data and may fail to capture the propagating uncertainty in interaction behaviors. The multi-vehicle behaviors are assumed to be generated from a stochastic process. This paper proposes an attentive recurrent neural process (ARNP) approach to overcome the above limitations, which uses a neural process (NP) to learn a distribution of multi-vehicle interaction behavior. Our proposed model inherits the flexibility of neural networks while maintaining Bayesian probabilistic characteristics. Constructed by incorporating NPs with recurrent neural networks (RNNs), the ARNP model predicts the distribution of a target vehicle trajectory conditioned on the observed long-term sequential data of all surrounding vehicles. This approach is verified by learning and predicting lane-changing trajectories in complex traffic scenarios. Experimental results demonstrate that our proposed method outperforms previous counterparts in terms of accuracy and uncertainty expressiveness. Moreover, the meta-learning instinct of NPs enables our proposed ARNP model to capture global information of all observations, thereby being able to adapt to new targets efficiently.
Tasks	Autonomous Vehicles, Meta-Learning, Trajectory Prediction
Published	2019-10-17
URL	https://arxiv.org/abs/1910.08102v1
PDF	https://arxiv.org/pdf/1910.08102v1.pdf
PWC	https://paperswithcode.com/paper/probabilistic-trajectory-prediction-for
Repo
Framework

More chemical detection through less sampling: amplifying chemical signals in hyperspectral data cubes through compressive sensing


Title	More chemical detection through less sampling: amplifying chemical signals in hyperspectral data cubes through compressive sensing
Authors	Henry Kvinge, Elin Farnell, Julia R. Dupuis, Michael Kirby, Chris Peterson, Elizabeth C. Schundler
Abstract	Compressive sensing (CS) is a method of sampling which permits some classes of signals to be reconstructed with high accuracy even when they were under-sampled. In this paper we explore a phenomenon in which bandwise CS sampling of a hyperspectral data cube followed by reconstruction can actually result in amplification of chemical signals contained in the cube. Perhaps most surprisingly, chemical signal amplification generally seems to increase as the level of sampling decreases. In some examples, the chemical signal is significantly stronger in a data cube reconstructed from 10% CS sampling than it is in the raw, 100% sampled data cube. We explore this phenomenon in two real-world datasets including the Physical Sciences Inc. Fabry-P'{e}rot interferometer sensor multispectral dataset and the Johns Hopkins Applied Physics Lab FTIR-based longwave infrared sensor hyperspectral dataset. Each of these datasets contains the release of a chemical simulant, such as glacial acetic acid, triethyl phospate, and sulfur hexafluoride, and in all cases we use the adaptive coherence estimator (ACE) to detect a target signal in the hyperspectral data cube. We end the paper by suggesting some theoretical justifications for why chemical signals would be amplified in CS sampled and reconstructed hyperspectral data cubes and discuss some practical implications.
Tasks	Compressive Sensing
Published	2019-06-27
URL	https://arxiv.org/abs/1906.11818v1
PDF	https://arxiv.org/pdf/1906.11818v1.pdf
PWC	https://paperswithcode.com/paper/more-chemical-detection-through-less-sampling
Repo
Framework

Low Precision Constant Parameter CNN on FPGA


Title	Low Precision Constant Parameter CNN on FPGA
Authors	Thiam Khean Hah, Yeong Tat Liew, Jason Ong
Abstract	We report FPGA implementation results of low precision CNN convolution layers optimized for sparse and constant parameters. We describe techniques that amortizes the cost of common factor multiplication and automatically leverage dense hand tuned LUT structures. We apply this method to corner case residual blocks of Resnet on a sparse Resnet50 model to assess achievable utilization and frequency and demonstrate an effective performance of 131 and 23 TOP/chip for the corner case blocks. The projected performance on a multichip persistent implementation of all Resnet50 convolution layers is 10k im/s/chip at batch size 2. This is 1.37x higher than V100 GPU upper bound at the same batch size after normalizing for sparsity.
Tasks
Published	2019-01-11
URL	http://arxiv.org/abs/1901.04969v1
PDF	http://arxiv.org/pdf/1901.04969v1.pdf
PWC	https://paperswithcode.com/paper/low-precision-constant-parameter-cnn-on-fpga
Repo
Framework

Learning to Cope with Adversarial Attacks


Title	Learning to Cope with Adversarial Attacks
Authors	Xian Yeow Lee, Aaron Havens, Girish Chowdhary, Soumik Sarkar
Abstract	The security of Deep Reinforcement Learning (Deep RL) algorithms deployed in real life applications are of a primary concern. In particular, the robustness of RL agents in cyber-physical systems against adversarial attacks are especially vital since the cost of a malevolent intrusions can be extremely high. Studies have shown Deep Neural Networks (DNN), which forms the core decision-making unit in most modern RL algorithms, are easily subjected to adversarial attacks. Hence, it is imperative that RL agents deployed in real-life applications have the capability to detect and mitigate adversarial attacks in an online fashion. An example of such a framework is the Meta-Learned Advantage Hierarchy (MLAH) agent that utilizes a meta-learning framework to learn policies robustly online. Since the mechanism of this framework are still not fully explored, we conducted multiple experiments to better understand the framework’s capabilities and limitations. Our results shows that the MLAH agent exhibits interesting coping behaviors when subjected to different adversarial attacks to maintain a nominal reward. Additionally, the framework exhibits a hierarchical coping capability, based on the adaptability of the Master policy and sub-policies themselves. From empirical results, we also observed that as the interval of adversarial attacks increase, the MLAH agent can maintain a higher distribution of rewards, though at the cost of higher instabilities.
Tasks	Decision Making, Meta-Learning
Published	2019-06-28
URL	https://arxiv.org/abs/1906.12061v1
PDF	https://arxiv.org/pdf/1906.12061v1.pdf
PWC	https://paperswithcode.com/paper/learning-to-cope-with-adversarial-attacks
Repo
Framework

Privacy-Preserving Machine Learning Using EtC Images


Title	Privacy-Preserving Machine Learning Using EtC Images
Authors	Ayana Kawamura, Yuma Kinoshita, Hitoshi Kiya
Abstract	In this paper, we propose a novel privacy-preserving machine learning scheme with encrypted images, called EtC (Encryption-then-Compression) images. Using machine learning algorithms in cloud environments has been spreading in many fields. However, there are serious issues with it for end users, due to semi-trusted cloud providers. Accordingly, we propose using EtC images, which have been proposed for EtC systems with JPEG compression. In this paper, a novel property of EtC images is considered under the use of z-score normalization. It is demonstrated that the use of EtC images allows us not only to protect visual information of images, but also to preserve both the Euclidean distance and the inner product between vectors. In addition, dimensionality reduction is shown to can be applied to EtC images for fast and accurate matching. In an experiment, the proposed scheme is applied to a facial recognition algorithm with classifiers for confirming the effectiveness of the scheme under the use of support vector machine (SVM) with the kernel trick.
Tasks	Dimensionality Reduction
Published	2019-11-01
URL	https://arxiv.org/abs/1911.00227v1
PDF	https://arxiv.org/pdf/1911.00227v1.pdf
PWC	https://paperswithcode.com/paper/privacy-preserving-machine-learning-using-etc
Repo
Framework