October 21, 2019

3343 words 16 mins read

Paper Group AWR 162

A simple yet effective baseline for non-attributed graph classification. Accelerating Neural Transformer via an Average Attention Network. Vehicle Pose and Shape Estimation through Multiple Monocular Vision. VFPred: A Fusion of Signal Processing and Machine Learning techniques in Detecting Ventricular Fibrillation from ECG Signals. How Powerful are …

A simple yet effective baseline for non-attributed graph classification


Title	A simple yet effective baseline for non-attributed graph classification
Authors	Chen Cai, Yusu Wang
Abstract	Graphs are complex objects that do not lend themselves easily to typical learning tasks. Recently, a range of approaches based on graph kernels or graph neural networks have been developed for graph classification and for representation learning on graphs in general. As the developed methodologies become more sophisticated, it is important to understand which components of the increasingly complex methods are necessary or most effective. As a first step, we develop a simple yet meaningful graph representation, and explore its effectiveness in graph classification. We test our baseline representation for the graph classification task on a range of graph datasets. Interestingly, this simple representation achieves similar performance as the state-of-the-art graph kernels and graph neural networks for non-attributed graph classification. Its performance on classifying attributed graphs is slightly weaker as it does not incorporate attributes. However, given its simplicity and efficiency, we believe that it still serves as an effective baseline for attributed graph classification. Our graph representation is efficient (linear-time) to compute. We also provide a simple connection with the graph neural networks. Note that these observations are only for the task of graph classification while existing methods are often designed for a broader scope including node embedding and link prediction. The results are also likely biased due to the limited amount of benchmark datasets available. Nevertheless, the good performance of our simple baseline calls for the development of new, more comprehensive benchmark datasets so as to better evaluate and analyze different graph learning methods. Furthermore, given the computational efficiency of our graph summary, we believe that it is a good candidate as a baseline method for future graph classification (or even other graph learning) studies.
Tasks	Graph Classification, Link Prediction, Representation Learning
Published	2018-11-08
URL	https://arxiv.org/abs/1811.03508v2
PDF	https://arxiv.org/pdf/1811.03508v2.pdf
PWC	https://paperswithcode.com/paper/a-simple-yet-effective-baseline-for-non
Repo	https://github.com/Chen-Cai-OSU/LDP
Framework	none

Accelerating Neural Transformer via an Average Attention Network


Title	Accelerating Neural Transformer via an Average Attention Network
Authors	Biao Zhang, Deyi Xiong, Jinsong Su
Abstract	With parallelizable attention networks, the neural Transformer is very fast to train. However, due to the auto-regressive architecture and self-attention in the decoder, the decoding procedure becomes slow. To alleviate this issue, we propose an average attention network as an alternative to the self-attention network in the decoder of the neural Transformer. The average attention network consists of two layers, with an average layer that models dependencies on previous positions and a gating layer that is stacked over the average layer to enhance the expressiveness of the proposed attention network. We apply this network on the decoder part of the neural Transformer to replace the original target-side self-attention model. With masking tricks and dynamic programming, our model enables the neural Transformer to decode sentences over four times faster than its original version with almost no loss in training time and translation performance. We conduct a series of experiments on WMT17 translation tasks, where on 6 different language pairs, we obtain robust and consistent speed-ups in decoding.
Tasks
Published	2018-05-02
URL	http://arxiv.org/abs/1805.00631v3
PDF	http://arxiv.org/pdf/1805.00631v3.pdf
PWC	https://paperswithcode.com/paper/accelerating-neural-transformer-via-an
Repo	https://github.com/anoidgit/transformer
Framework	pytorch

Vehicle Pose and Shape Estimation through Multiple Monocular Vision


Title	Vehicle Pose and Shape Estimation through Multiple Monocular Vision
Authors	Wenhao Ding, Shuaijun Li, Guilin Zhang, Xiangyu Lei, Huihuan Qian
Abstract	In this paper, we present an accurate approach to estimate vehicles’ pose and shape from off-board multiview images. The images are taken by monocular cameras and have small overlaps. We utilize state-of-the-art convolutional neural networks (CNNs) to extract vehicles’ semantic keypoints and introduce a Cross Projection Optimization (CPO) method to estimate the 3D pose. During the iterative CPO process, an adaptive shape adjustment method named Hierarchical Wireframe Constraint (HWC) is implemented to estimate the shape. Our approach is evaluated under both simulated and real-world scenes for performance verification. It’s shown that our algorithm outperforms other existing monocular and stereo methods for vehicles’ pose and shape estimation. This approach provides a new and robust solution for off-board visual vehicle localization and tracking, which can be applied to massive surveillance camera networks for intelligent transportation.
Tasks
Published	2018-02-10
URL	http://arxiv.org/abs/1802.03515v5
PDF	http://arxiv.org/pdf/1802.03515v5.pdf
PWC	https://paperswithcode.com/paper/vehicle-pose-and-shape-estimation-through
Repo	https://github.com/GilgameshD/Multiple-View-Car-Localization
Framework	none

VFPred: A Fusion of Signal Processing and Machine Learning techniques in Detecting Ventricular Fibrillation from ECG Signals


Title	VFPred: A Fusion of Signal Processing and Machine Learning techniques in Detecting Ventricular Fibrillation from ECG Signals
Authors	Nabil Ibtehaz, M. Saifur Rahman, M. Sohel Rahman
Abstract	Ventricular Fibrillation (VF), one of the most dangerous arrhythmias, is responsible for sudden cardiac arrests. Thus, various algorithms have been developed to predict VF from Electrocardiogram (ECG), which is a binary classification problem. In the literature, we find a number of algorithms based on signal processing, where, after some robust mathematical operations the decision is given based on a predefined threshold over a single value. On the other hand, some machine learning based algorithms are also reported in the literature; however, these algorithms merely combine some parameters and make a prediction using those as features. Both the approaches have their perks and pitfalls; thus our motivation was to coalesce them to get the best out of the both worlds. Hence we have developed, VFPred that, in addition to employing a signal processing pipeline, namely, Empirical Mode Decomposition and Discrete Time Fourier Transform for useful feature extraction, uses a Support Vector Machine for efficient classification. VFPred turns out to be a robust algorithm as it is able to successfully segregate the two classes with equal confidence (Sensitivity = 99.99%, Specificity = 98.40%) even from a short signal of 5 seconds long, whereas existing works though requires longer signals, flourishes in one but fails in the other.
Tasks	Arrhythmia Detection, Electrocardiography (ECG), Ventricular fibrillation detection
Published	2018-07-07
URL	http://arxiv.org/abs/1807.02684v3
PDF	http://arxiv.org/pdf/1807.02684v3.pdf
PWC	https://paperswithcode.com/paper/vfpred-a-fusion-of-signal-processing-and
Repo	https://github.com/robin-0/VFPred
Framework	none

How Powerful are Graph Neural Networks?


Title	How Powerful are Graph Neural Networks?
Authors	Keyulu Xu, Weihua Hu, Jure Leskovec, Stefanie Jegelka
Abstract	Graph Neural Networks (GNNs) are an effective framework for representation learning of graphs. GNNs follow a neighborhood aggregation scheme, where the representation vector of a node is computed by recursively aggregating and transforming representation vectors of its neighboring nodes. Many GNN variants have been proposed and have achieved state-of-the-art results on both node and graph classification tasks. However, despite GNNs revolutionizing graph representation learning, there is limited understanding of their representational properties and limitations. Here, we present a theoretical framework for analyzing the expressive power of GNNs to capture different graph structures. Our results characterize the discriminative power of popular GNN variants, such as Graph Convolutional Networks and GraphSAGE, and show that they cannot learn to distinguish certain simple graph structures. We then develop a simple architecture that is provably the most expressive among the class of GNNs and is as powerful as the Weisfeiler-Lehman graph isomorphism test. We empirically validate our theoretical findings on a number of graph classification benchmarks, and demonstrate that our model achieves state-of-the-art performance.
Tasks	Graph Classification, Graph Representation Learning, Representation Learning
Published	2018-10-01
URL	http://arxiv.org/abs/1810.00826v3
PDF	http://arxiv.org/pdf/1810.00826v3.pdf
PWC	https://paperswithcode.com/paper/how-powerful-are-graph-neural-networks
Repo	https://github.com/pfnet-research/treewidth-prediction
Framework	none

Implementation of Stochastic Quasi-Newton’s Method in PyTorch


Title	Implementation of Stochastic Quasi-Newton’s Method in PyTorch
Authors	Yingkai Li, Huidong Liu
Abstract	In this paper, we implement the Stochastic Damped LBFGS (SdLBFGS) for stochastic non-convex optimization. We make two important modifications to the original SdLBFGS algorithm. First, by initializing the Hessian at each step using an identity matrix, the algorithm converges better than original algorithm. Second, by performing direction normalization we could gain stable optimization procedure without line search. Experiments on minimizing a 2D non-convex function shows that our improved algorithm converges better than original algorithm, and experiments on the CIFAR10 and MNIST datasets show that our improved algorithm works stably and gives comparable or even better testing accuracies than first order optimizers SGD, Adagrad, and second order optimizers LBFGS in PyTorch.
Tasks
Published	2018-05-07
URL	http://arxiv.org/abs/1805.02338v1
PDF	http://arxiv.org/pdf/1805.02338v1.pdf
PWC	https://paperswithcode.com/paper/implementation-of-stochastic-quasi-newtons
Repo	https://github.com/harryliew/SdLBFGS
Framework	pytorch

Efficient Neural Network Compression


Title	Efficient Neural Network Compression
Authors	Hyeji Kim, Muhammad Umar Karim Khan, Chong-Min Kyung
Abstract	Network compression reduces the computational complexity and memory consumption of deep neural networks by reducing the number of parameters. In SVD-based network compression, the right rank needs to be decided for every layer of the network. In this paper, we propose an efficient method for obtaining the rank configuration of the whole network. Unlike previous methods which consider each layer separately, our method considers the whole network to choose the right rank configuration. We propose novel accuracy metrics to represent the accuracy and complexity relationship for a given neural network. We use these metrics in a non-iterative fashion to obtain the right rank configuration which satisfies the constraints on FLOPs and memory while maintaining sufficient accuracy. Experiments show that our method provides better compromise between accuracy and computational complexity/memory consumption while performing compression at much higher speed. For VGG-16 our network can reduce the FLOPs by 25% and improve accuracy by 0.7% compared to the baseline, while requiring only 3 minutes on a CPU to search for the right rank configuration. Previously, similar results were achieved in 4 hours with 8 GPUs. The proposed method can be used for lossless compression of a neural network as well. The better accuracy and complexity compromise, as well as the extremely fast speed of our method makes it suitable for neural network compression.
Tasks	Accuracy Metrics, Neural Network Compression
Published	2018-11-30
URL	http://arxiv.org/abs/1811.12781v3
PDF	http://arxiv.org/pdf/1811.12781v3.pdf
PWC	https://paperswithcode.com/paper/a-framework-for-fast-and-efficient-neural
Repo	https://github.com/Hyeji-Kim/ENC
Framework	caffe2

Fast Incremental von Neumann Graph Entropy Computation: Theory, Algorithm, and Applications


Title	Fast Incremental von Neumann Graph Entropy Computation: Theory, Algorithm, and Applications
Authors	Pin-Yu Chen, Lingfei Wu, Sijia Liu, Indika Rajapakse
Abstract	The von Neumann graph entropy (VNGE) facilitates measurement of information divergence and distance between graphs in a graph sequence. It has been successfully applied to various learning tasks driven by network-based data. While effective, VNGE is computationally demanding as it requires the full eigenspectrum of the graph Laplacian matrix. In this paper, we propose a new computational framework, Fast Incremental von Neumann Graph EntRopy (FINGER), which approaches VNGE with a performance guarantee. FINGER reduces the cubic complexity of VNGE to linear complexity in the number of nodes and edges, and thus enables online computation based on incremental graph changes. We also show asymptotic equivalence of FINGER to the exact VNGE, and derive its approximation error bounds. Based on FINGER, we propose efficient algorithms for computing Jensen-Shannon distance between graphs. Our experimental results on different random graph models demonstrate the computational efficiency and the asymptotic equivalence of FINGER. In addition, we apply FINGER to two real-world applications and one synthesized anomaly detection dataset, and corroborate its superior performance over seven baseline graph similarity methods.
Tasks	Anomaly Detection, Graph Similarity
Published	2018-05-30
URL	https://arxiv.org/abs/1805.11769v2
PDF	https://arxiv.org/pdf/1805.11769v2.pdf
PWC	https://paperswithcode.com/paper/fast-incremental-von-neumann-graph-entropy
Repo	https://github.com/pinyuchen/FINGER
Framework	none

PydMobileNet: Improved Version of MobileNets with Pyramid Depthwise Separable Convolution


Title	PydMobileNet: Improved Version of MobileNets with Pyramid Depthwise Separable Convolution
Authors	Van-Thanh Hoang, Kang-Hyun Jo
Abstract	Convolutional neural networks (CNNs) have shown remarkable performance in various computer vision tasks in recent years. However, the increasing model size has raised challenges in adopting them in real-time applications as well as mobile and embedded vision applications. Many works try to build networks as small as possible while still have acceptable performance. The state-of-the-art architecture is MobileNets. They use Depthwise Separable Convolution (DWConvolution) in place of standard Convolution to reduce the size of networks. This paper describes an improved version of MobileNet, called Pyramid Mobile Network. Instead of using just a $3\times 3$ kernel size for DWConvolution like in MobileNet, the proposed network uses a pyramid kernel size to capture more spatial information. The proposed architecture is evaluated on two highly competitive object recognition benchmark datasets (CIFAR-10, CIFAR-100). The experiments demonstrate that the proposed network achieves better performance compared with MobileNet as well as other state-of-the-art networks. Additionally, it is more flexible in fine-tuning the trade-off between accuracy, latency and model size than MobileNets.
Tasks	Object Recognition
Published	2018-11-17
URL	http://arxiv.org/abs/1811.07083v1
PDF	http://arxiv.org/pdf/1811.07083v1.pdf
PWC	https://paperswithcode.com/paper/pydmobilenet-improved-version-of-mobilenets
Repo	https://github.com/thanhhvnqb/pydmobilenet_mxnet
Framework	mxnet

Deep Neural Decision Trees


Title	Deep Neural Decision Trees
Authors	Yongxin Yang, Irene Garcia Morillo, Timothy M. Hospedales
Abstract	Deep neural networks have been proven powerful at processing perceptual data, such as images and audio. However for tabular data, tree-based models are more popular. A nice property of tree-based models is their natural interpretability. In this work, we present Deep Neural Decision Trees (DNDT) – tree models realised by neural networks. A DNDT is intrinsically interpretable, as it is a tree. Yet as it is also a neural network (NN), it can be easily implemented in NN toolkits, and trained with gradient descent rather than greedy splitting. We evaluate DNDT on several tabular datasets, verify its efficacy, and investigate similarities and differences between DNDT and vanilla decision trees. Interestingly, DNDT self-prunes at both split and feature-level.
Tasks
Published	2018-06-19
URL	http://arxiv.org/abs/1806.06988v1
PDF	http://arxiv.org/pdf/1806.06988v1.pdf
PWC	https://paperswithcode.com/paper/deep-neural-decision-trees
Repo	https://github.com/wOOL/DNDT
Framework	tf

EnKCF: Ensemble of Kernelized Correlation Filters for High-Speed Object Tracking


Title	EnKCF: Ensemble of Kernelized Correlation Filters for High-Speed Object Tracking
Authors	Burak Uzkent, YoungWoo Seo
Abstract	Computer vision technologies are very attractive for practical applications running on embedded systems. For such an application, it is desirable for the deployed algorithms to run in high-speed and require no offline training. To develop a single-target tracking algorithm with these properties, we propose an ensemble of the kernelized correlation filters (KCF), we call it EnKCF. A committee of KCFs is specifically designed to address the variations in scale and translation of moving objects. To guarantee a high-speed run-time performance, we deploy each of KCFs in turn, instead of applying multiple KCFs to each frame. To minimize any potential drifts between individual KCFs transition, we developed a particle filter. Experimental results showed that the performance of ours is, on average, 70.10% for precision at 20 pixels, 53.00% for success rate for the OTB100 data, and 54.50% and 40.2% for the UAV123 data. Experimental results showed that our method is better than other high-speed trackers over 5% on precision on 20 pixels and 10-20% on AUC on average. Moreover, our implementation ran at 340 fps for the OTB100 and at 416 fps for the UAV123 dataset that is faster than DCF (292 fps) for the OTB100 and KCF (292 fps) for the UAV123. To increase flexibility of the proposed EnKCF running on various platforms, we also explored different levels of deep convolutional features.
Tasks	Object Tracking
Published	2018-01-20
URL	http://arxiv.org/abs/1801.06729v1
PDF	http://arxiv.org/pdf/1801.06729v1.pdf
PWC	https://paperswithcode.com/paper/enkcf-ensemble-of-kernelized-correlation
Repo	https://github.com/buzkent86/EnKCF_Tracker
Framework	none

EarthMapper: A Tool Box for the Semantic Segmentation of Remote Sensing Imagery


Title	EarthMapper: A Tool Box for the Semantic Segmentation of Remote Sensing Imagery
Authors	Ronald Kemker, Utsav B. Gewali, Christopher Kanan
Abstract	Deep learning continues to push state-of-the-art performance for the semantic segmentation of color (i.e., RGB) imagery; however, the lack of annotated data for many remote sensing sensors (i.e. hyperspectral imagery (HSI)) prevents researchers from taking advantage of this recent success. Since generating sensor specific datasets is time intensive and cost prohibitive, remote sensing researchers have embraced deep unsupervised feature extraction. Although these methods have pushed state-of-the-art performance on current HSI benchmarks, many of these tools are not readily accessible to many researchers. In this letter, we introduce a software pipeline, which we call EarthMapper, for the semantic segmentation of non-RGB remote sensing imagery. It includes self-taught spatial-spectral feature extraction, various standard and deep learning classifiers, and undirected graphical models for post-processing. We evaluated EarthMapper on the Indian Pines and Pavia University datasets and have released this code for public use.
Tasks	Segmentation Of Remote Sensing Imagery, Semantic Segmentation, The Semantic Segmentation Of Remote Sensing Imagery
Published	2018-04-01
URL	http://arxiv.org/abs/1804.00292v1
PDF	http://arxiv.org/pdf/1804.00292v1.pdf
PWC	https://paperswithcode.com/paper/earthmapper-a-tool-box-for-the-semantic
Repo	https://github.com/rmkemker/EarthMapper
Framework	tf

Low-Shot Learning for the Semantic Segmentation of Remote Sensing Imagery


Title	Low-Shot Learning for the Semantic Segmentation of Remote Sensing Imagery
Authors	Ronald Kemker, Ryan Luu, Christopher Kanan
Abstract	Recent advances in computer vision using deep learning with RGB imagery (e.g., object recognition and detection) have been made possible thanks to the development of large annotated RGB image datasets. In contrast, multispectral image (MSI) and hyperspectral image (HSI) datasets contain far fewer labeled images, in part due to the wide variety of sensors used. These annotations are especially limited for semantic segmentation, or pixel-wise classification, of remote sensing imagery because it is labor intensive to generate image annotations. Low-shot learning algorithms can make effective inferences despite smaller amounts of annotated data. In this paper, we study low-shot learning using self-taught feature learning for semantic segmentation. We introduce 1) an improved self-taught feature learning framework for HSI and MSI data and 2) a semi-supervised classification algorithm. When these are combined, they achieve state-of-the-art performance on remote sensing datasets that have little annotated training data available. These low-shot learning frameworks will reduce the manual image annotation burden and improve semantic segmentation performance for remote sensing imagery.
Tasks	Few-Shot Image Classification, Few-Shot Learning, Object Recognition, Segmentation Of Remote Sensing Imagery, Semantic Segmentation, The Semantic Segmentation Of Remote Sensing Imagery
Published	2018-03-26
URL	http://arxiv.org/abs/1803.09824v1
PDF	http://arxiv.org/pdf/1803.09824v1.pdf
PWC	https://paperswithcode.com/paper/low-shot-learning-for-the-semantic
Repo	https://github.com/rmkemker/EarthMapper
Framework	tf

Adaptive Skip Intervals: Temporal Abstraction for Recurrent Dynamical Models


Title	Adaptive Skip Intervals: Temporal Abstraction for Recurrent Dynamical Models
Authors	Alexander Neitz, Giambattista Parascandolo, Stefan Bauer, Bernhard Schölkopf
Abstract	We introduce a method which enables a recurrent dynamics model to be temporally abstract. Our approach, which we call Adaptive Skip Intervals (ASI), is based on the observation that in many sequential prediction tasks, the exact time at which events occur is irrelevant to the underlying objective. Moreover, in many situations, there exist prediction intervals which result in particularly easy-to-predict transitions. We show that there are prediction tasks for which we gain both computational efficiency and prediction accuracy by allowing the model to make predictions at a sampling rate which it can choose itself.
Tasks
Published	2018-08-14
URL	http://arxiv.org/abs/1808.04768v3
PDF	http://arxiv.org/pdf/1808.04768v3.pdf
PWC	https://paperswithcode.com/paper/adaptive-skip-intervals-temporal-abstraction
Repo	https://github.com/neitzal/asi-tasks
Framework	tf

Human Activity Recognition Based on Wearable Sensor Data: A Standardization of the State-of-the-Art


Title	Human Activity Recognition Based on Wearable Sensor Data: A Standardization of the State-of-the-Art
Authors	Artur Jordao, Antonio C. Nazare Jr., Jessica Sena, William Robson Schwartz
Abstract	Human activity recognition based on wearable sensor data has been an attractive research topic due to its application in areas such as healthcare and smart environments. In this context, many works have presented remarkable results using accelerometer, gyroscope and magnetometer data to represent the activities categories. However, current studies do not consider important issues that lead to skewed results, making it hard to assess the quality of sensor-based human activity recognition and preventing a direct comparison of previous works. These issues include the samples generation processes and the validation protocols used. We emphasize that in other research areas, such as image classification and object detection, these issues are already well-defined, which brings more efforts towards the application. Inspired by this, we conduct an extensive set of experiments that analyze different sample generation processes and validation protocols to indicate the vulnerable points in human activity recognition based on wearable sensor data. For this purpose, we implement and evaluate several top-performance methods, ranging from handcrafted-based approaches to convolutional neural networks. According to our study, most of the experimental evaluations that are currently employed are not adequate to perform the activity recognition in the context of wearable sensor data, in which the recognition accuracy drops considerably when compared to an appropriate evaluation approach. To the best of our knowledge, this is the first study that tackles essential issues that compromise the understanding of the performance in human activity recognition based on wearable sensor data.
Tasks	Activity Recognition, Human Activity Recognition, Image Classification, Object Detection
Published	2018-06-13
URL	http://arxiv.org/abs/1806.05226v3
PDF	http://arxiv.org/pdf/1806.05226v3.pdf
PWC	https://paperswithcode.com/paper/human-activity-recognition-based-on-wearable
Repo	https://github.com/arturjordao/WearableSensorData
Framework	tf