January 27, 2020

3299 words 16 mins read

Paper Group ANR 1086

VITAMIN-E: VIsual Tracking And MappINg with Extremely Dense Feature Points. Energy-Based Continuous Inverse Optimal Control. AMAD: Adversarial Multiscale Anomaly Detection on High-Dimensional and Time-Evolving Categorical Data. Neural Network Based Parameter Estimation Method for the Pareto/NBD Model. Compressing GANs using Knowledge Distillation. …

VITAMIN-E: VIsual Tracking And MappINg with Extremely Dense Feature Points


Title	VITAMIN-E: VIsual Tracking And MappINg with Extremely Dense Feature Points
Authors	Masashi Yokozuka, Shuji Oishi, Thompson Simon, Atsuhiko Banno
Abstract	In this paper, we propose a novel indirect monocular SLAM algorithm called “VITAMIN-E,” which is highly accurate and robust as a result of tracking extremely dense feature points. Typical indirect methods have difficulty in reconstructing dense geometry because of their careful feature point selection for accurate matching. Unlike conventional methods, the proposed method processes an enormous number of feature points by tracking the local extrema of curvature informed by dominant flow estimation. Because this may lead to high computational cost during bundle adjustment, we propose a novel optimization technique, the “subspace Gauss–Newton method”, that significantly improves the computational efficiency of bundle adjustment by partially updating the variables. We concurrently generate meshes from the reconstructed points and merge them for an entire 3D model. The experimental results on the SLAM benchmark dataset EuRoC demonstrated that the proposed method outperformed state-of-the-art SLAM methods, such as DSO, ORB-SLAM, and LSD-SLAM, both in terms of accuracy and robustness in trajectory estimation. The proposed method simultaneously generated significantly detailed 3D geometry from the dense feature points in real time using only a CPU.
Tasks	Visual Tracking
Published	2019-04-23
URL	https://arxiv.org/abs/1904.10324v2
PDF	https://arxiv.org/pdf/1904.10324v2.pdf
PWC	https://paperswithcode.com/paper/vitamin-e-visual-tracking-and-mapping-with
Repo
Framework

Energy-Based Continuous Inverse Optimal Control


Title	Energy-Based Continuous Inverse Optimal Control
Authors	Yifei Xu, Jianwen Xie, Tianyang Zhao, Chris Baker, Yibiao Zhao, Ying Nian Wu
Abstract	The problem of continuous optimal control (over finite time horizon) is to minimize a given cost function over the sequence of continuous control variables. The problem of continuous inverse optimal control is to learn the unknown cost function from expert demonstrations. In this article, we study this fundamental problem in the framework of energy-based model, where the observed expert trajectories are assumed to be random samples from a probability density function defined as the exponential of the negative cost function up to a normalizing constant. The parameters of the cost function are learned by maximum likelihood via an “analysis by synthesis” scheme, which iterates the following two steps: (1) Synthesis step: sample the synthesized trajectories from the current probability density using the Langevin dynamics. (2) Analysis step: update the model parameters based on the difference between the synthesized trajectories and the observed trajectories. Given the fact that an efficient optimization algorithm is usually available for an optimal control problem, we also consider a variation of the above learning method, where we modify the synthesis step (1) into an optimization step while keeping the analysis step (2) unchanged. Specifically, instead of sampling each synthesized trajectory from the current probability density, we minimize the current cost function over the sequence of control variables using the existing optimization algorithm. We give justifications for this optimization-based method. We demonstrate the proposed energy-based continuous optimal control methods on autonomous driving tasks, and show that the proposed methods can learn suitable cost functions for optimal control.
Tasks	Autonomous Driving, Continuous Control, Trajectory Prediction
Published	2019-04-10
URL	https://arxiv.org/abs/1904.05453v3
PDF	https://arxiv.org/pdf/1904.05453v3.pdf
PWC	https://paperswithcode.com/paper/learning-trajectory-prediction-with
Repo
Framework

AMAD: Adversarial Multiscale Anomaly Detection on High-Dimensional and Time-Evolving Categorical Data


Title	AMAD: Adversarial Multiscale Anomaly Detection on High-Dimensional and Time-Evolving Categorical Data
Authors	Zheng Gao, Lin Guo, Chi Ma, Xiao Ma, Kai Sun, Hang Xiang, Xiaoqiang Zhu, Hongsong Li, Xiaozhong Liu
Abstract	Anomaly detection is facing with emerging challenges in many important industry domains, such as cyber security and online recommendation and advertising. The recent trend in these areas calls for anomaly detection on time-evolving data with high-dimensional categorical features without labeled samples. Also, there is an increasing demand for identifying and monitoring irregular patterns at multiple resolutions. In this work, we propose a unified end-to-end approach to solve these challenges by combining the advantages of Adversarial Autoencoder and Recurrent Neural Network. The model learns data representations cross different scales with attention mechanisms, on which an enhanced two-resolution anomaly detector is developed for both instances and data blocks. Extensive experiments are performed over three types of datasets to demonstrate the efficacy of our method and its superiority over the state-of-art approaches.
Tasks	Anomaly Detection
Published	2019-07-12
URL	https://arxiv.org/abs/1907.06582v1
PDF	https://arxiv.org/pdf/1907.06582v1.pdf
PWC	https://paperswithcode.com/paper/amad-adversarial-multiscale-anomaly-detection
Repo
Framework

Neural Network Based Parameter Estimation Method for the Pareto/NBD Model


Title	Neural Network Based Parameter Estimation Method for the Pareto/NBD Model
Authors	Shao-Ming Xie
Abstract	Whether stochastic or parametric, the Pareto/NBD model can only be utilized for an in-sample prediction rather than an out-of-sample prediction. This research thus provides a neural network based extension of the Pareto/NBD model to estimate the out-of-sample parameters, which overrides the estimation burden and the application dilemma of the Pareto/NBD approach. The empirical results indicate that the Pareto/NBD model and neural network algorithms have similar predictability for identifying inactive customers. Even with a strong trend fitting on the customer count of each repeat purchase point, the Pareto/NBD model underestimates repeat purchases at both the individual and aggregate levels. Nonetheless, when embedding the likelihood function of the Pareto/NBD model into the loss function, the proposed parameter estimation method shows extraordinary predictability on repeat purchases at these two levels. Furthermore, the proposed neural network based method is highly efficient and resource-friendly and can be deployed in cloud computing to handle with big data analysis.
Tasks
Published	2019-11-05
URL	https://arxiv.org/abs/1911.01919v1
PDF	https://arxiv.org/pdf/1911.01919v1.pdf
PWC	https://paperswithcode.com/paper/neural-network-based-parameter-estimation
Repo
Framework

Compressing GANs using Knowledge Distillation


Title	Compressing GANs using Knowledge Distillation
Authors	Angeline Aguinaldo, Ping-Yeh Chiang, Alex Gain, Ameya Patil, Kolten Pearson, Soheil Feizi
Abstract	Generative Adversarial Networks (GANs) have been used in several machine learning tasks such as domain transfer, super resolution, and synthetic data generation. State-of-the-art GANs often use tens of millions of parameters, making them expensive to deploy for applications in low SWAP (size, weight, and power) hardware, such as mobile devices, and for applications with real time capabilities. There has been no work found to reduce the number of parameters used in GANs. Therefore, we propose a method to compress GANs using knowledge distillation techniques, in which a smaller “student” GAN learns to mimic a larger “teacher” GAN. We show that the distillation methods used on MNIST, CIFAR-10, and Celeb-A datasets can compress teacher GANs at ratios of 1669:1, 58:1, and 87:1, respectively, while retaining the quality of the generated image. From our experiments, we observe a qualitative limit for GAN’s compression. Moreover, we observe that, with a fixed parameter budget, compressed GANs outperform GANs trained using standard training methods. We conjecture that this is partially owing to the optimization landscape of over-parameterized GANs which allows efficient training using alternating gradient descent. Thus, training an over-parameterized GAN followed by our proposed compression scheme provides a high quality generative model with a small number of parameters.
Tasks	Super-Resolution, Synthetic Data Generation
Published	2019-02-01
URL	http://arxiv.org/abs/1902.00159v1
PDF	http://arxiv.org/pdf/1902.00159v1.pdf
PWC	https://paperswithcode.com/paper/compressing-gans-using-knowledge-distillation
Repo
Framework

Improved MR to CT synthesis for PET/MR attenuation correction using Imitation Learning


Title	Improved MR to CT synthesis for PET/MR attenuation correction using Imitation Learning
Authors	Kerstin Kläser, Thomas Varsavsky, Pawel Markiewicz, Tom Vercauteren, David Atkinson, Kris Thielemans, Brian Hutton, M Jorge Cardoso, Sebastien Ourselin
Abstract	The ability to synthesise Computed Tomography images - commonly known as pseudo CT, or pCT - from MRI input data is commonly assessed using an intensity-wise similarity, such as an L2-norm between the ground truth CT and the pCT. However, given that the ultimate purpose is often to use the pCT as an attenuation map ($\mu$-map) in Positron Emission Tomography Magnetic Resonance Imaging (PET/MRI), minimising the error between pCT and CT is not necessarily optimal. The main objective should be to predict a pCT that, when used as $\mu$-map, reconstructs a pseudo PET (pPET) which is as close as possible to the gold standard PET. To this end, we propose a novel multi-hypothesis deep learning framework that generates pCTs by minimising a combination of the pixel-wise error between pCT and CT and a proposed metric-loss that itself is represented by a convolutional neural network (CNN) and aims to minimise subsequent PET residuals. The model is trained on a database of 400 paired MR/CT/PET image slices. Quantitative results show that the network generates pCTs that seem less accurate when evaluating the Mean Absolute Error on the pCT (69.68HU) compared to a baseline CNN (66.25HU), but lead to significant improvement in the PET reconstruction - 115a.u. compared to baseline 140a.u.
Tasks	Imitation Learning
Published	2019-08-21
URL	https://arxiv.org/abs/1908.08431v2
PDF	https://arxiv.org/pdf/1908.08431v2.pdf
PWC	https://paperswithcode.com/paper/improved-mr-to-ct-synthesis-for-petmr
Repo
Framework

Generic Multiview Visual Tracking


Title	Generic Multiview Visual Tracking
Authors	Minye Wu, Haibin Ling, Ning Bi, Shenghua Gao, Hao Sheng, Jingyi Yu
Abstract	Recent progresses in visual tracking have greatly improved the tracking performance. However, challenges such as occlusion and view change remain obstacles in real world deployment. A natural solution to these challenges is to use multiple cameras with multiview inputs, though existing systems are mostly limited to specific targets (e.g. human), static cameras, and/or camera calibration. To break through these limitations, we propose a generic multiview tracking (GMT) framework that allows camera movement, while requiring neither specific object model nor camera calibration. A key innovation in our framework is a cross-camera trajectory prediction network (TPN), which implicitly and dynamically encodes camera geometric relations, and hence addresses missing target issues such as occlusion. Moreover, during tracking, we assemble information across different cameras to dynamically update a novel collaborative correlation filter (CCF), which is shared among cameras to achieve robustness against view change. The two components are integrated into a correlation filter tracking framework, where the features are trained offline using existing single view tracking datasets. For evaluation, we first contribute a new generic multiview tracking dataset (GMTD) with careful annotations, and then run experiments on GMTD and the PETS2009 datasets. On both datasets, the proposed GMT algorithm shows clear advantages over state-of-the-art ones.
Tasks	Calibration, Trajectory Prediction, Visual Tracking
Published	2019-04-04
URL	http://arxiv.org/abs/1904.02553v1
PDF	http://arxiv.org/pdf/1904.02553v1.pdf
PWC	https://paperswithcode.com/paper/generic-multiview-visual-tracking
Repo
Framework

Holographic MIMO Surfaces for 6G Wireless Networks: Opportunities, Challenges, and Trends


Title	Holographic MIMO Surfaces for 6G Wireless Networks: Opportunities, Challenges, and Trends
Authors	Chongwen Huang, Sha Hu, George C. Alexandropoulos, Alessio Zappone, Chau Yuen, Rui Zhang, Marco Di Renzo, Mérouane Debbah
Abstract	Future wireless networks are expected to evolve towards an intelligent and software reconfigurable functionality paradigm enabling ubiquitous communication between humans and mobile devices, but also being capable of sensing, controlling, and optimizing the wireless environment to fulfill the visions of low powered, high throughput, massive connectivity, and low latency communications. A key conceptual enabler that is recently gaining increasing popularity is the Holographic Multiple Input Multiple Output Surface (HMIMOS) that refers to a low cost transformative wireless planar structure comprising of sub-wavelength metallic or dielectric scattering particles, which is capable of impacting electromagnetic waves according to desired objectives. In this article, we provide an overview of HMIMOS communications by introducing the available hardware architectures for reconfigurable metasurfaces and their main characteristics, as well as highlighting the opportunities and key design challenges when multiple HMIMOS are considered.
Tasks
Published	2019-11-27
URL	https://arxiv.org/abs/1911.12296v2
PDF	https://arxiv.org/pdf/1911.12296v2.pdf
PWC	https://paperswithcode.com/paper/holographic-mimo-surfaces-for-6g-wireless
Repo
Framework

Irregular Convolutional Auto-Encoder on Point Clouds


Title	Irregular Convolutional Auto-Encoder on Point Clouds
Authors	Zhang Yuhui, Greg Gutmann, Konagaya Akihiko
Abstract	We proposed a novel graph convolutional neural network that could construct a coarse, sparse latent point cloud from a dense, raw point cloud. With a novel non-isotropic convolution operation defined on irregular geometries, the model then can reconstruct the original point cloud from this latent cloud with fine details. Furthermore, we proposed that it is even possible to perform particle simulation using the latent cloud encoded from some simulated particle cloud (e.g. fluids), to accelerate the particle simulation process. Our model has been tested on ShapeNetCore dataset for Auto-Encoding with a limited latent dimension and tested on a synthesis dataset for fluids simulation. We also compare the model with other state-of-the-art models, and several visualizations were done to intuitively understand the model.
Tasks
Published	2019-10-07
URL	https://arxiv.org/abs/1910.02686v1
PDF	https://arxiv.org/pdf/1910.02686v1.pdf
PWC	https://paperswithcode.com/paper/irregular-convolutional-auto-encoder-on-point
Repo
Framework

A bi-partite generative model framework for analyzing and simulating large scale multiple discrete-continuous travel behaviour data


Title	A bi-partite generative model framework for analyzing and simulating large scale multiple discrete-continuous travel behaviour data
Authors	Melvin Wong, Bilal Farooq
Abstract	The emergence of data-driven demand analysis have led to the increased use of generative modelling to learn the probabilistic dependencies between random variables. Although their apparent use has largely been limited to image recognition and classification in recent years, generative machine learning algorithms can be a powerful tool for travel behaviour research by replicating travel behaviour by the underlying properties of data structures. In this paper, we examine the use of generative machine learning approach for analyzing multiple discrete-continuous (MDC) travel behaviour data. We provide a plausible perspective of how we can exploit the use of machine learning techniques to interpret the underlying heterogeneities in the data. We show that generative models are conceptually similar to choice selection behaviour process through information entropy and variational Bayesian inference. Without loss of generality, we consider a restricted Boltzmann machine (RBM) based algorithm with multiple discrete-continuous layer, formulated as a variational Bayesian inference optimization problem. We systematically describe the proposed machine learning algorithm and develop a process of analyzing travel behaviour data from a generative learning perspective. We show parameter stability from model analysis and simulation tests on an open dataset with multiple discrete-continuous dimensions from a data size of 293,330 observations. For interpretability, we derive the conditional probabilities, elasticities and perform statistical analysis on the latent variables. We show that our model can generate statistically similar data distributions for travel forecasting and prediction and performs better than purely discriminative methods in validation. Our results indicate that latent constructs in generative models can accurately represent the joint distribution consistently on MDC data.
Tasks	Bayesian Inference, Dimensionality Reduction
Published	2019-01-18
URL	https://arxiv.org/abs/1901.06415v2
PDF	https://arxiv.org/pdf/1901.06415v2.pdf
PWC	https://paperswithcode.com/paper/a-combined-entropy-and-utility-based
Repo
Framework

Depth creates no more spurious local minima


Title	Depth creates no more spurious local minima
Authors	Li Zhang
Abstract	We show that for any convex differentiable loss, a deep linear network has no spurious local minima as long as it is true for the two layer case. This reduction greatly simplifies the study on the existence of spurious local minima in deep linear networks. When applied to the quadratic loss, our result immediately implies the powerful result in [Kawaguchi 2016]. Further, with the work in [Zhou and Liang 2018], we can remove all the assumptions in [Kawaguchi 2016]. This property holds for more general “multi-tower” linear networks too. Our proof builds on [Laurent and von Brecht 2018] and develops a new perturbation argument to show that any spurious local minimum must have full rank, a structural property which can be useful more generally.
Tasks
Published	2019-01-28
URL	https://arxiv.org/abs/1901.09827v2
PDF	https://arxiv.org/pdf/1901.09827v2.pdf
PWC	https://paperswithcode.com/paper/depth-creates-no-more-spurious-local-minima
Repo
Framework

MedGCN: Graph Convolutional Networks for Multiple Medical Tasks


Title	MedGCN: Graph Convolutional Networks for Multiple Medical Tasks
Authors	Chengsheng Mao, Liang Yao, Yuan Luo
Abstract	Laboratory testing and medication prescription are two of the most important routines in daily clinical practice. Developing an artificial intelligence system that can automatically make lab test imputations and medication recommendations can save cost on potentially redundant lab tests and inform physicians in more effective prescription. We present an intelligent model that can automatically recommend the patients’ medications based on their incomplete lab tests, and can even accurately estimate the lab values that have not been taken. We model the complex relations between multiple types of medical entities with their inherent features in a heterogeneous graph. Then we learn a distributed representation for each entity in the graph based on graph convolutional networks to make the representations integrate information from multiple types of entities. Since the entity representations incorporate multiple types of medical information, they can be used for multiple medical tasks. In our experiments, we construct a graph to associate patients, encounters, lab tests and medications, and conduct the two tasks: medication recommendation and lab test imputation. The experimental results demonstrate that our model can outperform the state-of-the-art models in both tasks.
Tasks	Imputation
Published	2019-03-31
URL	http://arxiv.org/abs/1904.00326v1
PDF	http://arxiv.org/pdf/1904.00326v1.pdf
PWC	https://paperswithcode.com/paper/medgcn-graph-convolutional-networks-for
Repo
Framework

Robust Federated Learning Through Representation Matching and Adaptive Hyper-parameters


Title	Robust Federated Learning Through Representation Matching and Adaptive Hyper-parameters
Authors	Hesham Mostafa
Abstract	Federated learning is a distributed, privacy-aware learning scenario which trains a single model on data belonging to several clients. Each client trains a local model on its data and the local models are then aggregated by a central party. Current federated learning methods struggle in cases with heterogeneous client-side data distributions which can quickly lead to divergent local models and a collapse in performance. Careful hyper-parameter tuning is particularly important in these cases but traditional automated hyper-parameter tuning methods would require several training trials which is often impractical in a federated learning setting. We describe a two-pronged solution to the issues of robustness and hyper-parameter tuning in federated learning settings. We propose a novel representation matching scheme that reduces the divergence of local models by ensuring the feature representations in the global (aggregate) model can be derived from the locally learned representations. We also propose an online hyper-parameter tuning scheme which uses an online version of the REINFORCE algorithm to find a hyper-parameter distribution that maximizes the expected improvements in training loss. We show on several benchmarks that our two-part scheme of local representation matching and global adaptive hyper-parameters significantly improves performance and training robustness.
Tasks
Published	2019-12-30
URL	https://arxiv.org/abs/1912.13075v1
PDF	https://arxiv.org/pdf/1912.13075v1.pdf
PWC	https://paperswithcode.com/paper/robust-federated-learning-through-1
Repo
Framework

Stage-based Hyper-parameter Optimization for Deep Learning


Title	Stage-based Hyper-parameter Optimization for Deep Learning
Authors	Ahnjae Shin, Dong-Jin Shin, Sungwoo Cho, Do Yoon Kim, Eunji Jeong, Gyeong-In Yu, Byung-Gon Chun
Abstract	As deep learning techniques advance more than ever, hyper-parameter optimization is the new major workload in deep learning clusters. Although hyper-parameter optimization is crucial in training deep learning models for high model performance, effectively executing such a computation-heavy workload still remains a challenge. We observe that numerous trials issued from existing hyper-parameter optimization algorithms share common hyper-parameter sequence prefixes, which implies that there are redundant computations from training the same hyper-parameter sequence multiple times. We propose a stage-based execution strategy for efficient execution of hyper-parameter optimization algorithms. Our strategy removes redundancy in the training process by splitting the hyper-parameter sequences of trials into homogeneous stages, and generating a tree of stages by merging the common prefixes. Our preliminary experiment results show that applying stage-based execution to hyper-parameter optimization algorithms outperforms the original trial-based method, saving required GPU-hours and end-to-end training time by up to 6.60 times and 4.13 times, respectively.
Tasks
Published	2019-11-24
URL	https://arxiv.org/abs/1911.10504v1
PDF	https://arxiv.org/pdf/1911.10504v1.pdf
PWC	https://paperswithcode.com/paper/stage-based-hyper-parameter-optimization-for
Repo
Framework

Globally Guided Progressive Fusion Network for 3D Pancreas Segmentation


Title	Globally Guided Progressive Fusion Network for 3D Pancreas Segmentation
Authors	Chaowei Fang, Guanbin Li, Chengwei Pan, Yiming Li, Yizhou Yu
Abstract	Recently 3D volumetric organ segmentation attracts much research interest in medical image analysis due to its significance in computer aided diagnosis. This paper aims to address the pancreas segmentation task in 3D computed tomography volumes. We propose a novel end-to-end network, Globally Guided Progressive Fusion Network, as an effective and efficient solution to volumetric segmentation, which involves both global features and complicated 3D geometric information. A progressive fusion network is devised to extract 3D information from a moderate number of neighboring slices and predict a probability map for the segmentation of each slice. An independent branch for excavating global features from downsampled slices is further integrated into the network. Extensive experimental results demonstrate that our method achieves state-of-the-art performance on two pancreas datasets.
Tasks	Pancreas Segmentation
Published	2019-11-23
URL	https://arxiv.org/abs/1911.10360v1
PDF	https://arxiv.org/pdf/1911.10360v1.pdf
PWC	https://paperswithcode.com/paper/globally-guided-progressive-fusion-network
Repo
Framework