April 2, 2020

# Paper Group ANR 113

Generalized Bayesian Filtering via Sequential Monte Carlo. Robust Wireless Fingerprinting: Generalizing Across Space and Time. On Unbalanced Optimal Transport: An Analysis of Sinkhorn Algorithm. Graph Attention Network based Pruning for Reconstructing 3D Liver Vessel Morphology from Contrasted CT Images. Rapidly Adaptable Legged Robots via Evolutio …

#### Generalized Bayesian Filtering via Sequential Monte Carlo

Title Generalized Bayesian Filtering via Sequential Monte Carlo
Authors Ayman Boustati, Ömer Deniz Akyildiz, Theodoros Damoulas, Adam Johansen
Abstract We introduce a framework for inference in general state-space hidden Markov models (HMMs) under likelihood misspecification. In particular, we leverage the loss-theoretic perspective of generalized Bayesian inference (GBI) to define generalized filtering recursions in HMMs, that can tackle the problem of inference under model misspecification. In doing so, we arrive at principled procedures for robust inference against observation contamination through the $\beta$-divergence. Operationalizing the proposed framework is made possible via sequential Monte Carlo methods (SMC). The standard particle methods, and their associated convergence results, are readily generalized to the new setting. We demonstrate our approach to object tracking and Gaussian process regression problems, and observe improved performance over standard filtering algorithms.
Published 2020-02-23
URL https://arxiv.org/abs/2002.09998v1
PDF https://arxiv.org/pdf/2002.09998v1.pdf
PWC https://paperswithcode.com/paper/generalized-bayesian-filtering-via-sequential
Repo
Framework

#### Robust Wireless Fingerprinting: Generalizing Across Space and Time

Title Robust Wireless Fingerprinting: Generalizing Across Space and Time
Authors Metehan Cekic, Soorya Gopalakrishnan, Upamanyu Madhow
Abstract Can we distinguish between two wireless transmitters sending exactly the same message, using the same protocol? The opportunity for doing so arises due to subtle nonlinear variations across transmitters, even those made by the same manufacturer. Since these effects are difficult to model explicitly, we investigate learning device fingerprints using complex-valued deep neural networks (DNNs) that take as input the complex baseband signal at the receiver. Such fingerprints should be robust to ID spoofing, and to distribution shifts across days and locations due to clock drift and variations in the wireless channel. In this paper, we point out that, unless proactively discouraged from doing so, DNNs learn these strong confounding features rather than the subtle nonlinear characteristics that are the basis for stable signatures. Thus, a network trained on data collected during one day performs poorly on a different day, and networks allowed access to post-preamble information rely on easily-spoofed ID fields. We propose and evaluate strategies, based on augmentation and estimation, to promote generalization across realizations of these confounding factors, using data from WiFi and ADS-B protocols. We conclude that, while DNN training has the advantage of not requiring explicit signal models, significant modeling insights are required to focus the learning on the effects we wish to capture.
Published 2020-02-25
URL https://arxiv.org/abs/2002.10791v1
PDF https://arxiv.org/pdf/2002.10791v1.pdf
PWC https://paperswithcode.com/paper/robust-wireless-fingerprinting-generalizing
Repo
Framework

#### On Unbalanced Optimal Transport: An Analysis of Sinkhorn Algorithm

Title On Unbalanced Optimal Transport: An Analysis of Sinkhorn Algorithm
Authors Khiem Pham, Khang Le, Nhat Ho, Tung Pham, Hung Bui
Abstract We provide a computational complexity analysis for the Sinkhorn algorithm that solves the entropic regularized Unbalanced Optimal Transport (UOT) problem between two measures of possibly different masses with at most $n$ components. We show that the complexity of the Sinkhorn algorithm for finding an $\varepsilon$-approximate solution to the UOT problem is of order $\widetilde{\mathcal{O}}(n^2/ \varepsilon)$, which is near-linear time. To the best of our knowledge, this complexity is better than the complexity of the Sinkhorn algorithm for solving the Optimal Transport (OT) problem, which is of order $\widetilde{\mathcal{O}}(n^2/\varepsilon^2)$. Our proof technique is based on the geometric convergence of the Sinkhorn updates to the optimal dual solution of the entropic regularized UOT problem and some properties of the primal solution. It is also different from the proof for the complexity of the Sinkhorn algorithm for approximating the OT problem since the UOT solution does not have to meet the marginal constraints.
Published 2020-02-09
URL https://arxiv.org/abs/2002.03293v1
PDF https://arxiv.org/pdf/2002.03293v1.pdf
PWC https://paperswithcode.com/paper/on-unbalanced-optimal-transport-an-analysis
Repo
Framework

#### Graph Attention Network based Pruning for Reconstructing 3D Liver Vessel Morphology from Contrasted CT Images

Title Graph Attention Network based Pruning for Reconstructing 3D Liver Vessel Morphology from Contrasted CT Images
Authors Donghao Zhang, Siqi Liu, Shikha Chaganti, Eli Gibson, Zhoubing Xu, Sasa Grbic, Weidong Cai, Dorin Comaniciu
Abstract With the injection of contrast material into blood vessels, multi-phase contrasted CT images can enhance the visibility of vessel networks in the human body. Reconstructing the 3D geometric morphology of liver vessels from the contrasted CT images can enable multiple liver preoperative surgical planning applications. Automatic reconstruction of liver vessel morphology remains a challenging problem due to the morphological complexity of liver vessels and the inconsistent vessel intensities among different multi-phase contrasted CT images. On the other side, high integrity is required for the 3D reconstruction to avoid decision making biases. In this paper, we propose a framework for liver vessel morphology reconstruction using both a fully convolutional neural network and a graph attention network. A fully convolutional neural network is first trained to produce the liver vessel centerline heatmap. An over-reconstructed liver vessel graph model is then traced based on the heatmap using an image processing based algorithm. We use a graph attention network to prune the false-positive branches by predicting the presence probability of each segmented branch in the initial reconstruction using the aggregated CNN features. We evaluated the proposed framework on an in-house dataset consisting of 418 multi-phase abdomen CT images with contrast. The proposed graph network pruning improves the overall reconstruction F1 score by 6.4% over the baseline. It also outperformed the other state-of-the-art curvilinear structure reconstruction algorithms.
Tasks 3D Reconstruction, Decision Making, Network Pruning
Published 2020-03-18
URL https://arxiv.org/abs/2003.07999v1
PDF https://arxiv.org/pdf/2003.07999v1.pdf
PWC https://paperswithcode.com/paper/graph-attention-network-based-pruning-for
Repo
Framework

#### Rapidly Adaptable Legged Robots via Evolutionary Meta-Learning

Title Rapidly Adaptable Legged Robots via Evolutionary Meta-Learning
Authors Xingyou Song, Yuxiang Yang, Krzysztof Choromanski, Ken Caluwaerts, Wenbo Gao, Chelsea Finn, Jie Tan
Abstract Learning adaptable policies is crucial for robots to operate autonomously in our complex and quickly changing world. In this work, we present a new meta-learning method that allows robots to quickly adapt to changes in dynamics. In contrast to gradient-based meta-learning algorithms that rely on second-order gradient estimation, we introduce a more noise-tolerant Batch Hill-Climbing adaptation operator and combine it with meta-learning based on evolutionary strategies. Our method significantly improves adaptation to changes in dynamics in high noise settings, which are common in robotics applications. We validate our approach on a quadruped robot that learns to walk while subject to changes in dynamics. We observe that our method significantly outperforms prior gradient-based approaches, enabling the robot to adapt its policy to changes based on less than 3 minutes of real data.
Published 2020-03-02
URL https://arxiv.org/abs/2003.01239v1
PDF https://arxiv.org/pdf/2003.01239v1.pdf
Repo
Framework

#### Towards Augmented Reality-based Suturing in Monocular Laparoscopic Training

Title Towards Augmented Reality-based Suturing in Monocular Laparoscopic Training
Authors Chandrakanth Jayachandran Preetha, Jonathan Kloss, Fabian Siegfried Wehrtmann, Lalith Sharan, Carolyn Fan, Beat Peter Müller-Stich, Felix Nickel, Sandy Engelhardt
Abstract Minimally Invasive Surgery (MIS) techniques have gained rapid popularity among surgeons since they offer significant clinical benefits including reduced recovery time and diminished post-operative adverse effects. However, conventional endoscopic systems output monocular video which compromises depth perception, spatial orientation and field of view. Suturing is one of the most complex tasks performed under these circumstances. Key components of this tasks are the interplay between needle holder and the surgical needle. Reliable 3D localization of needle and instruments in real time could be used to augment the scene with additional parameters that describe their quantitative geometric relation, e.g. the relation between the estimated needle plane and its rotation center and the instrument. This could contribute towards standardization and training of basic skills and operative techniques, enhance overall surgical performance, and reduce the risk of complications. The paper proposes an Augmented Reality environment with quantitative and qualitative visual representations to enhance laparoscopic training outcomes performed on a silicone pad. This is enabled by a multi-task supervised deep neural network which performs multi-class segmentation and depth map prediction. Scarcity of labels has been conquered by creating a virtual environment which resembles the surgical training scenario to generate dense depth maps and segmentation maps. The proposed convolutional neural network was tested on real surgical training scenarios and showed to be robust to occlusion of the needle. The network achieves a dice score of 0.67 for surgical needle segmentation, 0.81 for needle holder instrument segmentation and a mean absolute error of 6.5 mm for depth estimation.
Published 2020-01-19
URL https://arxiv.org/abs/2001.06894v1
PDF https://arxiv.org/pdf/2001.06894v1.pdf
PWC https://paperswithcode.com/paper/towards-augmented-reality-based-suturing-in
Repo
Framework

#### Pruning untrained neural networks: Principles and Analysis

Title Pruning untrained neural networks: Principles and Analysis
Authors Soufiane Hayou, Jean-Francois Ton, Arnaud Doucet, Yee Whye Teh
Abstract Overparameterized neural networks display state-of-the art performance. However, there is a growing need for smaller, energy-efficient, neural networks to be able to use machine learning applications on devices with limited computational resources. A popular approach consists of using pruning techniques. While these techniques have traditionally focused on pruning pre-trained neural networks (e.g. LeCun et al. (1990) and Hassabi et al. (1993)), recent work by Lee et al. (2018) showed promising results where pruning is performed at initialization. However, such procedures remain unsatisfactory as the resulting pruned networks can be difficult to train and, for instance, these procedures do not prevent one layer being fully pruned. In this paper we provide a comprehensive theoretical analysis of pruning at initialization and training sparse architectures. This analysis allows us to propose novel principled approaches which we validate experimentally on a variety of network architectures. We particularly show that we can prune up to 99.9% of the weights while keeping the model trainable.
Published 2020-02-19
URL https://arxiv.org/abs/2002.08797v1
PDF https://arxiv.org/pdf/2002.08797v1.pdf
PWC https://paperswithcode.com/paper/pruning-untrained-neural-networks-principles
Repo
Framework

#### Layer-wise Conditioning Analysis in Exploring the Learning Dynamics of DNNs

Title Layer-wise Conditioning Analysis in Exploring the Learning Dynamics of DNNs
Authors Lei Huang, Jie Qin, Li Liu, Fan Zhu, Ling Shao
Abstract Conditioning analysis uncovers the landscape of an optimization objective by exploring the spectrum of its curvature matrix. This has been well explored theoretically for linear models. We extend this analysis to deep neural networks (DNNs) in order to investigate their learning dynamics. To this end, we propose layer-wise conditioning analysis, which explores the optimization landscape with respect to each layer independently. Such an analysis is theoretically supported under mild assumptions that approximately hold in practice. Based on our analysis, we show that batch normalization (BN) can stabilize the training, but sometimes result in the false impression of a local minimum, which has detrimental effects on the learning. Besides, we experimentally observe that BN can improve the layer-wise conditioning of the optimization problem. Finally, we find that the last linear layer of a very deep residual network displays ill-conditioned behavior. We solve this problem by only adding one BN layer before the last linear layer, which achieves improved performance over the original and pre-activation residual networks.
Published 2020-02-25
URL https://arxiv.org/abs/2002.10801v2
PDF https://arxiv.org/pdf/2002.10801v2.pdf
PWC https://paperswithcode.com/paper/exploring-learning-dynamics-of-dnns-via
Repo
Framework

#### Thresholds of descending algorithms in inference problems

Title Thresholds of descending algorithms in inference problems
Authors Stefano Sarao Mannelli, Lenka Zdeborova
Abstract We review recent works on analyzing the dynamics of gradient-based algorithms in a prototypical statistical inference problem. Using methods and insights from the physics of glassy systems, these works showed how to understand quantitatively and qualitatively the performance of gradient-based algorithms. Here we review the key results and their interpretation in non-technical terms accessible to a wide audience of physicists in the context of related works.
Published 2020-01-02
URL https://arxiv.org/abs/2001.00479v2
PDF https://arxiv.org/pdf/2001.00479v2.pdf
PWC https://paperswithcode.com/paper/thresholds-of-descending-algorithms-in
Repo
Framework
Title NPENAS: Neural Predictor Guided Evolution for Neural Architecture Search
Authors Chen Wei, Chuang Niu, Yiping Tang, Jimin Liang
Abstract Neural architecture search (NAS) is a promising method for automatically finding excellent architectures.Commonly used search strategies such as evolutionary algorithm, Bayesian optimization, and Predictor method employs a predictor to rank sampled architectures. In this paper, we propose two predictor based algorithms NPUBO and NPENAS for neural architecture search. Firstly we propose NPUBO which takes a neural predictor with uncertainty estimation as surrogate model for Bayesian optimization. Secondly we propose a simple and effective predictor guided evolution algorithm(NPENAS), which uses neural predictor to guide evolutionary algorithm to perform selection and mutation. Finally we analyse the architecture sampling pipeline and find that mostly used random sampling pipeline tends to generate architectures in a subspace of the real underlying search space. Our proposed methods can find architecture achieves high test accuracy which is comparable with recently proposed methods on NAS-Bench-101 and NAS-Bench-201 dataset using less training and evaluated samples. Code will be publicly available after finish all the experiments.
Published 2020-03-28
URL https://arxiv.org/abs/2003.12857v1
PDF https://arxiv.org/pdf/2003.12857v1.pdf
PWC https://paperswithcode.com/paper/npenas-neural-predictor-guided-evolution-for
Repo
Framework

#### The Value of Nullspace Tuning Using Partial Label Information

Title The Value of Nullspace Tuning Using Partial Label Information
Authors Colin B. Hansen, Vishwesh Nath, Diego A. Mesa, Yuankai Huo, Bennett A. Landman, Thomas A. Lasko
Abstract In semi-supervised learning, information from unlabeled examples is used to improve the model learned from labeled examples. But in some learning problems, partial label information can be inferred from otherwise unlabeled examples and used to further improve the model. In particular, partial label information exists when subsets of training examples are known to have the same label, even though the label itself is missing. By encouraging a model to give the same label to all such examples, we can potentially improve its performance. We call this encouragement \emph{Nullspace Tuning} because the difference vector between any pair of examples with the same label should lie in the nullspace of a linear model. In this paper, we investigate the benefit of using partial label information using a careful comparison framework over well-characterized public datasets. We show that the additional information provided by partial labels reduces test error over good semi-supervised methods usually by a factor of 2, up to a factor of 5.5 in the best case. We also show that adding Nullspace Tuning to the newer and state-of-the-art MixMatch method decreases its test error by up to a factor of 1.8.
Published 2020-03-17
URL https://arxiv.org/abs/2003.07921v1
PDF https://arxiv.org/pdf/2003.07921v1.pdf
PWC https://paperswithcode.com/paper/the-value-of-nullspace-tuning-using-partial
Repo
Framework
Title DA-NAS: Data Adapted Pruning for Efficient Neural Architecture Search
Authors Xiyang Dai, Dongdong Chen, Mengchen Liu, Yinpeng Chen, Lu Yuan
Abstract Efficient search is a core issue in Neural Architecture Search (NAS). It is difficult for conventional NAS algorithms to directly search the architectures on large-scale tasks like ImageNet. In general, the cost of GPU hours for NAS grows with regard to training dataset size and candidate set size. One common way is searching on a smaller proxy dataset (e.g., CIFAR-10) and then transferring to the target task (e.g., ImageNet). These architectures optimized on proxy data are not guaranteed to be optimal on the target task. Another common way is learning with a smaller candidate set, which may require expert knowledge and indeed betrays the essence of NAS. In this paper, we present DA-NAS that can directly search the architecture for large-scale target tasks while allowing a large candidate set in a more efficient manner. Our method is based on an interesting observation that the learning speed for blocks in deep neural networks is related to the difficulty of recognizing distinct categories. We carefully design a progressive data adapted pruning strategy for efficient architecture search. It will quickly trim low performed blocks on a subset of target dataset (e.g., easy classes), and then gradually find the best blocks on the whole target dataset. At this time, the original candidate set becomes as compact as possible, providing a faster search in the target task. Experiments on ImageNet verify the effectiveness of our approach. It is 2x faster than previous methods while the accuracy is currently state-of-the-art, at 76.2% under small FLOPs constraint. It supports an argument search space (i.e., more candidate blocks) to efficiently search the best-performing architecture.
Published 2020-03-27
URL https://arxiv.org/abs/2003.12563v1
PDF https://arxiv.org/pdf/2003.12563v1.pdf
Repo
Framework

#### ASFD: Automatic and Scalable Face Detector

Title ASFD: Automatic and Scalable Face Detector
Authors Bin Zhang, Jian Li, Yabiao Wang, Ying Tai, Chengjie Wang, Jilin Li, Feiyue Huang, Yili Xia, Wenjiang Pei, Rongrong Ji
Abstract In this paper, we propose a novel Automatic and Scalable Face Detector (ASFD), which is based on a combination of neural architecture search techniques as well as a new loss design. First, we propose an automatic feature enhance module named Auto-FEM by improved differential architecture search, which allows efficient multi-scale feature fusion and context enhancement. Second, we use Distance-based Regression and Margin-based Classification (DRMC) multi-task loss to predict accurate bounding boxes and learn highly discriminative deep features. Third, we adopt compound scaling methods and uniformly scale the backbone, feature modules, and head networks to develop a family of ASFD, which are consistently more efficient than the state-of-the-art face detectors. Extensive experiments conducted on popular benchmarks, e.g. WIDER FACE and FDDB, demonstrate that our ASFD-D6 outperforms the prior strong competitors, and our lightweight ASFD-D0 runs at more than 120 FPS with Mobilenet for VGA-resolution images.
Published 2020-03-25
URL https://arxiv.org/abs/2003.11228v3
PDF https://arxiv.org/pdf/2003.11228v3.pdf
PWC https://paperswithcode.com/paper/asfd-automatic-and-scalable-face-detector
Repo
Framework

#### A New Multiple Max-pooling Integration Module and Cross Multiscale Deconvolution Network Based on Image Semantic Segmentation

Title A New Multiple Max-pooling Integration Module and Cross Multiscale Deconvolution Network Based on Image Semantic Segmentation
Authors Hongfeng You, Shengwei Tian, Long Yu, Xiang Ma, Yan Xing, Ning Xin
Abstract To better retain the deep features of an image and solve the sparsity problem of the end-to-end segmentation model, we propose a new deep convolutional network model for medical image pixel segmentation, called MC-Net. The core of this network model consists of four parts, namely, an encoder network, a multiple max-pooling integration module, a cross multiscale deconvolution decoder network and a pixel-level classification layer. In the network structure of the encoder, we use multiscale convolution instead of the traditional single-channel convolution. The multiple max-pooling integration module first integrates the output features of each submodule of the encoder network and reduces the number of parameters by convolution using a kernel size of 1. At the same time, each max-pooling layer (the pooling size of each layer is different) is spliced after each convolution to achieve the translation invariance of the feature maps of each submodule. We use the output feature maps from the multiple max-pooling integration module as the input of the decoder network; the multiscale convolution of each submodule in the decoder network is cross-fused with the feature maps generated by the corresponding multiscale convolution in the encoder network. Using the above feature map processing methods solves the sparsity problem after the max-pooling layer-generating matrix and enhances the robustness of the classification. We compare our proposed model with the well-known Fully Convolutional Networks for Semantic Segmentation (FCNs), DecovNet, PSPNet, U-net, SgeNet and other state-of-the-art segmentation networks such as HyperDenseNet, MS-Dual, Espnetv2, Denseaspp using one binary Kaggle 2018 data science bowl dataset and two multiclass dataset and obtain encouraging experimental results.
Published 2020-03-25
URL https://arxiv.org/abs/2003.11213v1
PDF https://arxiv.org/pdf/2003.11213v1.pdf
PWC https://paperswithcode.com/paper/a-new-multiple-max-pooling-integration-module
Repo
Framework

#### Run-time Deep Model Multiplexing

Title Run-time Deep Model Multiplexing
Authors Amir Erfan Eshratifar, Massoud Pedram
Abstract We propose a framework to design a light-weight neural multiplexer that given input and resource budgets, decides upon the appropriate model to be called for the inference. Mobile devices can use this framework to offload the hard inputs to the cloud while inferring the easy ones locally. Besides, in the large scale cloud-based intelligent applications, instead of replicating the most-accurate model, a range of small and large models can be multiplexed from depending on the input’s complexity and resource budgets. Our experimental results demonstrate the effectiveness of our framework benefiting both mobile users and cloud providers.