Paper Group ANR 113
Generalized Bayesian Filtering via Sequential Monte Carlo. Robust Wireless Fingerprinting: Generalizing Across Space and Time. On Unbalanced Optimal Transport: An Analysis of Sinkhorn Algorithm. Graph Attention Network based Pruning for Reconstructing 3D Liver Vessel Morphology from Contrasted CT Images. Rapidly Adaptable Legged Robots via Evolutio …
Generalized Bayesian Filtering via Sequential Monte Carlo
| Title | Generalized Bayesian Filtering via Sequential Monte Carlo |
| Authors | Ayman Boustati, Ömer Deniz Akyildiz, Theodoros Damoulas, Adam Johansen |
| Abstract | We introduce a framework for inference in general state-space hidden Markov models (HMMs) under likelihood misspecification. In particular, we leverage the loss-theoretic perspective of generalized Bayesian inference (GBI) to define generalized filtering recursions in HMMs, that can tackle the problem of inference under model misspecification. In doing so, we arrive at principled procedures for robust inference against observation contamination through the $\beta$-divergence. Operationalizing the proposed framework is made possible via sequential Monte Carlo methods (SMC). The standard particle methods, and their associated convergence results, are readily generalized to the new setting. We demonstrate our approach to object tracking and Gaussian process regression problems, and observe improved performance over standard filtering algorithms. |
| Tasks | Bayesian Inference, Object Tracking |
| Published | 2020-02-23 |
| URL | https://arxiv.org/abs/2002.09998v1 |
| https://arxiv.org/pdf/2002.09998v1.pdf | |
| PWC | https://paperswithcode.com/paper/generalized-bayesian-filtering-via-sequential |
| Repo | |
| Framework | |
Robust Wireless Fingerprinting: Generalizing Across Space and Time
| Title | Robust Wireless Fingerprinting: Generalizing Across Space and Time |
| Authors | Metehan Cekic, Soorya Gopalakrishnan, Upamanyu Madhow |
| Abstract | Can we distinguish between two wireless transmitters sending exactly the same message, using the same protocol? The opportunity for doing so arises due to subtle nonlinear variations across transmitters, even those made by the same manufacturer. Since these effects are difficult to model explicitly, we investigate learning device fingerprints using complex-valued deep neural networks (DNNs) that take as input the complex baseband signal at the receiver. Such fingerprints should be robust to ID spoofing, and to distribution shifts across days and locations due to clock drift and variations in the wireless channel. In this paper, we point out that, unless proactively discouraged from doing so, DNNs learn these strong confounding features rather than the subtle nonlinear characteristics that are the basis for stable signatures. Thus, a network trained on data collected during one day performs poorly on a different day, and networks allowed access to post-preamble information rely on easily-spoofed ID fields. We propose and evaluate strategies, based on augmentation and estimation, to promote generalization across realizations of these confounding factors, using data from WiFi and ADS-B protocols. We conclude that, while DNN training has the advantage of not requiring explicit signal models, significant modeling insights are required to focus the learning on the effects we wish to capture. |
| Tasks | |
| Published | 2020-02-25 |
| URL | https://arxiv.org/abs/2002.10791v1 |
| https://arxiv.org/pdf/2002.10791v1.pdf | |
| PWC | https://paperswithcode.com/paper/robust-wireless-fingerprinting-generalizing |
| Repo | |
| Framework | |
On Unbalanced Optimal Transport: An Analysis of Sinkhorn Algorithm
| Title | On Unbalanced Optimal Transport: An Analysis of Sinkhorn Algorithm |
| Authors | Khiem Pham, Khang Le, Nhat Ho, Tung Pham, Hung Bui |
| Abstract | We provide a computational complexity analysis for the Sinkhorn algorithm that solves the entropic regularized Unbalanced Optimal Transport (UOT) problem between two measures of possibly different masses with at most $n$ components. We show that the complexity of the Sinkhorn algorithm for finding an $\varepsilon$-approximate solution to the UOT problem is of order $\widetilde{\mathcal{O}}(n^2/ \varepsilon)$, which is near-linear time. To the best of our knowledge, this complexity is better than the complexity of the Sinkhorn algorithm for solving the Optimal Transport (OT) problem, which is of order $\widetilde{\mathcal{O}}(n^2/\varepsilon^2)$. Our proof technique is based on the geometric convergence of the Sinkhorn updates to the optimal dual solution of the entropic regularized UOT problem and some properties of the primal solution. It is also different from the proof for the complexity of the Sinkhorn algorithm for approximating the OT problem since the UOT solution does not have to meet the marginal constraints. |
| Tasks | |
| Published | 2020-02-09 |
| URL | https://arxiv.org/abs/2002.03293v1 |
| https://arxiv.org/pdf/2002.03293v1.pdf | |
| PWC | https://paperswithcode.com/paper/on-unbalanced-optimal-transport-an-analysis |
| Repo | |
| Framework | |
Graph Attention Network based Pruning for Reconstructing 3D Liver Vessel Morphology from Contrasted CT Images
| Title | Graph Attention Network based Pruning for Reconstructing 3D Liver Vessel Morphology from Contrasted CT Images |
| Authors | Donghao Zhang, Siqi Liu, Shikha Chaganti, Eli Gibson, Zhoubing Xu, Sasa Grbic, Weidong Cai, Dorin Comaniciu |
| Abstract | With the injection of contrast material into blood vessels, multi-phase contrasted CT images can enhance the visibility of vessel networks in the human body. Reconstructing the 3D geometric morphology of liver vessels from the contrasted CT images can enable multiple liver preoperative surgical planning applications. Automatic reconstruction of liver vessel morphology remains a challenging problem due to the morphological complexity of liver vessels and the inconsistent vessel intensities among different multi-phase contrasted CT images. On the other side, high integrity is required for the 3D reconstruction to avoid decision making biases. In this paper, we propose a framework for liver vessel morphology reconstruction using both a fully convolutional neural network and a graph attention network. A fully convolutional neural network is first trained to produce the liver vessel centerline heatmap. An over-reconstructed liver vessel graph model is then traced based on the heatmap using an image processing based algorithm. We use a graph attention network to prune the false-positive branches by predicting the presence probability of each segmented branch in the initial reconstruction using the aggregated CNN features. We evaluated the proposed framework on an in-house dataset consisting of 418 multi-phase abdomen CT images with contrast. The proposed graph network pruning improves the overall reconstruction F1 score by 6.4% over the baseline. It also outperformed the other state-of-the-art curvilinear structure reconstruction algorithms. |
| Tasks | 3D Reconstruction, Decision Making, Network Pruning |
| Published | 2020-03-18 |
| URL | https://arxiv.org/abs/2003.07999v1 |
| https://arxiv.org/pdf/2003.07999v1.pdf | |
| PWC | https://paperswithcode.com/paper/graph-attention-network-based-pruning-for |
| Repo | |
| Framework | |
Rapidly Adaptable Legged Robots via Evolutionary Meta-Learning
| Title | Rapidly Adaptable Legged Robots via Evolutionary Meta-Learning |
| Authors | Xingyou Song, Yuxiang Yang, Krzysztof Choromanski, Ken Caluwaerts, Wenbo Gao, Chelsea Finn, Jie Tan |
| Abstract | Learning adaptable policies is crucial for robots to operate autonomously in our complex and quickly changing world. In this work, we present a new meta-learning method that allows robots to quickly adapt to changes in dynamics. In contrast to gradient-based meta-learning algorithms that rely on second-order gradient estimation, we introduce a more noise-tolerant Batch Hill-Climbing adaptation operator and combine it with meta-learning based on evolutionary strategies. Our method significantly improves adaptation to changes in dynamics in high noise settings, which are common in robotics applications. We validate our approach on a quadruped robot that learns to walk while subject to changes in dynamics. We observe that our method significantly outperforms prior gradient-based approaches, enabling the robot to adapt its policy to changes based on less than 3 minutes of real data. |
| Tasks | Legged Robots, Meta-Learning |
| Published | 2020-03-02 |
| URL | https://arxiv.org/abs/2003.01239v1 |
| https://arxiv.org/pdf/2003.01239v1.pdf | |
| PWC | https://paperswithcode.com/paper/rapidly-adaptable-legged-robots-via |
| Repo | |
| Framework | |
Towards Augmented Reality-based Suturing in Monocular Laparoscopic Training
| Title | Towards Augmented Reality-based Suturing in Monocular Laparoscopic Training |
| Authors | Chandrakanth Jayachandran Preetha, Jonathan Kloss, Fabian Siegfried Wehrtmann, Lalith Sharan, Carolyn Fan, Beat Peter Müller-Stich, Felix Nickel, Sandy Engelhardt |
| Abstract | Minimally Invasive Surgery (MIS) techniques have gained rapid popularity among surgeons since they offer significant clinical benefits including reduced recovery time and diminished post-operative adverse effects. However, conventional endoscopic systems output monocular video which compromises depth perception, spatial orientation and field of view. Suturing is one of the most complex tasks performed under these circumstances. Key components of this tasks are the interplay between needle holder and the surgical needle. Reliable 3D localization of needle and instruments in real time could be used to augment the scene with additional parameters that describe their quantitative geometric relation, e.g. the relation between the estimated needle plane and its rotation center and the instrument. This could contribute towards standardization and training of basic skills and operative techniques, enhance overall surgical performance, and reduce the risk of complications. The paper proposes an Augmented Reality environment with quantitative and qualitative visual representations to enhance laparoscopic training outcomes performed on a silicone pad. This is enabled by a multi-task supervised deep neural network which performs multi-class segmentation and depth map prediction. Scarcity of labels has been conquered by creating a virtual environment which resembles the surgical training scenario to generate dense depth maps and segmentation maps. The proposed convolutional neural network was tested on real surgical training scenarios and showed to be robust to occlusion of the needle. The network achieves a dice score of 0.67 for surgical needle segmentation, 0.81 for needle holder instrument segmentation and a mean absolute error of 6.5 mm for depth estimation. |
| Tasks | Depth Estimation |
| Published | 2020-01-19 |
| URL | https://arxiv.org/abs/2001.06894v1 |
| https://arxiv.org/pdf/2001.06894v1.pdf | |
| PWC | https://paperswithcode.com/paper/towards-augmented-reality-based-suturing-in |
| Repo | |
| Framework | |
Pruning untrained neural networks: Principles and Analysis
| Title | Pruning untrained neural networks: Principles and Analysis |
| Authors | Soufiane Hayou, Jean-Francois Ton, Arnaud Doucet, Yee Whye Teh |
| Abstract | Overparameterized neural networks display state-of-the art performance. However, there is a growing need for smaller, energy-efficient, neural networks to be able to use machine learning applications on devices with limited computational resources. A popular approach consists of using pruning techniques. While these techniques have traditionally focused on pruning pre-trained neural networks (e.g. LeCun et al. (1990) and Hassabi et al. (1993)), recent work by Lee et al. (2018) showed promising results where pruning is performed at initialization. However, such procedures remain unsatisfactory as the resulting pruned networks can be difficult to train and, for instance, these procedures do not prevent one layer being fully pruned. In this paper we provide a comprehensive theoretical analysis of pruning at initialization and training sparse architectures. This analysis allows us to propose novel principled approaches which we validate experimentally on a variety of network architectures. We particularly show that we can prune up to 99.9% of the weights while keeping the model trainable. |
| Tasks | |
| Published | 2020-02-19 |
| URL | https://arxiv.org/abs/2002.08797v1 |
| https://arxiv.org/pdf/2002.08797v1.pdf | |
| PWC | https://paperswithcode.com/paper/pruning-untrained-neural-networks-principles |
| Repo | |
| Framework | |
Layer-wise Conditioning Analysis in Exploring the Learning Dynamics of DNNs
| Title | Layer-wise Conditioning Analysis in Exploring the Learning Dynamics of DNNs |
| Authors | Lei Huang, Jie Qin, Li Liu, Fan Zhu, Ling Shao |
| Abstract | Conditioning analysis uncovers the landscape of an optimization objective by exploring the spectrum of its curvature matrix. This has been well explored theoretically for linear models. We extend this analysis to deep neural networks (DNNs) in order to investigate their learning dynamics. To this end, we propose layer-wise conditioning analysis, which explores the optimization landscape with respect to each layer independently. Such an analysis is theoretically supported under mild assumptions that approximately hold in practice. Based on our analysis, we show that batch normalization (BN) can stabilize the training, but sometimes result in the false impression of a local minimum, which has detrimental effects on the learning. Besides, we experimentally observe that BN can improve the layer-wise conditioning of the optimization problem. Finally, we find that the last linear layer of a very deep residual network displays ill-conditioned behavior. We solve this problem by only adding one BN layer before the last linear layer, which achieves improved performance over the original and pre-activation residual networks. |
| Tasks | |
| Published | 2020-02-25 |
| URL | https://arxiv.org/abs/2002.10801v2 |
| https://arxiv.org/pdf/2002.10801v2.pdf | |
| PWC | https://paperswithcode.com/paper/exploring-learning-dynamics-of-dnns-via |
| Repo | |
| Framework | |
Thresholds of descending algorithms in inference problems
| Title | Thresholds of descending algorithms in inference problems |
| Authors | Stefano Sarao Mannelli, Lenka Zdeborova |
| Abstract | We review recent works on analyzing the dynamics of gradient-based algorithms in a prototypical statistical inference problem. Using methods and insights from the physics of glassy systems, these works showed how to understand quantitatively and qualitatively the performance of gradient-based algorithms. Here we review the key results and their interpretation in non-technical terms accessible to a wide audience of physicists in the context of related works. |
| Tasks | |
| Published | 2020-01-02 |
| URL | https://arxiv.org/abs/2001.00479v2 |
| https://arxiv.org/pdf/2001.00479v2.pdf | |
| PWC | https://paperswithcode.com/paper/thresholds-of-descending-algorithms-in |
| Repo | |
| Framework | |
NPENAS: Neural Predictor Guided Evolution for Neural Architecture Search
| Title | NPENAS: Neural Predictor Guided Evolution for Neural Architecture Search |
| Authors | Chen Wei, Chuang Niu, Yiping Tang, Jimin Liang |
| Abstract | Neural architecture search (NAS) is a promising method for automatically finding excellent architectures.Commonly used search strategies such as evolutionary algorithm, Bayesian optimization, and Predictor method employs a predictor to rank sampled architectures. In this paper, we propose two predictor based algorithms NPUBO and NPENAS for neural architecture search. Firstly we propose NPUBO which takes a neural predictor with uncertainty estimation as surrogate model for Bayesian optimization. Secondly we propose a simple and effective predictor guided evolution algorithm(NPENAS), which uses neural predictor to guide evolutionary algorithm to perform selection and mutation. Finally we analyse the architecture sampling pipeline and find that mostly used random sampling pipeline tends to generate architectures in a subspace of the real underlying search space. Our proposed methods can find architecture achieves high test accuracy which is comparable with recently proposed methods on NAS-Bench-101 and NAS-Bench-201 dataset using less training and evaluated samples. Code will be publicly available after finish all the experiments. |
| Tasks | Neural Architecture Search |
| Published | 2020-03-28 |
| URL | https://arxiv.org/abs/2003.12857v1 |
| https://arxiv.org/pdf/2003.12857v1.pdf | |
| PWC | https://paperswithcode.com/paper/npenas-neural-predictor-guided-evolution-for |
| Repo | |
| Framework | |
The Value of Nullspace Tuning Using Partial Label Information
| Title | The Value of Nullspace Tuning Using Partial Label Information |
| Authors | Colin B. Hansen, Vishwesh Nath, Diego A. Mesa, Yuankai Huo, Bennett A. Landman, Thomas A. Lasko |
| Abstract | In semi-supervised learning, information from unlabeled examples is used to improve the model learned from labeled examples. But in some learning problems, partial label information can be inferred from otherwise unlabeled examples and used to further improve the model. In particular, partial label information exists when subsets of training examples are known to have the same label, even though the label itself is missing. By encouraging a model to give the same label to all such examples, we can potentially improve its performance. We call this encouragement \emph{Nullspace Tuning} because the difference vector between any pair of examples with the same label should lie in the nullspace of a linear model. In this paper, we investigate the benefit of using partial label information using a careful comparison framework over well-characterized public datasets. We show that the additional information provided by partial labels reduces test error over good semi-supervised methods usually by a factor of 2, up to a factor of 5.5 in the best case. We also show that adding Nullspace Tuning to the newer and state-of-the-art MixMatch method decreases its test error by up to a factor of 1.8. |
| Tasks | |
| Published | 2020-03-17 |
| URL | https://arxiv.org/abs/2003.07921v1 |
| https://arxiv.org/pdf/2003.07921v1.pdf | |
| PWC | https://paperswithcode.com/paper/the-value-of-nullspace-tuning-using-partial |
| Repo | |
| Framework | |
DA-NAS: Data Adapted Pruning for Efficient Neural Architecture Search
| Title | DA-NAS: Data Adapted Pruning for Efficient Neural Architecture Search |
| Authors | Xiyang Dai, Dongdong Chen, Mengchen Liu, Yinpeng Chen, Lu Yuan |
| Abstract | Efficient search is a core issue in Neural Architecture Search (NAS). It is difficult for conventional NAS algorithms to directly search the architectures on large-scale tasks like ImageNet. In general, the cost of GPU hours for NAS grows with regard to training dataset size and candidate set size. One common way is searching on a smaller proxy dataset (e.g., CIFAR-10) and then transferring to the target task (e.g., ImageNet). These architectures optimized on proxy data are not guaranteed to be optimal on the target task. Another common way is learning with a smaller candidate set, which may require expert knowledge and indeed betrays the essence of NAS. In this paper, we present DA-NAS that can directly search the architecture for large-scale target tasks while allowing a large candidate set in a more efficient manner. Our method is based on an interesting observation that the learning speed for blocks in deep neural networks is related to the difficulty of recognizing distinct categories. We carefully design a progressive data adapted pruning strategy for efficient architecture search. It will quickly trim low performed blocks on a subset of target dataset (e.g., easy classes), and then gradually find the best blocks on the whole target dataset. At this time, the original candidate set becomes as compact as possible, providing a faster search in the target task. Experiments on ImageNet verify the effectiveness of our approach. It is 2x faster than previous methods while the accuracy is currently state-of-the-art, at 76.2% under small FLOPs constraint. It supports an argument search space (i.e., more candidate blocks) to efficiently search the best-performing architecture. |
| Tasks | Neural Architecture Search |
| Published | 2020-03-27 |
| URL | https://arxiv.org/abs/2003.12563v1 |
| https://arxiv.org/pdf/2003.12563v1.pdf | |
| PWC | https://paperswithcode.com/paper/da-nas-data-adapted-pruning-for-efficient |
| Repo | |
| Framework | |
ASFD: Automatic and Scalable Face Detector
| Title | ASFD: Automatic and Scalable Face Detector |
| Authors | Bin Zhang, Jian Li, Yabiao Wang, Ying Tai, Chengjie Wang, Jilin Li, Feiyue Huang, Yili Xia, Wenjiang Pei, Rongrong Ji |
| Abstract | In this paper, we propose a novel Automatic and Scalable Face Detector (ASFD), which is based on a combination of neural architecture search techniques as well as a new loss design. First, we propose an automatic feature enhance module named Auto-FEM by improved differential architecture search, which allows efficient multi-scale feature fusion and context enhancement. Second, we use Distance-based Regression and Margin-based Classification (DRMC) multi-task loss to predict accurate bounding boxes and learn highly discriminative deep features. Third, we adopt compound scaling methods and uniformly scale the backbone, feature modules, and head networks to develop a family of ASFD, which are consistently more efficient than the state-of-the-art face detectors. Extensive experiments conducted on popular benchmarks, e.g. WIDER FACE and FDDB, demonstrate that our ASFD-D6 outperforms the prior strong competitors, and our lightweight ASFD-D0 runs at more than 120 FPS with Mobilenet for VGA-resolution images. |
| Tasks | Neural Architecture Search |
| Published | 2020-03-25 |
| URL | https://arxiv.org/abs/2003.11228v3 |
| https://arxiv.org/pdf/2003.11228v3.pdf | |
| PWC | https://paperswithcode.com/paper/asfd-automatic-and-scalable-face-detector |
| Repo | |
| Framework | |
A New Multiple Max-pooling Integration Module and Cross Multiscale Deconvolution Network Based on Image Semantic Segmentation
| Title | A New Multiple Max-pooling Integration Module and Cross Multiscale Deconvolution Network Based on Image Semantic Segmentation |
| Authors | Hongfeng You, Shengwei Tian, Long Yu, Xiang Ma, Yan Xing, Ning Xin |
| Abstract | To better retain the deep features of an image and solve the sparsity problem of the end-to-end segmentation model, we propose a new deep convolutional network model for medical image pixel segmentation, called MC-Net. The core of this network model consists of four parts, namely, an encoder network, a multiple max-pooling integration module, a cross multiscale deconvolution decoder network and a pixel-level classification layer. In the network structure of the encoder, we use multiscale convolution instead of the traditional single-channel convolution. The multiple max-pooling integration module first integrates the output features of each submodule of the encoder network and reduces the number of parameters by convolution using a kernel size of 1. At the same time, each max-pooling layer (the pooling size of each layer is different) is spliced after each convolution to achieve the translation invariance of the feature maps of each submodule. We use the output feature maps from the multiple max-pooling integration module as the input of the decoder network; the multiscale convolution of each submodule in the decoder network is cross-fused with the feature maps generated by the corresponding multiscale convolution in the encoder network. Using the above feature map processing methods solves the sparsity problem after the max-pooling layer-generating matrix and enhances the robustness of the classification. We compare our proposed model with the well-known Fully Convolutional Networks for Semantic Segmentation (FCNs), DecovNet, PSPNet, U-net, SgeNet and other state-of-the-art segmentation networks such as HyperDenseNet, MS-Dual, Espnetv2, Denseaspp using one binary Kaggle 2018 data science bowl dataset and two multiclass dataset and obtain encouraging experimental results. |
| Tasks | Semantic Segmentation |
| Published | 2020-03-25 |
| URL | https://arxiv.org/abs/2003.11213v1 |
| https://arxiv.org/pdf/2003.11213v1.pdf | |
| PWC | https://paperswithcode.com/paper/a-new-multiple-max-pooling-integration-module |
| Repo | |
| Framework | |
Run-time Deep Model Multiplexing
| Title | Run-time Deep Model Multiplexing |
| Authors | Amir Erfan Eshratifar, Massoud Pedram |
| Abstract | We propose a framework to design a light-weight neural multiplexer that given input and resource budgets, decides upon the appropriate model to be called for the inference. Mobile devices can use this framework to offload the hard inputs to the cloud while inferring the easy ones locally. Besides, in the large scale cloud-based intelligent applications, instead of replicating the most-accurate model, a range of small and large models can be multiplexed from depending on the input’s complexity and resource budgets. Our experimental results demonstrate the effectiveness of our framework benefiting both mobile users and cloud providers. |
| Tasks | |
| Published | 2020-01-14 |
| URL | https://arxiv.org/abs/2001.05870v1 |
| https://arxiv.org/pdf/2001.05870v1.pdf | |
| PWC | https://paperswithcode.com/paper/run-time-deep-model-multiplexing |
| Repo | |
| Framework | |
