April 1, 2020

3288 words 16 mins read

Paper Group ANR 495

Ranger: Boosting Error Resilience of Deep Neural Networks through Range Restriction. Faster than FAST: GPU-Accelerated Frontend for High-Speed VIO. Enhancing Feature Invariance with Learned Image Transformations for Image Retrieval. Semi-Federated Learning. QC-SPHRAM: Quasi-conformal Spherical Harmonics Based Geometric Distortions on Hippocampal Su …

Ranger: Boosting Error Resilience of Deep Neural Networks through Range Restriction


Title	Ranger: Boosting Error Resilience of Deep Neural Networks through Range Restriction
Authors	Zitao Chen, Guanpeng Li, Karthik Pattabiraman
Abstract	With the emerging adoption of deep neural networks (DNNs) in the HPC domain, the reliability of DNNs is also growing in importance. As prior studies demonstrate the vulnerability of DNNs to hardware transient faults (i.e., soft errors), there is a compelling need for an efficient technique to protect DNNs from soft errors. While the inherent resilience of DNNs can tolerate some transient faults (which would not affect the system’s output), prior work has found there are critical faults that cause safety violations (e.g., misclassification). In this work, we exploit the inherent resilience of DNNs to protect the DNNs from critical faults. In particular, we propose Ranger, an automated technique to selectively restrict the ranges of values in particular DNN layers, which can dampen the large deviations typically caused by critical faults to smaller ones. Such reduced deviations can usually be tolerated by the inherent resilience of DNNs. Ranger can be integrated into existing DNNs without retraining, and with minimal effort. Our evaluation on 8 DNNs (including two used in self-driving car applications) demonstrates that Ranger can achieve significant resilience boosting without degrading the accuracy of the model, and incurring negligible overheads.
Tasks
Published	2020-03-30
URL	https://arxiv.org/abs/2003.13874v1
PDF	https://arxiv.org/pdf/2003.13874v1.pdf
PWC	https://paperswithcode.com/paper/ranger-boosting-error-resilience-of-deep
Repo
Framework

Faster than FAST: GPU-Accelerated Frontend for High-Speed VIO


Title	Faster than FAST: GPU-Accelerated Frontend for High-Speed VIO
Authors	Balazs Nagy, Philipp Foehn, Davide Scaramuzza
Abstract	The recent introduction of powerful embedded graphics processing units (GPUs) has allowed for unforeseen improvements in real-time computer vision applications. It has enabled algorithms to run onboard, well above the standard video rates, yielding not only higher information processing capability, but also reduced latency. This work focuses on the applicability of efficient low-level, GPU hardware-specific instructions to improve on existing computer vision algorithms in the field of visual-inertial odometry (VIO). While most steps of a VIO pipeline work on visual features, they rely on image data for detection and tracking, of which both steps are well suited for parallelization. Especially non-maxima suppression and the subsequent feature selection are prominent contributors to the overall image processing latency. Our work first revisits the problem of non-maxima suppression for feature detection specifically on GPUs, and proposes a solution that selects local response maxima, imposes spatial feature distribution, and extracts features simultaneously. Our second contribution introduces an enhanced FAST feature detector that applies the aforementioned non-maxima suppression method. Finally, we compare our method to other state-of-the-art CPU and GPU implementations, where we always outperform all of them in feature tracking and detection, resulting in over 1000fps throughput on an embedded Jetson TX2 platform. Additionally, we demonstrate our work integrated in a VIO pipeline achieving a metric state estimation at ~200fps.
Tasks	Feature Selection
Published	2020-03-30
URL	https://arxiv.org/abs/2003.13493v1
PDF	https://arxiv.org/pdf/2003.13493v1.pdf
PWC	https://paperswithcode.com/paper/faster-than-fast-gpu-accelerated-frontend-for
Repo
Framework

Enhancing Feature Invariance with Learned Image Transformations for Image Retrieval


Title	Enhancing Feature Invariance with Learned Image Transformations for Image Retrieval
Authors	Osman Tursun, Simon Denman, Sridha Sridharan, Clinton Fookes
Abstract	Off-the-shelf convolutional neural network features achieve state-of-the-art results in many image retrieval tasks. However, their invariance is pre-defined by the network architecture and training data. In this work, we propose using features aggregated from transformed images to increase the invariance of off-the-shelf features without fine-tuning or modifying the network. We learn an ensemble of beneficial image transformations through reinforcement learning in an efficient way. Experiment results show the learned ensemble of transformations is effective and transferable.
Tasks	Image Retrieval
Published	2020-02-05
URL	https://arxiv.org/abs/2002.01642v1
PDF	https://arxiv.org/pdf/2002.01642v1.pdf
PWC	https://paperswithcode.com/paper/enhancing-feature-invariance-with-learned
Repo
Framework

Semi-Federated Learning


Title	Semi-Federated Learning
Authors	Zhikun Chen, Daofeng Li, Ming Zhao, Sihai Zhang, Jinkang Zhu
Abstract	Federated learning (FL) enables massive distributed Information and Communication Technology (ICT) devices to learn a global consensus model without any participants revealing their own data to the central server. However, the practicality, communication expense and non-independent and identical distribution (Non-IID) data challenges in FL still need to be concerned. In this work, we propose the Semi-Federated Learning (Semi-FL) which differs from the FL in two aspects, local clients clustering and in-cluster training. A sequential training manner is designed for our in-cluster training in this paper which enables the neighboring clients to share their learning models. The proposed Semi-FL can be easily applied to future mobile communication networks and require less up-link transmission bandwidth. Numerical experiments validate the feasibility, learning performance and the robustness to Non-IID data of the proposed Semi-FL. The Semi-FL extends the existing potentials of FL.
Tasks
Published	2020-03-28
URL	https://arxiv.org/abs/2003.12795v1
PDF	https://arxiv.org/pdf/2003.12795v1.pdf
PWC	https://paperswithcode.com/paper/semi-federated-learning
Repo
Framework

QC-SPHRAM: Quasi-conformal Spherical Harmonics Based Geometric Distortions on Hippocampal Surfaces for Early Detection of the Alzheimer’s Disease


Title	QC-SPHRAM: Quasi-conformal Spherical Harmonics Based Geometric Distortions on Hippocampal Surfaces for Early Detection of the Alzheimer’s Disease
Authors	Anthony Hei-Long Chan, Yishan Luo, Lin Shi, Ronald Lok-Ming Lui
Abstract	We propose a disease classification model, called the QC-SPHARM, for the early detection of the Alzheimer’s Disease (AD). The proposed QC-SPHARM can distinguish between normal control (NC) subjects and AD patients, as well as between amnestic mild cognitive impairment (aMCI) patients having high possibility progressing into AD and those who do not. Using the spherical harmonics (SPHARM) based registration, hippocampal surfaces segmented from the ADNI data are individually registered to a template surface constructed from the NC subjects using SPHARM. Local geometric distortions of the deformation from the template surface to each subject are quantified in terms of conformality distortions and curvatures distortions. The measurements are combined with the spherical harmonics coefficients and the total volume change of the subject from the template. Afterwards, a t-test based feature selection method incorporating the bagging strategy is applied to extract those local regions having high discriminating power of the two classes. The disease diagnosis machine can therefore be built using the data under the Support Vector Machine (SVM) setting. Using 110 NC subjects and 110 AD patients from the ADNI database, the proposed algorithm achieves 85:2% testing accuracy on 80 random samples as testing subjects, with the incorporation of surface geometry in the classification machine. Using 20 aMCI patients who has advanced to AD during a two-year period and another 20 aMCI patients who remain non-AD for the next two years, the algorithm achieves 81:2% accuracy using 10 randomly picked subjects as testing data. Our proposed method is 6%-15% better than other classification models without the incorporation of surface geometry. The results demonstrate the advantages of using local geometric distortions as the discriminating criterion for early AD diagnosis.
Tasks	Feature Selection
Published	2020-03-20
URL	https://arxiv.org/abs/2003.10229v1
PDF	https://arxiv.org/pdf/2003.10229v1.pdf
PWC	https://paperswithcode.com/paper/qc-sphram-quasi-conformal-spherical-harmonics
Repo
Framework

Polarized Reflection Removal with Perfect Alignment in the Wild


Title	Polarized Reflection Removal with Perfect Alignment in the Wild
Authors	Chenyang Lei, Xuhua Huang, Mengdi Zhang, Qiong Yan, Wenxiu Sun, Qifeng Chen
Abstract	We present a novel formulation to removing reflection from polarized images in the wild. We first identify the misalignment issues of existing reflection removal datasets where the collected reflection-free images are not perfectly aligned with input mixed images due to glass refraction. Then we build a new dataset with more than 100 types of glass in which obtained transmission images are perfectly aligned with input mixed images. Second, capitalizing on the special relationship between reflection and polarized light, we propose a polarized reflection removal model with a two-stage architecture. In addition, we design a novel perceptual NCC loss that can improve the performance of reflection removal and general image decomposition tasks. We conduct extensive experiments, and results suggest that our model outperforms state-of-the-art methods on reflection removal.
Tasks
Published	2020-03-28
URL	https://arxiv.org/abs/2003.12789v1
PDF	https://arxiv.org/pdf/2003.12789v1.pdf
PWC	https://paperswithcode.com/paper/polarized-reflection-removal-with-perfect
Repo
Framework

Pursuing Sources of Heterogeneity in Modeling Clustered Population


Title	Pursuing Sources of Heterogeneity in Modeling Clustered Population
Authors	Yan Li, Chun Yu, Yize Zhao, Robert H. Aseltine, Weixin Yao, Kun Chen
Abstract	Researchers often have to deal with heterogeneous population with mixed regression relationships, increasingly so in the era of data explosion. In such problems, when there are many candidate predictors, it is not only of interest to identify the predictors that are associated with the outcome, but also to distinguish the true sources of heterogeneity, i.e., to identify the predictors that have different effects among the clusters and thus are the true contributors to the formation of the clusters. We clarify the concepts of the source of heterogeneity that account for potential scale differences of the clusters and propose a regularized finite mixture effects regression to achieve heterogeneity pursuit and feature selection simultaneously. As the name suggests, the problem is formulated under an effects-model parameterization, in which the cluster labels are missing and the effect of each predictor on the outcome is decomposed to a common effect term and a set of cluster-specific terms. A constrained sparse estimation of these effects leads to the identification of both the variables with common effects and those with heterogeneous effects. We propose an efficient algorithm and show that our approach can achieve both estimation and selection consistency. Simulation studies further demonstrate the effectiveness of our method under various practical scenarios. Three applications are presented, namely, an imaging genetics study for linking genetic factors and brain neuroimaging traits in Alzheimer’s disease, a public health study for exploring the association between suicide risk among adolescents and their school district characteristics, and a sport analytics study for understanding how the salary levels of baseball players are associated with their performance and contractual status.
Tasks	Feature Selection
Published	2020-03-10
URL	https://arxiv.org/abs/2003.04787v1
PDF	https://arxiv.org/pdf/2003.04787v1.pdf
PWC	https://paperswithcode.com/paper/pursuing-sources-of-heterogeneity-in-modeling
Repo
Framework

Real-time Federated Evolutionary Neural Architecture Search


Title	Real-time Federated Evolutionary Neural Architecture Search
Authors	Hangyu Zhu, Yaochu Jin
Abstract	Federated learning is a distributed machine learning approach to privacy preservation and two major technical challenges prevent a wider application of federated learning. One is that federated learning raises high demands on communication, since a large number of model parameters must be transmitted between the server and the clients. The other challenge is that training large machine learning models such as deep neural networks in federated learning requires a large amount of computational resources, which may be unrealistic for edge devices such as mobile phones. The problem becomes worse when deep neural architecture search is to be carried out in federated learning. To address the above challenges, we propose an evolutionary approach to real-time federated neural architecture search that not only optimize the model performance but also reduces the local payload. During the search, a double-sampling technique is introduced, in which for each individual, a randomly sampled sub-model of a master model is transmitted to a number of randomly sampled clients for training without reinitialization. This way, we effectively reduce computational and communication costs required for evolutionary optimization and avoid big performance fluctuations of the local models, making the proposed framework well suited for real-time federated neural architecture search.
Tasks	Neural Architecture Search
Published	2020-03-04
URL	https://arxiv.org/abs/2003.02793v1
PDF	https://arxiv.org/pdf/2003.02793v1.pdf
PWC	https://paperswithcode.com/paper/real-time-federated-evolutionary-neural
Repo
Framework

Real-MFF Dataset: A Large Realistic Multi-focus Image Dataset with Ground Truth


Title	Real-MFF Dataset: A Large Realistic Multi-focus Image Dataset with Ground Truth
Authors	Juncheng Zhang, Qingmin Liao, Shaojun Liu, Haoyu Ma, Wenming Yang, Jing-hao Xue
Abstract	Multi-focus image fusion, a technique to generate an all-in-focus image from two or more source images, can benefit many computer vision tasks. However, currently there is no large and realistic dataset to perform convincing evaluation and comparison for exiting multi-focus image fusion. For deep learning methods, it is difficult to train a network without a suitable dataset. In this paper, we introduce a large and realistic multi-focus dataset containing 800 pairs of source images with the corresponding ground truth images. The dataset is generated using a light field camera, consequently, the source images as well as the ground truth images are realistic. Moreover, the dataset contains a variety of scenes, including buildings, plants, humans, shopping malls, squares and so on, to serve as a well-founded benchmark for multi-focus image fusion tasks. For illustration, we evaluate 10 typical multi-focus algorithms on this dataset.
Tasks
Published	2020-03-28
URL	https://arxiv.org/abs/2003.12779v1
PDF	https://arxiv.org/pdf/2003.12779v1.pdf
PWC	https://paperswithcode.com/paper/real-mff-dataset-a-large-realistic-multi
Repo
Framework

Differentiable Top-k Operator with Optimal Transport


Title	Differentiable Top-k Operator with Optimal Transport
Authors	Yujia Xie, Hanjun Dai, Minshuo Chen, Bo Dai, Tuo Zhao, Hongyuan Zha, Wei Wei, Tomas Pfister
Abstract	The top-k operation, i.e., finding the k largest or smallest elements from a collection of scores, is an important model component, which is widely used in information retrieval, machine learning, and data mining. However, if the top-k operation is implemented in an algorithmic way, e.g., using bubble algorithm, the resulting model cannot be trained in an end-to-end way using prevalent gradient descent algorithms. This is because these implementations typically involve swapping indices, whose gradient cannot be computed. Moreover, the corresponding mapping from the input scores to the indicator vector of whether this element belongs to the top-k set is essentially discontinuous. To address the issue, we propose a smoothed approximation, namely the SOFT (Scalable Optimal transport-based diFferenTiable) top-k operator. Specifically, our SOFT top-k operator approximates the output of the top-k operation as the solution of an Entropic Optimal Transport (EOT) problem. The gradient of the SOFT operator can then be efficiently approximated based on the optimality conditions of EOT problem. We apply the proposed operator to the k-nearest neighbors and beam search algorithms, and demonstrate improved performance.
Tasks	Information Retrieval
Published	2020-02-16
URL	https://arxiv.org/abs/2002.06504v2
PDF	https://arxiv.org/pdf/2002.06504v2.pdf
PWC	https://paperswithcode.com/paper/differentiable-top-k-operator-with-optimal
Repo
Framework

Comparison of Multi-Class and Binary Classification Machine Learning Models in Identifying Strong Gravitational Lenses


Title	Comparison of Multi-Class and Binary Classification Machine Learning Models in Identifying Strong Gravitational Lenses
Authors	Hossen Teimoorinia, Robert D. Toyonaga, Sebastien Fabbro, Connor Bottrell
Abstract	Typically, binary classification lens-finding schemes are used to discriminate between lens candidates and non-lenses. However, these models often suffer from substantial false-positive classifications. Such false positives frequently occur due to images containing objects such as crowded sources, galaxies with arms, and also images with a central source and smaller surrounding sources. Therefore, a model might confuse the stated circumstances with an Einstein ring. It has been proposed that by allowing such commonly misclassified image types to constitute their own classes, machine learning models will more easily be able to learn the difference between images that contain real lenses, and images that contain lens imposters. Using Hubble Space Telescope (HST) images, in the F814W filter, we compare the usage of binary and multi-class classification models applied to the lens finding task. From our findings, we conclude there is not a significant benefit to using the multi-class model over a binary model. We will also present the results of a simple lens search using a multi-class machine learning model, and potential new lens candidates.
Tasks
Published	2020-02-27
URL	https://arxiv.org/abs/2002.11849v1
PDF	https://arxiv.org/pdf/2002.11849v1.pdf
PWC	https://paperswithcode.com/paper/comparison-of-multi-class-and-binary
Repo
Framework

Multiple Access in Dynamic Cell-Free Networks: Outage Performance and Deep Reinforcement Learning-Based Design


Title	Multiple Access in Dynamic Cell-Free Networks: Outage Performance and Deep Reinforcement Learning-Based Design
Authors	Yasser Al-Eryani, Mohamed Akrout, Ekram Hossain
Abstract	In future cell-free (or cell-less) wireless networks, a large number of devices in a geographical area will be served simultaneously in non-orthogonal multiple access scenarios by a large number of distributed access points (APs), which coordinate with a centralized processing pool. For such a centralized cell-free network with static predefined beamforming design, we first derive a closed-form expression of the uplink per-user probability of outage. To significantly reduce the complexity of joint processing of users’ signals in presence of a large number of devices and APs, we propose a novel dynamic cell-free network architecture. In this architecture, the distributed APs are partitioned (i.e. clustered) among a set of subgroups with each subgroup acting as a virtual AP equipped with a distributed antenna system (DAS). The conventional static cell-free network is a special case of this dynamic cell-free network when the cluster size is one. For this dynamic cell-free network, we propose a successive interference cancellation (SIC)-enabled signal detection method and an inter-user-interference (IUI)-aware DAS’s receive diversity combining scheme. We then formulate the general problem of clustering APs and designing the beamforming vectors with an objective to maximizing the sum rate or maximizing the minimum rate. To this end, we propose a hybrid deep reinforcement learning (DRL) model, namely, a deep deterministic policy gradient (DDPG)-deep double Q-network (DDQN) model, to solve the optimization problem for online implementation with low complexity. The DRL model for sum-rate optimization significantly outperforms that for maximizing the minimum rate in terms of average per-user rate performance. Also, in our system setting, the proposed DDPG-DDQN scheme is found to achieve around $78%$ of the rate achievable through an exhaustive search-based design.
Tasks
Published	2020-01-29
URL	https://arxiv.org/abs/2002.02801v2
PDF	https://arxiv.org/pdf/2002.02801v2.pdf
PWC	https://paperswithcode.com/paper/multiple-access-in-dynamic-cell-free-networks
Repo
Framework

Residual Knowledge Distillation


Title	Residual Knowledge Distillation
Authors	Mengya Gao, Yujun Shen, Quanquan Li, Chen Change Loy
Abstract	Knowledge distillation (KD) is one of the most potent ways for model compression. The key idea is to transfer the knowledge from a deep teacher model (T) to a shallower student (S). However, existing methods suffer from performance degradation due to the substantial gap between the learning capacities of S and T. To remedy this problem, this work proposes Residual Knowledge Distillation (RKD), which further distills the knowledge by introducing an assistant (A). Specifically, S is trained to mimic the feature maps of T, and A aids this process by learning the residual error between them. In this way, S and A complement with each other to get better knowledge from T. Furthermore, we devise an effective method to derive S and A from a given model without increasing the total computational cost. Extensive experiments show that our approach achieves appealing results on popular classification datasets, CIFAR-100 and ImageNet, surpassing state-of-the-art methods.
Tasks	Model Compression
Published	2020-02-21
URL	https://arxiv.org/abs/2002.09168v1
PDF	https://arxiv.org/pdf/2002.09168v1.pdf
PWC	https://paperswithcode.com/paper/residual-knowledge-distillation
Repo
Framework

Layer-wise Pruning and Auto-tuning of Layer-wise Learning Rates in Fine-tuning of Deep Networks


Title	Layer-wise Pruning and Auto-tuning of Layer-wise Learning Rates in Fine-tuning of Deep Networks
Authors	Youngmin Ro, Jin Young Choi
Abstract	Existing fine-tuning methods use a single learning rate over all layers. In this paper, first, we discuss that trends of layer-wise weight variations by fine-tuning using a single learning rate do not match the well-known notion that lower-level layers extract general features and higher-level layers extract specific features. Based on our discussion, we propose an algorithm that improves fine-tuning performance and reduces network complexity through layer-wise pruning and auto-tuning of layer-wise learning rates. Through in-depth experiments on image retrieval (CUB-200-2011, Stanford online products, and Inshop) and fine-grained classification (Stanford cars, Aircraft) datasets, the effectiveness of the proposed algorithm is verified.
Tasks	Image Retrieval
Published	2020-02-14
URL	https://arxiv.org/abs/2002.06048v2
PDF	https://arxiv.org/pdf/2002.06048v2.pdf
PWC	https://paperswithcode.com/paper/layer-wise-pruning-and-auto-tuning-of-layer
Repo
Framework

Seeing the World in a Bag of Chips


Title	Seeing the World in a Bag of Chips
Authors	Jeong Joon Park, Aleksander Holynski, Steve Seitz
Abstract	We address the dual problems of novel view synthesis and environment reconstruction from hand-held RGBD sensors. Our contributions include 1) modeling highly specular objects, 2) modeling inter-reflections and Fresnel effects, and 3) enabling surface light field reconstruction with the same input needed to reconstruct shape alone. In cases where scene surface has a strong mirror-like material component, we generate highly detailed environment images, revealing room composition, objects, people, buildings, and trees visible through windows. Our approach yields state of the art view synthesis techniques, operates on low dynamic range imagery, and is robust to geometric and calibration errors.
Tasks	Calibration, Novel View Synthesis
Published	2020-01-14
URL	https://arxiv.org/abs/2001.04642v1
PDF	https://arxiv.org/pdf/2001.04642v1.pdf
PWC	https://paperswithcode.com/paper/seeing-the-world-in-a-bag-of-chips
Repo
Framework