April 2, 2020

3068 words 15 mins read

Paper Group ANR 167

Learn to Forget: User-Level Memorization Elimination in Federated Learning. A GAN-based Tunable Image Compression System. AdvectiveNet: An Eulerian-Lagrangian Fluidic reservoir for Point Cloud Processing. Novel Change of Measure Inequalities and PAC-Bayesian Bounds. Rethinking Bias-Variance Trade-off for Generalization of Neural Networks. Spherical …

Learn to Forget: User-Level Memorization Elimination in Federated Learning


Title	Learn to Forget: User-Level Memorization Elimination in Federated Learning
Authors	Yang Liu, Zhuo Ma, Ximeng Liu, Jianfeng Ma
Abstract	Federated learning is a decentralized machine learning technique that evokes widespread attention in both the research field and the real-world market. However, the current privacy-preserving federated learning scheme only provides a secure way for the users to contribute their private data but never leaves a way to withdraw the contribution to model update. Such an irreversible setting potentially breaks the regulations about data protection and increases the risk of data extraction. To resolve the problem, this paper describes a novel concept for federated learning, called memorization elimination. Based on the concept, we propose \sysname, a federated learning framework that allows the user to eliminate the memorization of its private data in the trained model. Specifically, each user in \sysname is deployed with a trainable dummy gradient generator. After steps of training, the generator can produce dummy gradients to stimulate the neurons of a machine learning model to eliminate the memorization of the specific data. Also, we prove that the additional memorization elimination service of \sysname does not break the common procedure of federated learning or lower its security.
Tasks
Published	2020-03-24
URL	https://arxiv.org/abs/2003.10933v1
PDF	https://arxiv.org/pdf/2003.10933v1.pdf
PWC	https://paperswithcode.com/paper/learn-to-forget-user-level-memorization
Repo
Framework

A GAN-based Tunable Image Compression System


Title	A GAN-based Tunable Image Compression System
Authors	Lirong Wu, Kejie Huang, Haibin Shen
Abstract	The method of importance map has been widely adopted in DNN-based lossy image compression to achieve bit allocation according to the importance of image contents. However, insufficient allocation of bits in non-important regions often leads to severe distortion at low bpp (bits per pixel), which hampers the development of efficient content-weighted image compression systems. This paper rethinks content-based compression by using Generative Adversarial Network (GAN) to reconstruct the non-important regions. Moreover, multiscale pyramid decomposition is applied to both the encoder and the discriminator to achieve global compression of high-resolution images. A tunable compression scheme is also proposed in this paper to compress an image to any specific compression ratio without retraining the model. The experimental results show that our proposed method improves MS-SSIM by more than 10.3% compared to the recently reported GAN-based method to achieve the same low bpp (0.05) on the Kodak dataset.
Tasks	Image Compression
Published	2020-01-18
URL	https://arxiv.org/abs/2001.06580v1
PDF	https://arxiv.org/pdf/2001.06580v1.pdf
PWC	https://paperswithcode.com/paper/a-gan-based-tunable-image-compression-system
Repo
Framework

AdvectiveNet: An Eulerian-Lagrangian Fluidic reservoir for Point Cloud Processing


Title	AdvectiveNet: An Eulerian-Lagrangian Fluidic reservoir for Point Cloud Processing
Authors	Xingzhe He, Helen Lu Cao, Bo Zhu
Abstract	This paper presents a novel physics-inspired deep learning approach for point cloud processing motivated by the natural flow phenomena in fluid mechanics. Our learning architecture jointly defines data in an Eulerian world space, using a static background grid, and a Lagrangian material space, using moving particles. By introducing this Eulerian-Lagrangian representation, we are able to naturally evolve and accumulate particle features using flow velocities generated from a generalized, high-dimensional force field. We demonstrate the efficacy of this system by solving various point cloud classification and segmentation problems with state-of-the-art performance. The entire geometric reservoir and data flow mimics the pipeline of the classic PIC/FLIP scheme in modeling natural flow, bridging the disciplines of geometric machine learning and physical simulation.
Tasks
Published	2020-02-01
URL	https://arxiv.org/abs/2002.00118v2
PDF	https://arxiv.org/pdf/2002.00118v2.pdf
PWC	https://paperswithcode.com/paper/advectivenet-an-eulerian-lagrangian-fluidic-1
Repo
Framework

Novel Change of Measure Inequalities and PAC-Bayesian Bounds


Title	Novel Change of Measure Inequalities and PAC-Bayesian Bounds
Authors	Yuki Ohnishi, Jean Honorio
Abstract	PAC-Bayesian theory has received a growing attention in the machine learning community. Our work extends the PAC-Bayesian theory by introducing several novel change of measure inequalities for two families of divergences: $f$-divergences and $\alpha$-divergences. First, we show how the variational representation for $f$-divergences leads to novel change of measure inequalities. Second, we propose a multiplicative change of measure inequality for $\alpha$-divergences, which leads to tighter bounds under some technical conditions. Finally, we present several PAC-Bayesian bounds for various classes of random variables, by using our novel change of measure inequalities.
Tasks
Published	2020-02-25
URL	https://arxiv.org/abs/2002.10678v1
PDF	https://arxiv.org/pdf/2002.10678v1.pdf
PWC	https://paperswithcode.com/paper/novel-change-of-measure-inequalities-and-pac
Repo
Framework

Rethinking Bias-Variance Trade-off for Generalization of Neural Networks


Title	Rethinking Bias-Variance Trade-off for Generalization of Neural Networks
Authors	Zitong Yang, Yaodong Yu, Chong You, Jacob Steinhardt, Yi Ma
Abstract	The classical bias-variance trade-off predicts that bias decreases and variance increase with model complexity, leading to a U-shaped risk curve. Recent work calls this into question for neural networks and other over-parameterized models, for which it is often observed that larger models generalize better. We provide a simple explanation for this by measuring the bias and variance of neural networks: while the bias is monotonically decreasing as in the classical theory, the variance is unimodal or bell-shaped: it increases then decreases with the width of the network. We vary the network architecture, loss function, and choice of dataset and confirm that variance unimodality occurs robustly for all models we considered. The risk curve is the sum of the bias and variance curves and displays different qualitative shapes depending on the relative scale of bias and variance, with the double descent curve observed in recent literature as a special case. We corroborate these empirical results with a theoretical analysis of two-layer linear networks with random first layer. Finally, evaluation on out-of-distribution data shows that most of the drop in accuracy comes from increased bias while variance increases by a relatively small amount. Moreover, we find that deeper models decrease bias and increase variance for both in-distribution and out-of-distribution data.
Tasks
Published	2020-02-26
URL	https://arxiv.org/abs/2002.11328v2
PDF	https://arxiv.org/pdf/2002.11328v2.pdf
PWC	https://paperswithcode.com/paper/rethinking-bias-variance-trade-off-for
Repo
Framework

Spherical formulation of moving object geometric constraints for monocular fisheye cameras


Title	Spherical formulation of moving object geometric constraints for monocular fisheye cameras
Authors	Letizia Mariotti, Ciaran Hughes
Abstract	In this paper, we introduce a moving object detection algorithm for fisheye cameras used in autonomous driving. We reformulate the three commonly used constraints in rectilinear images (epipolar, positive depth and positive height constraints) to spherical coordinates which is invariant to specific camera configuration once the calibration is known. One of the main challenging use case in autonomous driving is to detect parallel moving objects which suffer from motion-parallax ambiguity. To alleviate this, we formulate an additional fourth constraint, called the anti-parallel constraint, which aids the detection of objects with motion that mirrors the ego-vehicle possible. We analyze the proposed algorithm in different scenarios and demonstrate that it works effectively operating directly on fisheye images.
Tasks	Autonomous Driving, Calibration, Object Detection
Published	2020-03-06
URL	https://arxiv.org/abs/2003.03262v1
PDF	https://arxiv.org/pdf/2003.03262v1.pdf
PWC	https://paperswithcode.com/paper/spherical-formulation-of-moving-object
Repo
Framework

DASNet: Dual attentive fully convolutional siamese networks for change detection of high resolution satellite images


Title	DASNet: Dual attentive fully convolutional siamese networks for change detection of high resolution satellite images
Authors	Jie Chen, Ziyang Yuan, Jian Peng, Li Chen, Haozhe Huang, Jiawei Zhu, Tao Lin, Haifeng Li
Abstract	Change detection is a basic task of remote sensing image processing. The research objective is to identity the change information of interest and filter out the irrelevant change information as interference factors. Recently, the rise of deep learning has provided new tools for change detection, which have yielded impressive results. However, the available methods focus mainly on the difference information between multitemporal remote sensing images and lack robustness to pseudo-change information. To overcome the lack of resistance of current methods to pseudo-changes, in this paper, we propose a new method, namely, dual attentive fully convolutional Siamese networks (DASNet) for change detection in high-resolution images. Through the dual-attention mechanism, long-range dependencies are captured to obtain more discriminant feature representations to enhance the recognition performance of the model. Moreover, the imbalanced sample is a serious problem in change detection, i.e. unchanged samples are much more than changed samples, which is one of the main reasons resulting in pseudo-changes. We put forward the weighted double margin contrastive loss to address this problem by punishing the attention to unchanged feature pairs and increase attention to changed feature pairs. The experimental results of our method on the change detection dataset (CDD) and the building change detection dataset (BCDD) demonstrate that compared with other baseline methods, the proposed method realizes maximum improvements of 2.1% and 3.6%, respectively, in the F1 score. Our Pytorch implementation is available at https://github.com/lehaifeng/DASNet.
Tasks
Published	2020-03-07
URL	https://arxiv.org/abs/2003.03608v1
PDF	https://arxiv.org/pdf/2003.03608v1.pdf
PWC	https://paperswithcode.com/paper/dasnet-dual-attentive-fully-convolutional
Repo
Framework

Latent Space Roadmap for Visual Action Planning of Deformable and Rigid Object Manipulation


Title	Latent Space Roadmap for Visual Action Planning of Deformable and Rigid Object Manipulation
Authors	Martina Lippi, Petra Poklukar, Michael C. Welle, Anastasiia Varava, Hang Yin, Alessandro Marino, Danica Kragic
Abstract	We present a framework for visual action planning of complex manipulation tasks with high-dimensional state spaces such as manipulation of deformable objects. Planning is performed in a low-dimensional latent state space that embeds images. We define and implement a Latent Space Roadmap (LSR) which is a graph-based structure that globally captures the latent system dynamics. Our framework consists of two main components: a Visual Foresight Module (VFM) that generates a visual plan as a sequence of images, and an Action Proposal Network (APN) that predicts the actions between them. We show the effectiveness of the method on a simulated box stacking task as well as a T-shirt folding task performed with a real robot.
Tasks
Published	2020-03-19
URL	https://arxiv.org/abs/2003.08974v1
PDF	https://arxiv.org/pdf/2003.08974v1.pdf
PWC	https://paperswithcode.com/paper/latent-space-roadmap-for-visual-action
Repo
Framework

Nonlinear Equation Solving: A Faster Alternative to Feedforward Computation


Title	Nonlinear Equation Solving: A Faster Alternative to Feedforward Computation
Authors	Yang Song, Chenlin Meng, Renjie Liao, Stefano Ermon
Abstract	Feedforward computations, such as evaluating a neural network or sampling from an autoregressive model, are ubiquitous in machine learning. The sequential nature of feedforward computation, however, requires a strict order of execution and cannot be easily accelerated with parallel computing. To enable parrallelization, we frame the task of feedforward computation as solving a system of nonlinear equations. We then propose to find the solution using a Jacobi or Gauss-Seidel fixed-point iteration method, as well as hybrid methods of both. Crucially, Jacobi updates operate independently on each equation and can be executed in parallel. Our method is guaranteed to give exactly the same values as the original feedforward computation with a reduced (or equal) number of parallel iterations. Experimentally, we demonstrate the effectiveness of our approach in accelerating 1) the evaluation of DenseNets on ImageNet and 2) autoregressive sampling of MADE and PixelCNN. We are able to achieve between 1.2 and 33 speedup factors under various conditions and computation models.
Tasks
Published	2020-02-10
URL	https://arxiv.org/abs/2002.03629v1
PDF	https://arxiv.org/pdf/2002.03629v1.pdf
PWC	https://paperswithcode.com/paper/nonlinear-equation-solving-a-faster
Repo
Framework

Automatic structured variational inference


Title	Automatic structured variational inference
Authors	Luca Ambrogioni, Max Hinne, Marcel van Gerven
Abstract	The aim of probabilistic programming is to automatize every aspect of probabilistic inference in arbitrary probabilistic models (programs) so that the user can focus her attention on modeling, without dealing with ad-hoc inference methods. Gradient based automatic differentiation stochastic variational inference offers an attractive option as the default method for (differentiable) probabilistic programming as it combines high performance with high computational efficiency. However, the performance of any (parametric) variational approach depends on the choice of an appropriate variational family. Here, we introduced a fully automatic method for constructing structured variational families inspired to the closed-form update in conjugate models. These pseudo-conjugate families incorporate the forward pass of the input probabilistic program and can capture complex statistical dependencies. Pseudo-conjugate families have the same space and time complexity of the input probabilistic program and are therefore tractable in a very large class of models. We validate our automatic variational method on a wide range of high dimensional inference problems including deep learning components.
Tasks	Probabilistic Programming
Published	2020-02-03
URL	https://arxiv.org/abs/2002.00643v1
PDF	https://arxiv.org/pdf/2002.00643v1.pdf
PWC	https://paperswithcode.com/paper/automatic-structured-variational-inference
Repo
Framework

Weakly Supervised PET Tumor Detection Using Class Response


Title	Weakly Supervised PET Tumor Detection Using Class Response
Authors	Amine Amyar, Romain Modzelewski, Pierre Vera, Vincent Morard, Su Ruan
Abstract	One of the most challenges in medical imaging is the lack of data and annotated data. It is proven that classical segmentation methods such as U-NET are useful but still limited due to the lack of annotated data. Using a weakly supervised learning is a promising way to address this problem, however, it is challenging to train one model to detect and locate efficiently different type of lesions due to the huge variation in images. In this paper, we present a novel approach to locate different type of lesions in positron emission tomography (PET) images using only a class label at the image-level. First, a simple convolutional neural network classifier is trained to predict the type of cancer on two 2D MIP images. Then, a pseudo-localization of the tumor is generated using class activation maps, back-propagated and corrected in a multitask learning approach with prior knowledge, resulting in a tumor detection mask. Finally, we use the mask generated from the two 2D images to detect the tumor in the 3D image. The advantage of our proposed method consists of detecting the whole tumor volume in 3D images, using only two 2D images of PET image, and showing a very promising results. It can be used as a tool to locate very efficiently tumors in a PET scan, which is a time-consuming task for physicians. In addition, we show that our proposed method can be used to conduct a radiomics study with state of the art results.
Tasks
Published	2020-03-18
URL	https://arxiv.org/abs/2003.08337v2
PDF	https://arxiv.org/pdf/2003.08337v2.pdf
PWC	https://paperswithcode.com/paper/weakly-supervised-pet-tumor-detection
Repo
Framework

DeepURL: Deep Pose Estimation Framework for Underwater Relative Localization


Title	DeepURL: Deep Pose Estimation Framework for Underwater Relative Localization
Authors	Bharat Joshi, Md Modasshir, Travis Manderson, Hunter Damron, Marios Xanthidis, Alberto Quattrini Li, Ioannis Rekleitis, Gregory Dudek
Abstract	In this paper, we propose a real-time deep-learning approach for determining the 6D relative pose of Autonomous Underwater Vehicles (AUV) from a single image. A team of autonomous robots localizing themselves, in a communication-constrained underwater environment, is essential for many applications such as underwater exploration, mapping, multi-robot convoying, and other multi-robot tasks. Due to the profound difficulty of collecting ground truth images with accurate 6D poses underwater, this work utilizes rendered images from the Unreal Game Engine simulation for training. An image translation network is employed to bridge the gap between the rendered and the real images producing synthetic images for training. The proposed method predicts the 6D pose of an AUV from a single image as 2D image keypoints representing 8 corners of the 3D model of the AUV, and then the 6D pose in the camera coordinates is determined using RANSAC-based PnP. Experimental results in underwater environments (swimming pool and ocean) with different cameras demonstrate the robustness of the proposed technique, where the trained system decreased translation error by 75.5% and orientation error by 64.6% over the state-of-the-art methods.
Tasks	Pose Estimation
Published	2020-03-11
URL	https://arxiv.org/abs/2003.05523v2
PDF	https://arxiv.org/pdf/2003.05523v2.pdf
PWC	https://paperswithcode.com/paper/deepurl-deep-pose-estimation-framework-for
Repo
Framework

Temporal Pulses Driven Spiking Neural Network for Fast Object Recognition in Autonomous Driving


Title	Temporal Pulses Driven Spiking Neural Network for Fast Object Recognition in Autonomous Driving
Authors	Wei Wang, Shibo Zhou, Jingxi Li, Xiaohua Li, Junsong Yuan, Zhanpeng Jin
Abstract	Accurate real-time object recognition from sensory data has long been a crucial and challenging task for autonomous driving. Even though deep neural networks (DNNs) have been successfully applied in this area, most existing methods still heavily rely on the pre-processing of the pulse signals derived from LiDAR sensors, and therefore introduce additional computational overhead and considerable latency. In this paper, we propose an approach to address the object recognition problem directly with raw temporal pulses utilizing the spiking neural network (SNN). Being evaluated on various datasets (including Sim LiDAR, KITTI and DVS-barrel) derived from LiDAR and dynamic vision sensor (DVS), our proposed method has shown comparable performance as the state-of-the-art methods, while achieving remarkable time efficiency. It highlights the SNN’s great potentials in autonomous driving and related applications. To the best of our knowledge, this is the first attempt to use SNN to directly perform object recognition on raw temporal pulses.
Tasks	Autonomous Driving, Object Recognition
Published	2020-01-24
URL	https://arxiv.org/abs/2001.09220v1
PDF	https://arxiv.org/pdf/2001.09220v1.pdf
PWC	https://paperswithcode.com/paper/temporal-pulses-driven-spiking-neural-network
Repo
Framework

The Vector Poisson Channel: On the Linearity of the Conditional Mean Estimator


Title	The Vector Poisson Channel: On the Linearity of the Conditional Mean Estimator
Authors	Alex Dytso, Michael Fauss, H. Vincent Poor
Abstract	This work studies properties of the conditional mean estimator in vector Poisson noise. The main emphasis is to study conditions on prior distributions that induce linearity of the conditional mean estimator. The paper consists of two main results. The first result shows that the only distribution that induces the linearity of the conditional mean estimator is a product gamma distribution. Moreover, it is shown that the conditional mean estimator cannot be linear when the dark current parameter of the Poisson noise is non-zero. The second result produces a quantitative refinement of the first result. Specifically, it is shown that if the conditional mean estimator is close to linear in a mean squared error sense, then the prior distribution must be close to a product gamma distribution in terms of their characteristic functions. Finally, the results are compared to their Gaussian counterparts.
Tasks
Published	2020-03-19
URL	https://arxiv.org/abs/2003.08967v1
PDF	https://arxiv.org/pdf/2003.08967v1.pdf
PWC	https://paperswithcode.com/paper/the-vector-poisson-channel-on-the-linearity
Repo
Framework

RSSI-Based Hybrid Beamforming Design with Deep Learning


Title	RSSI-Based Hybrid Beamforming Design with Deep Learning
Authors	Hamed Hojatian, Vu Nguyen Ha, Jérémy Nadal, Jean-François Frigon, François Leduc-Primeau
Abstract	Hybrid beamforming is a promising technology for 5G millimetre-wave communications. However, its implementation is challenging in practical multiple-input multiple-output (MIMO) systems because non-convex optimization problems have to be solved, introducing additional latency and energy consumption. In addition, the channel-state information (CSI) must be either estimated from pilot signals or fed back through dedicated channels, introducing a large signaling overhead. In this paper, a hybrid precoder is designed based only on received signal strength indicator (RSSI) feedback from each user. A deep learning method is proposed to perform the associated optimization with reasonable complexity. Results demonstrate that the obtained sum-rates are very close to the ones obtained with full-CSI optimal but complex solutions. Finally, the proposed solution allows to greatly increase the spectral efficiency of the system when compared to existing techniques, as minimal CSI feedback is required.
Tasks
Published	2020-03-12
URL	https://arxiv.org/abs/2003.06042v1
PDF	https://arxiv.org/pdf/2003.06042v1.pdf
PWC	https://paperswithcode.com/paper/rssi-based-hybrid-beamforming-design-with
Repo
Framework