April 1, 2020

3050 words 15 mins read

Paper Group ANR 502

Paper Group ANR 502

Precise Tradeoffs in Adversarial Training for Linear Regression. RelatIF: Identifying Explanatory Training Examples via Relative Influence. UDD: An Underwater Open-sea Farm Object Detection Dataset for Underwater Robot Picking. Trained Model Fusion for Object Detection using Gating Network. Unsupervisedly Learned Representations: Should the Quest b …

Precise Tradeoffs in Adversarial Training for Linear Regression

Title Precise Tradeoffs in Adversarial Training for Linear Regression
Authors Adel Javanmard, Mahdi Soltanolkotabi, Hamed Hassani
Abstract Despite breakthrough performance, modern learning models are known to be highly vulnerable to small adversarial perturbations in their inputs. While a wide variety of recent \emph{adversarial training} methods have been effective at improving robustness to perturbed inputs (robust accuracy), often this benefit is accompanied by a decrease in accuracy on benign inputs (standard accuracy), leading to a tradeoff between often competing objectives. Complicating matters further, recent empirical evidence suggest that a variety of other factors (size and quality of training data, model size, etc.) affect this tradeoff in somewhat surprising ways. In this paper we provide a precise and comprehensive understanding of the role of adversarial training in the context of linear regression with Gaussian features. In particular, we characterize the fundamental tradeoff between the accuracies achievable by any algorithm regardless of computational power or size of the training data. Furthermore, we precisely characterize the standard/robust accuracy and the corresponding tradeoff achieved by a contemporary mini-max adversarial training approach in a high-dimensional regime where the number of data points and the parameters of the model grow in proportion to each other. Our theory for adversarial training algorithms also facilitates the rigorous study of how a variety of factors (size and quality of training data, model overparametrization etc.) affect the tradeoff between these two competing accuracies.
Published 2020-02-24
URL https://arxiv.org/abs/2002.10477v1
PDF https://arxiv.org/pdf/2002.10477v1.pdf
PWC https://paperswithcode.com/paper/precise-tradeoffs-in-adversarial-training-for

RelatIF: Identifying Explanatory Training Examples via Relative Influence

Title RelatIF: Identifying Explanatory Training Examples via Relative Influence
Authors Elnaz Barshan, Marc-Etienne Brunet, Gintare Karolina Dziugaite
Abstract In this work, we focus on the use of influence functions to identify relevant training examples that one might hope “explain” the predictions of a machine learning model. One shortcoming of influence functions is that the training examples deemed most “influential” are often outliers or mislabelled, making them poor choices for explanation. In order to address this shortcoming, we separate the role of global versus local influence. We introduce RelatIF, a new class of criteria for choosing relevant training examples by way of an optimization objective that places a constraint on global influence. RelatIF considers the local influence that an explanatory example has on a prediction relative to its global effects on the model. In empirical evaluations, we find that the examples returned by RelatIF are more intuitive when compared to those found using influence functions.
Published 2020-03-25
URL https://arxiv.org/abs/2003.11630v1
PDF https://arxiv.org/pdf/2003.11630v1.pdf
PWC https://paperswithcode.com/paper/relatif-identifying-explanatory-training

UDD: An Underwater Open-sea Farm Object Detection Dataset for Underwater Robot Picking

Title UDD: An Underwater Open-sea Farm Object Detection Dataset for Underwater Robot Picking
Authors Zhihui Wang, Chongwei Liu, Shijie Wang, Tao Tang, Yulong Tao, Caifei Yang, Haojie Li, Xing Liu, Xin Fan
Abstract To promote the development of underwater robot picking in sea farms, we propose an underwater open-sea farm object detection dataset called UDD. Concretely, UDD consists of 3 categories (seacucumber, seaurchin, and scallop) with 2227 images. To the best of our knowledge, it’s the first dataset collected in a real open-sea farm for underwater robot picking and we also propose a novel Poisson-blending-embedded Generative Adversarial Network (Poisson GAN) to overcome the class-imbalance and massive small objects issues in UDD. By utilizing Poisson GAN to change the number, position, even size of objects in UDD, we construct a large scale augmented dataset (AUDD) containing 18K images. Besides, in order to make the detector better adapted to the underwater picking environment, a dataset (Pre-trained dataset) for pre-training containing 590K images is also proposed. Finally, we design a lightweight network (UnderwaterNet) to address the problems that detecting small objects from cloudy underwater pictures and meeting the efficiency requirements in robots. Specifically, we design a depth-wise-convolution-based Multi-scale Contextual Features Fusion (MFF) block and a Multi-scale Blursampling (MBP) module to reduce the parameters of the network to 1.3M at 48FPS, without any loss on accuracy. Extensive experiments verify the effectiveness of the proposed UnderwaterNet, Poisson GAN, UDD, AUDD, and Pre-trained datasets.
Tasks Object Detection
Published 2020-03-03
URL https://arxiv.org/abs/2003.01446v1
PDF https://arxiv.org/pdf/2003.01446v1.pdf
PWC https://paperswithcode.com/paper/udd-an-underwater-open-sea-farm-object

Trained Model Fusion for Object Detection using Gating Network

Title Trained Model Fusion for Object Detection using Gating Network
Authors Tetsuo Inoshita, Yuichi Nakatani, Katsuhiko Takahashi, Asuka Ishii, Gaku Nakano
Abstract The major approaches of transfer learning in computer vision have tried to adapt the source domain to the target domain one-to-one. However, this scenario is difficult to apply to real applications such as video surveillance systems. As those systems have many cameras installed at each location regarded as source domains, it is difficult to identify the proper source domain. In this paper, we introduce a new transfer learning scenario that has various source domains and one target domain, assuming video surveillance system integration. Also, we propose a novel method for automatically producing a high accuracy model by fusing models trained at various source domains. In particular, we show how to apply a gating network to fuse source domains for object detection tasks, which is a new approach. We demonstrate the effectiveness of our method through experiments on traffic surveillance datasets.
Tasks Object Detection, Transfer Learning
Published 2020-03-03
URL https://arxiv.org/abs/2003.01288v1
PDF https://arxiv.org/pdf/2003.01288v1.pdf
PWC https://paperswithcode.com/paper/trained-model-fusion-for-object-detection

Unsupervisedly Learned Representations: Should the Quest be Over?

Title Unsupervisedly Learned Representations: Should the Quest be Over?
Authors Daniel N. Nissani
Abstract There exists a Classification accuracy gap of about 20% between our best methods of generating Unsupervisedly Learned Representations and the accuracy rates achieved by (naturally Unsupervisedly Learning) humans. We are at our fourth decade at least in search of this class of paradigms. It thus may well be that we are looking in the wrong direction. We present in this paper a possible solution to this puzzle. We demonstrate that Reinforcement Learning schemes can learn representations, which may be used for Pattern Recognition tasks such as Classification, achieving practically the same accuracy as that of humans. Our main modest contribution lies in the observations that: a. when applied to a real world environment (e.g. nature itself) Reinforcement Learning does not require labels, and thus may be considered a natural candidate for the long sought, accuracy competitive Unsupervised Learning method, and b. in contrast, when Reinforcement Learning is applied in a simulated or symbolic processing environment (e.g. a computer program) it does inherently require labels and should thus be generally classified, with some exceptions, as Supervised Learning. The corollary of these observations is that further search for Unsupervised Learning competitive paradigms which may be trained in simulated environments like many of those found in research and applications may be futile.
Published 2020-01-21
URL https://arxiv.org/abs/2001.07495v2
PDF https://arxiv.org/pdf/2001.07495v2.pdf
PWC https://paperswithcode.com/paper/unsupervisedly-learned-representations-should

Extending Class Activation Mapping Using Gaussian Receptive Field

Title Extending Class Activation Mapping Using Gaussian Receptive Field
Authors Bum Jun Kim, Gyogwon Koo, Hyeyeon Choi, Sang Woo Kim
Abstract This paper addresses the visualization task of deep learning models. To improve Class Activation Mapping (CAM) based visualization method, we offer two options. First, we propose Gaussian upsampling, an improved upsampling method that can reflect the characteristics of deep learning models. Second, we identify and modify unnatural terms in the mathematical derivation of the existing CAM studies. Based on two options, we propose Extended-CAM, an advanced CAM-based visualization method, which exhibits improved theoretical properties. Experimental results show that Extended-CAM provides more accurate visualization than the existing methods.
Published 2020-01-15
URL https://arxiv.org/abs/2001.05153v1
PDF https://arxiv.org/pdf/2001.05153v1.pdf
PWC https://paperswithcode.com/paper/extending-class-activation-mapping-using

A Heterogeneous Dynamical Graph Neural Networks Approach to Quantify Scientific Impact

Title A Heterogeneous Dynamical Graph Neural Networks Approach to Quantify Scientific Impact
Authors Fan Zhou, Xovee Xu, Ce Li, Goce Trajcevski, Ting Zhong, Kunpeng Zhang
Abstract Quantifying and predicting the long-term impact of scientific writings or individual scholars has important implications for many policy decisions, such as funding proposal evaluation and identifying emerging research fields. In this work, we propose an approach based on Heterogeneous Dynamical Graph Neural Network (HDGNN) to explicitly model and predict the cumulative impact of papers and authors. HDGNN extends heterogeneous GNNs by incorporating temporally evolving characteristics and capturing both structural properties of attributed graph and the growing sequence of citation behavior. HDGNN is significantly different from previous models in its capability of modeling the node impact in a dynamic manner while taking into account the complex relations among nodes. Experiments conducted on a real citation dataset demonstrate its superior performance of predicting the impact of both papers and authors.
Published 2020-03-26
URL https://arxiv.org/abs/2003.12042v1
PDF https://arxiv.org/pdf/2003.12042v1.pdf
PWC https://paperswithcode.com/paper/a-heterogeneous-dynamical-graph-neural

Stable Policy Optimization via Off-Policy Divergence Regularization

Title Stable Policy Optimization via Off-Policy Divergence Regularization
Authors Ahmed Touati, Amy Zhang, Joelle Pineau, Pascal Vincent
Abstract Trust Region Policy Optimization (TRPO) and Proximal Policy Optimization (PPO) are among the most successful policy gradient approaches in deep reinforcement learning (RL). While these methods achieve state-of-the-art performance across a wide range of challenging tasks, there is room for improvement in the stabilization of the policy learning and how the off-policy data are used. In this paper we revisit the theoretical foundations of these algorithms and propose a new algorithm which stabilizes the policy improvement through a proximity term that constrains the discounted state-action visitation distribution induced by consecutive policies to be close to one another. This proximity term, expressed in terms of the divergence between the visitation distributions, is learned in an off-policy and adversarial manner. We empirically show that our proposed method can have a beneficial effect on stability and improve final performance in benchmark high-dimensional control tasks.
Published 2020-03-09
URL https://arxiv.org/abs/2003.04108v1
PDF https://arxiv.org/pdf/2003.04108v1.pdf
PWC https://paperswithcode.com/paper/stable-policy-optimization-via-off-policy

DHOG: Deep Hierarchical Object Grouping

Title DHOG: Deep Hierarchical Object Grouping
Authors Luke Nicholas Darlow, Amos Storkey
Abstract Recently, a number of competitive methods have tackled unsupervised representation learning by maximising the mutual information between the representations produced from augmentations. The resulting representations are then invariant to stochastic augmentation strategies, and can be used for downstream tasks such as clustering or classification. Yet data augmentations preserve many properties of an image and so there is potential for a suboptimal choice of representation that relies on matching easy-to-find features in the data. We demonstrate that greedy or local methods of maximising mutual information (such as stochastic gradient optimisation) discover local optima of the mutual information criterion; the resulting representations are also less-ideally suited to complex downstream tasks. Earlier work has not specifically identified or addressed this issue. We introduce deep hierarchical object grouping (DHOG) that computes a number of distinct discrete representations of images in a hierarchical order, eventually generating representations that better optimise the mutual information objective. We also find that these representations align better with the downstream task of grouping into underlying object classes. We tested DHOG on unsupervised clustering, which is a natural downstream test as the target representation is a discrete labelling of the data. We achieved new state-of-the-art results on the three main benchmarks without any prefiltering or Sobel-edge detection that proved necessary for many previous methods to work. We obtain accuracy improvements of: 4.3% on CIFAR-10, 1.5% on CIFAR-100-20, and 7.2% on SVHN.
Tasks Edge Detection, Representation Learning, Unsupervised Representation Learning
Published 2020-03-13
URL https://arxiv.org/abs/2003.08821v1
PDF https://arxiv.org/pdf/2003.08821v1.pdf
PWC https://paperswithcode.com/paper/dhog-deep-hierarchical-object-grouping

Large-Scale Screening of COVID-19 from Community Acquired Pneumonia using Infection Size-Aware Classification

Title Large-Scale Screening of COVID-19 from Community Acquired Pneumonia using Infection Size-Aware Classification
Authors Feng Shi, Liming Xia, Fei Shan, Dijia Wu, Ying Wei, Huan Yuan, Huiting Jiang, Yaozong Gao, He Sui, Dinggang Shen
Abstract The worldwide spread of coronavirus disease (COVID-19) has become a threatening risk for global public health. It is of great importance to rapidly and accurately screen patients with COVID-19 from community acquired pneumonia (CAP). In this study, a total of 1658 patients with COVID-19 and 1027 patients of CAP underwent thin-section CT. All images were preprocessed to obtain the segmentations of both infections and lung fields, which were used to extract location-specific features. An infection Size Aware Random Forest method (iSARF) was proposed, in which subjects were automated categorized into groups with different ranges of infected lesion sizes, followed by random forests in each group for classification. Experimental results show that the proposed method yielded sensitivity of 0.907, specificity of 0.833, and accuracy of 0.879 under five-fold cross-validation. Large performance margins against comparison methods were achieved especially for the cases with infection size in the medium range, from 0.01% to 10%. The further inclusion of Radiomics features show slightly improvement. It is anticipated that our proposed framework could assist clinical decision making.
Tasks Decision Making
Published 2020-03-22
URL https://arxiv.org/abs/2003.09860v1
PDF https://arxiv.org/pdf/2003.09860v1.pdf
PWC https://paperswithcode.com/paper/large-scale-screening-of-covid-19-from

Multistage Model for Robust Face Alignment Using Deep Neural Networks

Title Multistage Model for Robust Face Alignment Using Deep Neural Networks
Authors Huabin Wang, Rui Cheng, Jian Zhou, Liang Tao, Hon Keung Kwan
Abstract An ability to generalize unconstrained conditions such as severe occlusions and large pose variations remains a challenging goal to achieve in face alignment. In this paper, a multistage model based on deep neural networks is proposed which takes advantage of spatial transformer networks, hourglass networks and exemplar-based shape constraints. First, a spatial transformer - generative adversarial network which consists of convolutional layers and residual units is utilized to solve the initialization issues caused by face detectors, such as rotation and scale variations, to obtain improved face bounding boxes for face alignment. Then, stacked hourglass network is employed to obtain preliminary locations of landmarks as well as their corresponding scores. In addition, an exemplar-based shape dictionary is designed to determine landmarks with low scores based on those with high scores. By incorporating face shape constraints, misaligned landmarks caused by occlusions or cluttered backgrounds can be considerably improved. Extensive experiments based on challenging benchmark datasets are performed to demonstrate the superior performance of the proposed method over other state-of-the-art methods.
Tasks Face Alignment, Robust Face Alignment
Published 2020-02-04
URL https://arxiv.org/abs/2002.01075v1
PDF https://arxiv.org/pdf/2002.01075v1.pdf
PWC https://paperswithcode.com/paper/multistage-model-for-robust-face-alignment

Provably Efficient Exploration for RL with Unsupervised Learning

Title Provably Efficient Exploration for RL with Unsupervised Learning
Authors Fei Feng, Ruosong Wang, Wotao Yin, Simon S. Du, Lin F. Yang
Abstract We study how to use unsupervised learning for efficient exploration in reinforcement learning with rich observations generated from a small number of latent states. We present a novel algorithmic framework that is built upon two components: an unsupervised learning algorithm and a no-regret reinforcement learning algorithm. We show that our algorithm provably finds a near-optimal policy with sample complexity polynomial in the number of latent states, which is significantly smaller than the number of possible observations. Our result gives theoretical justification to the prevailing paradigm of using unsupervised learning for efficient exploration [tang2017exploration,bellemare2016unifying].
Tasks Efficient Exploration
Published 2020-03-15
URL https://arxiv.org/abs/2003.06898v1
PDF https://arxiv.org/pdf/2003.06898v1.pdf
PWC https://paperswithcode.com/paper/provably-efficient-exploration-for-rl-with

Scalable Uncertainty for Computer Vision with Functional Variational Inference

Title Scalable Uncertainty for Computer Vision with Functional Variational Inference
Authors Eduardo D C Carvalho, Ronald Clark, Andrea Nicastro, Paul H J Kelly
Abstract As Deep Learning continues to yield successful applications in Computer Vision, the ability to quantify all forms of uncertainty is a paramount requirement for its safe and reliable deployment in the real-world. In this work, we leverage the formulation of variational inference in function space, where we associate Gaussian Processes (GPs) to both Bayesian CNN priors and variational family. Since GPs are fully determined by their mean and covariance functions, we are able to obtain predictive uncertainty estimates at the cost of a single forward pass through any chosen CNN architecture and for any supervised learning task. By leveraging the structure of the induced covariance matrices, we propose numerically efficient algorithms which enable fast training in the context of high-dimensional tasks such as depth estimation and semantic segmentation. Additionally, we provide sufficient conditions for constructing regression loss functions whose probabilistic counterparts are compatible with aleatoric uncertainty quantification.
Tasks Depth Estimation, Gaussian Processes, Semantic Segmentation
Published 2020-03-06
URL https://arxiv.org/abs/2003.03396v1
PDF https://arxiv.org/pdf/2003.03396v1.pdf
PWC https://paperswithcode.com/paper/scalable-uncertainty-for-computer-vision-with

Particle Filter Based Monocular Human Tracking with a 3D Cardbox Model and a Novel Deterministic Resampling Strategy

Title Particle Filter Based Monocular Human Tracking with a 3D Cardbox Model and a Novel Deterministic Resampling Strategy
Authors Ziyuan Liu, Dongheui Lee, Wolfgang Sepp
Abstract The challenge of markerless human motion tracking is the high dimensionality of the search space. Thus, efficient exploration in the search space is of great significance. In this paper, a motion capturing algorithm is proposed for upper body motion tracking. The proposed system tracks human motion based on monocular silhouette-matching, and it is built on the top of a hierarchical particle filter, within which a novel deterministic resampling strategy (DRS) is applied. The proposed system is evaluated quantitatively with the ground truth data measured by an inertial sensor system. In addition, we compare the DRS with the stratified resampling strategy (SRS). It is shown in experiments that DRS outperforms SRS with the same amount of particles. Moreover, a new 3D articulated human upper body model with the name 3D cardbox model is created and is proven to work successfully for motion tracking. Experiments show that the proposed system can robustly track upper body motion without self-occlusion. Motions towards the camera can also be well tracked.
Tasks Efficient Exploration
Published 2020-02-21
URL https://arxiv.org/abs/2002.09554v1
PDF https://arxiv.org/pdf/2002.09554v1.pdf
PWC https://paperswithcode.com/paper/particle-filter-based-monocular-human

CNN-based InSAR Denoising and Coherence Metric

Title CNN-based InSAR Denoising and Coherence Metric
Authors Subhayan Mukherjee, Aaron Zimmer, Navaneeth Kamballur Kottayil, Xinyao Sun, Parwant Ghuman, Irene Cheng
Abstract Interferometric Synthetic Aperture Radar (InSAR) imagery for estimating ground movement, based on microwaves reflected off ground targets is gaining increasing importance in remote sensing. However, noise corrupts microwave reflections received at satellite and contaminates the signal’s wrapped phase. We introduce Convolutional Neural Networks (CNNs) to this problem domain and show the effectiveness of autoencoder CNN architectures to learn InSAR image denoising filters in the absence of clean ground truth images, and for artefact reduction in estimated coherence through intelligent preprocessing of training data. We compare our results with four established methods to illustrate superiority of proposed method.
Tasks Denoising, Image Denoising
Published 2020-01-20
URL https://arxiv.org/abs/2001.06954v1
PDF https://arxiv.org/pdf/2001.06954v1.pdf
PWC https://paperswithcode.com/paper/cnn-based-insar-denoising-and-coherence
comments powered by Disqus