January 28, 2020

3218 words 16 mins read

Paper Group ANR 995

Multiagent Evaluation under Incomplete Information. Full error analysis for the training of deep neural networks. Novelty Detection and Learning from Extremely Weak Supervision. Multiple Style-Transfer in Real-Time. Dataset Growth in Medical Image Analysis Research. FAMED-Net: A Fast and Accurate Multi-scale End-to-end Dehazing Network. The Error-F …

Multiagent Evaluation under Incomplete Information


Title	Multiagent Evaluation under Incomplete Information
Authors	Mark Rowland, Shayegan Omidshafiei, Karl Tuyls, Julien Perolat, Michal Valko, Georgios Piliouras, Remi Munos
Abstract	This paper investigates the evaluation of learned multiagent strategies in the incomplete information setting, which plays a critical role in ranking and training of agents. Traditionally, researchers have relied on Elo ratings for this purpose, with recent works also using methods based on Nash equilibria. Unfortunately, Elo is unable to handle intransitive agent interactions, and other techniques are restricted to zero-sum, two-player settings or are limited by the fact that the Nash equilibrium is intractable to compute. Recently, a ranking method called {\alpha}-Rank, relying on a new graph-based game-theoretic solution concept, was shown to tractably apply to general games. However, evaluations based on Elo or {\alpha}-Rank typically assume noise-free game outcomes, despite the data often being collected from noisy simulations, making this assumption unrealistic in practice. This paper investigates multiagent evaluation in the incomplete information regime, involving general-sum many-player games with noisy outcomes. We derive sample complexity guarantees required to confidently rank agents in this setting. We propose adaptive algorithms for accurate ranking, provide correctness and sample complexity guarantees, then introduce a means of connecting uncertainties in noisy match outcomes to uncertainties in rankings. We evaluate the performance of these approaches in several domains, including Bernoulli games, a soccer meta-game, and Kuhn poker.
Tasks
Published	2019-09-21
URL	https://arxiv.org/abs/1909.09849v4
PDF	https://arxiv.org/pdf/1909.09849v4.pdf
PWC	https://paperswithcode.com/paper/190909849
Repo
Framework

Full error analysis for the training of deep neural networks


Title	Full error analysis for the training of deep neural networks
Authors	Christan Beck, Arnulf Jentzen, Benno Kuckuck
Abstract	Deep learning algorithms have been applied very successfully in recent years to a range of problems out of reach for classical solution paradigms. Nevertheless, there is no completely rigorous mathematical error and convergence analysis which explains the success of deep learning algorithms. The error of a deep learning algorithm can in many situations be decomposed into three parts, the approximation error, the generalization error, and the optimization error. In this work we estimate for a certain deep learning algorithm each of these three errors and combine these three error estimates to obtain an overall error analysis for the deep learning algorithm under consideration. In particular, we thereby establish convergence with a suitable convergence speed for the overall error of the deep learning algorithm under consideration. Our convergence speed analysis is far from optimal and the convergence speed that we establish is rather slow, increases exponentially in the dimensions, and, in particular, suffers from the curse of dimensionality. The main contribution of this work is, instead, to provide a full error analysis (i) which covers each of the three different sources of errors usually emerging in deep learning algorithms and (ii) which merges these three sources of errors into one overall error estimate for the considered deep learning algorithm.
Tasks
Published	2019-09-30
URL	https://arxiv.org/abs/1910.00121v2
PDF	https://arxiv.org/pdf/1910.00121v2.pdf
PWC	https://paperswithcode.com/paper/full-error-analysis-for-the-training-of-deep
Repo
Framework

Novelty Detection and Learning from Extremely Weak Supervision


Title	Novelty Detection and Learning from Extremely Weak Supervision
Authors	Eduardo Soares, Plamen Angelov
Abstract	In this paper we offer a method and algorithm, which make possible fully autonomous (unsupervised) detection of new classes, and learning following a very parsimonious training priming (few labeled data samples only). Moreover, new unknown classes may appear at a later stage and the proposed xClass method and algorithm are able to successfully discover this and learn from the data autonomously. Furthermore, the features (inputs to the classifier) are automatically sub-selected by the algorithm based on the accumulated data density per feature per class. As a result, a highly efficient, lean, human-understandable, autonomously self-learning model (which only needs an extremely parsimonious priming) emerges from the data. To validate our proposal we tested it on two challenging problems, including imbalanced Caltech-101 data set and iRoads dataset. Not only we achieved higher precision, but, more significantly, we only used a single class beforehand, while other methods used all the available classes) and we generated interpretable models with smaller number of features used, through extremely weak and weak supervision.
Tasks
Published	2019-11-01
URL	https://arxiv.org/abs/1911.00616v1
PDF	https://arxiv.org/pdf/1911.00616v1.pdf
PWC	https://paperswithcode.com/paper/novelty-detection-and-learning-from-extremely
Repo
Framework

Multiple Style-Transfer in Real-Time


Title	Multiple Style-Transfer in Real-Time
Authors	Michael Maring, Kaustav Chakraborty
Abstract	Style transfer aims to combine the content of one image with the artistic style of another. It was discovered that lower levels of convolutional networks captured style information, while higher levels captures content information. The original style transfer formulation used a weighted combination of VGG-16 layer activations to achieve this goal. Later, this was accomplished in real-time using a feed-forward network to learn the optimal combination of style and content features from the respective images. The first aim of our project was to introduce a framework for capturing the style from several images at once. We propose a method that extends the original real-time style transfer formulation by combining the features of several style images. This method successfully captures color information from the separate style images. The other aim of our project was to improve the temporal style continuity from frame to frame. Accordingly, we have experimented with the temporal stability of the output images and discussed the various available techniques that could be employed as alternatives.
Tasks	Style Transfer
Published	2019-11-15
URL	https://arxiv.org/abs/1911.06464v2
PDF	https://arxiv.org/pdf/1911.06464v2.pdf
PWC	https://paperswithcode.com/paper/multiple-style-transfer-in-real-time
Repo
Framework

Dataset Growth in Medical Image Analysis Research


Title	Dataset Growth in Medical Image Analysis Research
Authors	Yuval Landau, Nahum Kiryati
Abstract	Medical image analysis studies usually require medical image datasets for training, testing and validation of algorithms. The need is underscored by the deep learning revolution and the dominance of machine learning in recent medical image analysis research. Nevertheless, due to ethical and legal constraints, commercial conflicts and the dependence on busy medical professionals, medical image analysis researchers have been described as “data starved”. Due to the lack of objective criteria for sufficiency of dataset size, the research community implicitly sets ad-hoc standards by means of the peer review process. We hypothesize that peer review requires researchers to report the use of ever-increasing datasets as one condition for acceptance of their work to reputable publication venues. To test this hypothesis, we scanned the proceedings of the eminent MICCAI (Medical Image Computing and Computer-Assisted Intervention) conferences from 2011 to 2018. From a total of 2136 articles, we focused on 907 papers involving human datasets of MRI (Magnetic Resonance Imaging), CT (Computed Tomography) and fMRI (functional MRI) images. For each modality, for each of the years 2011-2018 we calculated the average, geometric mean and median number of human subjects used in that year’s MICCAI articles. The results corroborate the dataset growth hypothesis. Specifically, the annual median dataset size in MICCAI articles has grown roughly 3-10 times from 2011 to 2018, depending on the imaging modality. Statistical analysis further supports the dataset growth hypothesis and reveals exponential growth of the geometric mean dataset size, with annual growth of about 21% for MRI, 24% for CT and 31% for fMRI. In slight analogy to Moore’s law, the results can provide guidance about trends in the expectations of the medical image analysis community regarding dataset size.
Tasks
Published	2019-08-21
URL	https://arxiv.org/abs/1908.07765v1
PDF	https://arxiv.org/pdf/1908.07765v1.pdf
PWC	https://paperswithcode.com/paper/190807765
Repo
Framework

FAMED-Net: A Fast and Accurate Multi-scale End-to-end Dehazing Network


Title	FAMED-Net: A Fast and Accurate Multi-scale End-to-end Dehazing Network
Authors	Jing Zhang, Dacheng Tao
Abstract	Single image dehazing is a critical image pre-processing step for subsequent high-level computer vision tasks. However, it remains challenging due to its ill-posed nature. Existing dehazing models tend to suffer from model overcomplexity and computational inefficiency or have limited representation capacity. To tackle these challenges, here we propose a fast and accurate multi-scale end-to-end dehazing network called FAMED-Net, which comprises encoders at three scales and a fusion module to efficiently and directly learn the haze-free image. Each encoder consists of cascaded and densely connected point-wise convolutional layers and pooling layers. Since no larger convolutional kernels are used and features are reused layer-by-layer, FAMED-Net is lightweight and computationally efficient. Thorough empirical studies on public synthetic datasets (including RESIDE) and real-world hazy images demonstrate the superiority of FAMED-Net over other representative state-of-the-art models with respect to model complexity, computational efficiency, restoration accuracy, and cross-set generalization. The code will be made publicly available.
Tasks	Image Dehazing, Single Image Dehazing
Published	2019-06-11
URL	https://arxiv.org/abs/1906.04334v2
PDF	https://arxiv.org/pdf/1906.04334v2.pdf
PWC	https://paperswithcode.com/paper/famed-net-a-fast-and-accurate-multi-scale-end
Repo
Framework

The Error-Feedback Framework: Better Rates for SGD with Delayed Gradients and Compressed Communication


Title	The Error-Feedback Framework: Better Rates for SGD with Delayed Gradients and Compressed Communication
Authors	Sebastian U. Stich, Sai Praneeth Karimireddy
Abstract	We analyze (stochastic) gradient descent (SGD) with delayed updates on smooth quasi-convex and non-convex functions and derive concise, non-asymptotic, convergence rates. We show that the rate of convergence in all cases consists of two terms: (i) a stochastic term which is not affected by the delay, and (ii) a higher order deterministic term which is only linearly slowed down by the delay. Thus, in the presence of noise, the effects of the delay become negligible after a few iterations and the algorithm converges at the same optimal rate as standard SGD. This result extends a line of research that showed similar results in the asymptotic regime or for strongly-convex quadratic functions only. We further show similar results for SGD with more intricate form of delayed gradients—compressed gradients under error compensation and for localSGD where multiple workers perform local steps before communicating with each other. In all of these settings, we improve upon the best known rates. These results show that SGD is robust to compressed and/or delayed stochastic gradient updates. This is in particular important for distributed parallel implementations, where asynchronous and communication efficient methods are the key to achieve linear speedups for optimization with multiple devices.
Tasks
Published	2019-09-11
URL	https://arxiv.org/abs/1909.05350v1
PDF	https://arxiv.org/pdf/1909.05350v1.pdf
PWC	https://paperswithcode.com/paper/the-error-feedback-framework-better-rates-for
Repo
Framework

VariBAD: A Very Good Method for Bayes-Adaptive Deep RL via Meta-Learning


Title	VariBAD: A Very Good Method for Bayes-Adaptive Deep RL via Meta-Learning
Authors	Luisa Zintgraf, Kyriacos Shiarlis, Maximilian Igl, Sebastian Schulze, Yarin Gal, Katja Hofmann, Shimon Whiteson
Abstract	Trading off exploration and exploitation in an unknown environment is key to maximising expected return during learning. A Bayes-optimal policy, which does so optimally, conditions its actions not only on the environment state but on the agent’s uncertainty about the environment. Computing a Bayes-optimal policy is however intractable for all but the smallest tasks. In this paper, we introduce variational Bayes-Adaptive Deep RL (variBAD), a way to meta-learn to perform approximate inference in an unknown environment, and incorporate task uncertainty directly during action selection. In a grid-world domain, we illustrate how variBAD performs structured online exploration as a function of task uncertainty. We further evaluate variBAD on MuJoCo domains widely used in meta-RL and show that it achieves higher online return than existing methods.
Tasks	Meta-Learning
Published	2019-10-18
URL	https://arxiv.org/abs/1910.08348v2
PDF	https://arxiv.org/pdf/1910.08348v2.pdf
PWC	https://paperswithcode.com/paper/varibad-a-very-good-method-for-bayes-adaptive-1
Repo
Framework

Bridging Commonsense Reasoning and Probabilistic Planning via a Probabilistic Action Language


Title	Bridging Commonsense Reasoning and Probabilistic Planning via a Probabilistic Action Language
Authors	Yi Wang, Shiqi Zhang, Joohyung Lee
Abstract	To be responsive to dynamically changing real-world environments, an intelligent agent needs to perform complex sequential decision-making tasks that are often guided by commonsense knowledge. The previous work on this line of research led to the framework called “interleaved commonsense reasoning and probabilistic planning” (icorpp), which used P-log for representing commmonsense knowledge and Markov Decision Processes (MDPs) or Partially Observable MDPs (POMDPs) for planning under uncertainty. A main limitation of icorpp is that its implementation requires non-trivial engineering efforts to bridge the commonsense reasoning and probabilistic planning formalisms. In this paper, we present a unified framework to integrate icorpp’s reasoning and planning components. In particular, we extend probabilistic action language pBC+ to express utility, belief states, and observation as in POMDP models. Inheriting the advantages of action languages, the new action language provides an elaboration tolerant representation of POMDP that reflects commonsense knowledge. The idea led to the design of the system pbcplus2pomdp, which compiles a pBC+ action description into a POMDP model that can be directly processed by off-the-shelf POMDP solvers to compute an optimal policy of the pBC+ action description. Our experiments show that it retains the advantages of icorpp while avoiding the manual efforts in bridging the commonsense reasoner and the probabilistic planner.
Tasks	Decision Making
Published	2019-07-31
URL	https://arxiv.org/abs/1907.13482v1
PDF	https://arxiv.org/pdf/1907.13482v1.pdf
PWC	https://paperswithcode.com/paper/bridging-commonsense-reasoning-and
Repo
Framework

Semi-supervised semantic segmentation needs strong, high-dimensional perturbations


Title	Semi-supervised semantic segmentation needs strong, high-dimensional perturbations
Authors	Geoff French, Timo Aila, Samuli Laine, Michal Mackiewicz, Graham Finlayson
Abstract	Consistency regularization describes a class of approaches that have yielded ground breaking results in semi-supervised classification problems. Prior work has established the cluster assumption – under which the data distribution consists of uniform class clusters of samples separated by low density regions – as key to its success. We analyze the problem of semantic segmentation and find that the data distribution does not exhibit low density regions separating classes and offer this as an explanation for why semi-supervised segmentation is a challenging problem. We then identify the conditions that allow consistency regularization to work even without such low-density regions. This allows us to generalize the recently proposed CutMix augmentation technique to a powerful masked variant, CowMix, leading to a successful application of consistency regularization in the semi-supervised semantic segmentation setting and reaching state-of-the-art results in several standard datasets.
Tasks	Semantic Segmentation, Semi-Supervised Semantic Segmentation
Published	2019-06-05
URL	https://arxiv.org/abs/1906.01916v2
PDF	https://arxiv.org/pdf/1906.01916v2.pdf
PWC	https://paperswithcode.com/paper/consistency-regularization-and-cutmix-for
Repo
Framework

Refining the Structure of Neural Networks Using Matrix Conditioning


Title	Refining the Structure of Neural Networks Using Matrix Conditioning
Authors	Roozbeh Yousefzadeh, Dianne P O’Leary
Abstract	Deep learning models have proven to be exceptionally useful in performing many machine learning tasks. However, for each new dataset, choosing an effective size and structure of the model can be a time-consuming process of trial and error. While a small network with few neurons might not be able to capture the intricacies of a given task, having too many neurons can lead to overfitting and poor generalization. Here, we propose a practical method that employs matrix conditioning to automatically design the structure of layers of a feed-forward network, by first adjusting the proportion of neurons among the layers of a network and then scaling the size of network up or down. Results on sample image and non-image datasets demonstrate that our method results in small networks with high accuracies. Finally, guided by matrix conditioning, we provide a method to effectively squeeze models that are already trained. Our techniques reduce the human cost of designing deep learning models and can also reduce training time and the expense of using neural networks for applications.
Tasks
Published	2019-08-06
URL	https://arxiv.org/abs/1908.02400v1
PDF	https://arxiv.org/pdf/1908.02400v1.pdf
PWC	https://paperswithcode.com/paper/refining-the-structure-of-neural-networks
Repo
Framework

Occluded Pedestrian Detection with Visible IoU and Box Sign Predictor


Title	Occluded Pedestrian Detection with Visible IoU and Box Sign Predictor
Authors	Ruiqi Lu, Huimin Ma
Abstract	Training a robust classifier and an accurate box regressor are difficult for occluded pedestrian detection. Traditionally adopted Intersection over Union (IoU) measurement does not consider the occluded region of the object and leads to improper training samples. To address such issue, a modification called visible IoU is proposed in this paper to explicitly incorporate the visible ratio in selecting samples. Then a newly designed box sign predictor is placed in parallel with box regressor to separately predict the moving direction of training samples. It leads to higher localization accuracy by introducing sign prediction loss during training and sign refining in testing. Following these novelties, we obtain state-of-the-art performance on CityPersons benchmark for occluded pedestrian detection.
Tasks	Pedestrian Detection
Published	2019-11-26
URL	https://arxiv.org/abs/1911.11449v1
PDF	https://arxiv.org/pdf/1911.11449v1.pdf
PWC	https://paperswithcode.com/paper/occluded-pedestrian-detection-with-visible
Repo
Framework

Auto-Calibration of Remote Sensing Solar Telescopes with Deep Learning


Title	Auto-Calibration of Remote Sensing Solar Telescopes with Deep Learning
Authors	Brad Neuberg, Souvik Bose, Valentina Salvatelli, Luiz F. G. dos Santos, Mark Cheung, Miho Janvier, Atilim Gunes Baydin, Yarin Gal, Meng Jin
Abstract	As a part of NASA’s Heliophysics System Observatory (HSO) fleet of satellites,the Solar Dynamics Observatory (SDO) has continuously monitored the Sun since2010. Ultraviolet (UV) and Extreme UV (EUV) instruments in orbit, such asSDO’s Atmospheric Imaging Assembly (AIA) instrument, suffer time-dependent degradation which reduces instrument sensitivity. Accurate calibration for (E)UV instruments currently depends on periodic sounding rockets, which are infrequent and not practical for heliophysics missions in deep space. In the present work, we develop a Convolutional Neural Network (CNN) that auto-calibrates SDO/AIA channels and corrects sensitivity degradation by exploiting spatial patterns in multi-wavelength observations to arrive at a self-calibration of (E)UV imaging instruments. Our results remove a major impediment to developing future HSOmissions of the same scientific caliber as SDO but in deep space, able to observe the Sun from more vantage points than just SDO’s current geosynchronous orbit.This approach can be adopted to perform autocalibration of other imaging systems exhibiting similar forms of degradation
Tasks	Calibration
Published	2019-11-10
URL	https://arxiv.org/abs/1911.04008v1
PDF	https://arxiv.org/pdf/1911.04008v1.pdf
PWC	https://paperswithcode.com/paper/auto-calibration-of-remote-sensing-solar
Repo
Framework

Winning the ICCV 2019 Learning to Drive Challenge


Title	Winning the ICCV 2019 Learning to Drive Challenge
Authors	Michael Diodato, Yu Li, Manik Goyal, Iddo Drori
Abstract	Autonomous driving has a significant impact on society. Predicting vehicle trajectories, specifically, angle and speed, is important for safe and comfortable driving. This work focuses on fusing inputs from camera sensors and visual map data which lead to significant improvement in performance and plays a key role in winning the challenge. We use pre-trained CNN’s for processing image frames, a neural network for fusing the image representation with visual map data, and train a sequence model for time series prediction. We demonstrate the best performing MSE angle and best performance overall, to win the ICCV 2019 Learning to Drive challenge. We make our models and code publicly available.
Tasks	Autonomous Driving, Time Series, Time Series Prediction
Published	2019-10-23
URL	https://arxiv.org/abs/1910.10318v1
PDF	https://arxiv.org/pdf/1910.10318v1.pdf
PWC	https://paperswithcode.com/paper/winning-the-iccv-2019-learning-to-drive
Repo
Framework

L2G Auto-encoder: Understanding Point Clouds by Local-to-Global Reconstruction with Hierarchical Self-Attention


Title	L2G Auto-encoder: Understanding Point Clouds by Local-to-Global Reconstruction with Hierarchical Self-Attention
Authors	Xinhai Liu, Zhizhong Han, Xin Wen, Yu-Shen Liu, Matthias Zwicker
Abstract	Auto-encoder is an important architecture to understand point clouds in an encoding and decoding procedure of self reconstruction. Current auto-encoder mainly focuses on the learning of global structure by global shape reconstruction, while ignoring the learning of local structures. To resolve this issue, we propose Local-to-Global auto-encoder (L2G-AE) to simultaneously learn the local and global structure of point clouds by local to global reconstruction. Specifically, L2G-AE employs an encoder to encode the geometry information of multiple scales in a local region at the same time. In addition, we introduce a novel hierarchical self-attention mechanism to highlight the important points, scales and regions at different levels in the information aggregation of the encoder. Simultaneously, L2G-AE employs a recurrent neural network (RNN) as decoder to reconstruct a sequence of scales in a local region, based on which the global point cloud is incrementally reconstructed. Our outperforming results in shape classification, retrieval and upsampling show that L2G-AE can understand point clouds better than state-of-the-art methods.
Tasks
Published	2019-08-02
URL	https://arxiv.org/abs/1908.00720v1
PDF	https://arxiv.org/pdf/1908.00720v1.pdf
PWC	https://paperswithcode.com/paper/l2g-auto-encoder-understanding-point-clouds
Repo
Framework