April 1, 2020

2990 words 15 mins read

Paper Group ANR 448

Paper Group ANR 448

A Hybrid Model-based and Data-driven Approach to Spectrum Sharing in mmWave Cellular Networks. Translating Diffusion, Wavelets, and Regularisation into Residual Networks. Generalized Hidden Parameter MDPs Transferable Model-based RL in a Handful of Trials. Handling the Positive-Definite Constraint in the Bayesian Learning Rule. A Tutorial on Learni …

A Hybrid Model-based and Data-driven Approach to Spectrum Sharing in mmWave Cellular Networks

Title A Hybrid Model-based and Data-driven Approach to Spectrum Sharing in mmWave Cellular Networks
Authors Hossein S. Ghadikolaei, Hadi Ghauch, Gabor Fodor, Mikael Skoglund, Carlo Fischione
Abstract Inter-operator spectrum sharing in millimeter-wave bands has the potential of substantially increasing the spectrum utilization and providing a larger bandwidth to individual user equipment at the expense of increasing inter-operator interference. Unfortunately, traditional model-based spectrum sharing schemes make idealistic assumptions about inter-operator coordination mechanisms in terms of latency and protocol overhead, while being sensitive to missing channel state information. In this paper, we propose hybrid model-based and data-driven multi-operator spectrum sharing mechanisms, which incorporate model-based beamforming and user association complemented by data-driven model refinements. Our solution has the same computational complexity as a model-based approach but has the major advantage of having substantially less signaling overhead. We discuss how limited channel state information and quantized codebook-based beamforming affect the learning and the spectrum sharing performance. We show that the proposed hybrid sharing scheme significantly improves spectrum utilization under realistic assumptions on inter-operator coordination and channel state information acquisition.
Tasks
Published 2020-03-19
URL https://arxiv.org/abs/2003.08611v1
PDF https://arxiv.org/pdf/2003.08611v1.pdf
PWC https://paperswithcode.com/paper/a-hybrid-model-based-and-data-driven-approach
Repo
Framework

Translating Diffusion, Wavelets, and Regularisation into Residual Networks

Title Translating Diffusion, Wavelets, and Regularisation into Residual Networks
Authors Tobias Alt, Joachim Weickert, Pascal Peter
Abstract Convolutional neural networks (CNNs) often perform well, but their stability is poorly understood. To address this problem, we consider the simple prototypical problem of signal denoising, where classical approaches such as nonlinear diffusion, wavelet-based methods and regularisation offer provable stability guarantees. To transfer such guarantees to CNNs, we interpret numerical approximations of these classical methods as a specific residual network (ResNet) architecture. This leads to a dictionary which allows to translate diffusivities, shrinkage functions, and regularisers into activation functions, and enables a direct communication between the four research communities. On the CNN side, it does not only inspire new families of nonmonotone activation functions, but also introduces intrinsically stable architectures for an arbitrary number of layers.
Tasks Denoising
Published 2020-02-07
URL https://arxiv.org/abs/2002.02753v1
PDF https://arxiv.org/pdf/2002.02753v1.pdf
PWC https://paperswithcode.com/paper/translating-diffusion-wavelets-and
Repo
Framework

Generalized Hidden Parameter MDPs Transferable Model-based RL in a Handful of Trials

Title Generalized Hidden Parameter MDPs Transferable Model-based RL in a Handful of Trials
Authors Christian F. Perez, Felipe Petroski Such, Theofanis Karaletsos
Abstract There is broad interest in creating RL agents that can solve many (related) tasks and adapt to new tasks and environments after initial training. Model-based RL leverages learned surrogate models that describe dynamics and rewards of individual tasks, such that planning in a good surrogate can lead to good control of the true system. Rather than solving each task individually from scratch, hierarchical models can exploit the fact that tasks are often related by (unobserved) causal factors of variation in order to achieve efficient generalization, as in learning how the mass of an item affects the force required to lift it can generalize to previously unobserved masses. We propose Generalized Hidden Parameter MDPs (GHP-MDPs) that describe a family of MDPs where both dynamics and reward can change as a function of hidden parameters that vary across tasks. The GHP-MDP augments model-based RL with latent variables that capture these hidden parameters, facilitating transfer across tasks. We also explore a variant of the model that incorporates explicit latent structure mirroring the causal factors of variation across tasks (for instance: agent properties, environmental factors, and goals). We experimentally demonstrate state-of-the-art performance and sample-efficiency on a new challenging MuJoCo task using reward and dynamics latent spaces, while beating a previous state-of-the-art baseline with $>10\times$ less data. Using test-time inference of the latent variables, our approach generalizes in a single episode to novel combinations of dynamics and reward, and to novel rewards.
Tasks
Published 2020-02-08
URL https://arxiv.org/abs/2002.03072v1
PDF https://arxiv.org/pdf/2002.03072v1.pdf
PWC https://paperswithcode.com/paper/generalized-hidden-parameter-mdps
Repo
Framework

Handling the Positive-Definite Constraint in the Bayesian Learning Rule

Title Handling the Positive-Definite Constraint in the Bayesian Learning Rule
Authors Wu Lin, Mark Schmidt, Mohammad Emtiyaz Khan
Abstract The Bayesian learning rule is a recently proposed variational inference method, which not only contains many existing learning algorithms as special cases but also enables the design of new algorithms. Unfortunately, when posterior parameters lie in an open constraint set, the rule may not satisfy the constraints and requires line-searches which could slow down the algorithm. In this paper, we fix this issue for the positive-definite constraint by proposing an improved rule that naturally handles the constraint. Our modification is obtained using Riemannian gradient methods, and is valid when the approximation attains a \emph{block-coordinate natural parameterization} (e.g., Gaussian distributions and their mixtures). Our method outperforms existing methods without any significant increase in computation. Our work makes it easier to apply the learning rule in the presence of positive-definite constraints in parameter spaces.
Tasks
Published 2020-02-24
URL https://arxiv.org/abs/2002.10060v3
PDF https://arxiv.org/pdf/2002.10060v3.pdf
PWC https://paperswithcode.com/paper/handling-the-positive-definite-constraint-in
Repo
Framework

A Tutorial on Learning With Bayesian Networks

Title A Tutorial on Learning With Bayesian Networks
Authors David Heckerman
Abstract A Bayesian network is a graphical model that encodes probabilistic relationships among variables of interest. When used in conjunction with statistical techniques, the graphical model has several advantages for data analysis. One, because the model encodes dependencies among all variables, it readily handles situations where some data entries are missing. Two, a Bayesian network can be used to learn causal relationships, and hence can be used to gain understanding about a problem domain and to predict the consequences of intervention. Three, because the model has both a causal and probabilistic semantics, it is an ideal representation for combining prior knowledge (which often comes in causal form) and data. Four, Bayesian statistical methods in conjunction with Bayesian networks offer an efficient and principled approach for avoiding the overfitting of data. In this paper, we discuss methods for constructing Bayesian networks from prior knowledge and summarize Bayesian statistical methods for using data to improve these models. With regard to the latter task, we describe methods for learning both the parameters and structure of a Bayesian network, including techniques for learning with incomplete data. In addition, we relate Bayesian-network methods for learning to techniques for supervised and unsupervised learning. We illustrate the graphical-modeling approach using a real-world case study.
Tasks
Published 2020-02-01
URL https://arxiv.org/abs/2002.00269v1
PDF https://arxiv.org/pdf/2002.00269v1.pdf
PWC https://paperswithcode.com/paper/a-tutorial-on-learning-with-bayesian-networks
Repo
Framework

Rapid Whole Slide Imaging via Learning-based Two-shot Virtual Autofocusing

Title Rapid Whole Slide Imaging via Learning-based Two-shot Virtual Autofocusing
Authors Qiang Li, Xianming Liu, Kaige Han, Cheng Guo, Xiangyang Ji, Xiaolin Wu
Abstract Whole slide imaging (WSI) is an emerging technology for digital pathology. The process of autofocusing is the main influence of the performance of WSI. Traditional autofocusing methods either are time-consuming due to repetitive mechanical motions, or require additional hardware and thus are not compatible to current WSI systems. In this paper, we propose the concept of \textit{virtual autofocusing}, which does not rely on mechanical adjustment to conduct refocusing but instead recovers in-focus images in an offline learning-based manner. With the initial focal position, we only perform two-shot imaging, in contrast traditional methods commonly need to conduct as many as 21 times image shooting in each tile scanning. Considering that the two captured out-of-focus images retain pieces of partial information about the underlying in-focus image, we propose a U-Net-inspired deep neural network based approach for fusing them into a recovered in-focus image. The proposed scheme is fast in tissue slides scanning, enabling a high-throughput generation of digital pathology images. Experimental results demonstrate that our scheme achieves satisfactory refocusing performance.
Tasks
Published 2020-03-14
URL https://arxiv.org/abs/2003.06630v1
PDF https://arxiv.org/pdf/2003.06630v1.pdf
PWC https://paperswithcode.com/paper/rapid-whole-slide-imaging-via-learning-based
Repo
Framework

Improving Adversarial Robustness Through Progressive Hardening

Title Improving Adversarial Robustness Through Progressive Hardening
Authors Chawin Sitawarin, Supriyo Chakraborty, David Wagner
Abstract Adversarial training (AT) has become a popular choice for training robust networks. However, by virtue of its formulation, AT tends to sacrifice clean accuracy heavily in favor of robustness. Furthermore, AT with a large perturbation budget can cause models to get stuck at poor local minima and behave like a constant function, always predicting the same class. To address the above concerns we propose Adversarial Training with Early Stopping (ATES). The design of ATES is guided by principles from curriculum learning that emphasizes on starting “easy” and gradually ramping up on the “difficulty” of training. We do so by early stopping the adversarial example generation step in AT, progressively increasing difficulty of the samples the network trains on. This stabilizes network training even for large perturbation budgets and allows the network to operate at a better clean accuracy versus robustness trade-off curve compared to AT. Functionally, this leads to a significant improvement in both clean accuracy and robustness for ATES models.
Tasks
Published 2020-03-18
URL https://arxiv.org/abs/2003.09347v1
PDF https://arxiv.org/pdf/2003.09347v1.pdf
PWC https://paperswithcode.com/paper/improving-adversarial-robustness-through
Repo
Framework

Causal Interpretability for Machine Learning – Problems, Methods and Evaluation

Title Causal Interpretability for Machine Learning – Problems, Methods and Evaluation
Authors Raha Moraffah, Mansooreh Karami, Ruocheng Guo, Adrienne Raglin, Huan Liu
Abstract Machine learning models have had discernible achievements in a myriad of applications. However, most of these models are black-boxes, and it is obscure how the decisions are made by them. This makes the models unreliable and untrustworthy. To provide insights into the decision making processes of these models, a variety of traditional interpretable models have been proposed. Moreover, to generate more human-friendly explanations, recent work on interpretability tries to answer questions related to causality such as “Why does this model makes such decisions?” or “Was it a specific feature that caused the decision made by the model?". In this work, models that aim to answer causal questions are referred to as causal interpretable models. The existing surveys have covered concepts and methodologies of traditional interpretability. In this work, we present a comprehensive survey on causal interpretable models from the aspects of the problems and methods. In addition, this survey provides in-depth insights into the existing evaluation metrics for measuring interpretability, which can help practitioners understand for what scenarios each evaluation metric is suitable.
Tasks Decision Making
Published 2020-03-09
URL https://arxiv.org/abs/2003.03934v3
PDF https://arxiv.org/pdf/2003.03934v3.pdf
PWC https://paperswithcode.com/paper/causal-interpretability-for-machine-learning
Repo
Framework

Variational Encoder-based Reliable Classification

Title Variational Encoder-based Reliable Classification
Authors Chitresh Bhushan, Zhaoyuan Yang, Nurali Virani, Naresh Iyer
Abstract Machine learning models provide statistically impressive results which might be individually unreliable. To provide reliability, we propose an Epistemic Classifier (EC) that can provide justification of its belief using support from the training dataset as well as quality of reconstruction. Our approach is based on modified variational auto-encoders that can identify a semantically meaningful low-dimensional space where perceptually similar instances are close in $\ell_2$-distance too. Our results demonstrate improved reliability of predictions and robust identification of samples with adversarial attacks as compared to baseline of softmax-based thresholding.
Tasks
Published 2020-02-19
URL https://arxiv.org/abs/2002.08289v1
PDF https://arxiv.org/pdf/2002.08289v1.pdf
PWC https://paperswithcode.com/paper/variational-encoder-based-reliable
Repo
Framework

1st Place Solutions for OpenImage2019 – Object Detection and Instance Segmentation

Title 1st Place Solutions for OpenImage2019 – Object Detection and Instance Segmentation
Authors Yu Liu, Guanglu Song, Yuhang Zang, Yan Gao, Enze Xie, Junjie Yan, Chen Change Loy, Xiaogang Wang
Abstract This article introduces the solutions of the two champion teams, MMfruit' for the detection track and MMfruitSeg’ for the segmentation track, in OpenImage Challenge 2019. It is commonly known that for an object detector, the shared feature at the end of the backbone is not appropriate for both classification and regression, which greatly limits the performance of both single stage detector and Faster RCNN \cite{ren2015faster} based detector. In this competition, we observe that even with a shared feature, different locations in one object has completely inconsistent performances for the two tasks. \textit{E.g. the features of salient locations are usually good for classification, while those around the object edge are good for regression.} Inspired by this, we propose the Decoupling Head (DH) to disentangle the object classification and regression via the self-learned optimal feature extraction, which leads to a great improvement. Furthermore, we adjust the soft-NMS algorithm to adj-NMS to obtain stable performance improvement. Finally, a well-designed ensemble strategy via voting the bounding box location and confidence is proposed. We will also introduce several training/inferencing strategies and a bag of tricks that give minor improvement. Given those masses of details, we train and aggregate 28 global models with various backbones, heads and 3+2 expert models, and achieves the 1st place on the OpenImage 2019 Object Detection Challenge on the both public and private leadboards. Given such good instance bounding box, we further design a simple instance-level semantic segmentation pipeline and achieve the 1st place on the segmentation challenge.
Tasks Instance Segmentation, Object Classification, Object Detection, Semantic Segmentation
Published 2020-03-17
URL https://arxiv.org/abs/2003.07557v1
PDF https://arxiv.org/pdf/2003.07557v1.pdf
PWC https://paperswithcode.com/paper/1st-place-solutions-for-openimage2019-object
Repo
Framework

A Power-Efficient Binary-Weight Spiking Neural Network Architecture for Real-Time Object Classification

Title A Power-Efficient Binary-Weight Spiking Neural Network Architecture for Real-Time Object Classification
Authors Pai-Yu Tan, Po-Yao Chuang, Yen-Ting Lin, Cheng-Wen Wu, Juin-Ming Lu
Abstract Neural network hardware is considered an essential part of future edge devices. In this paper, we propose a binary-weight spiking neural network (BW-SNN) hardware architecture for low-power real-time object classification on edge platforms. This design stores a full neural network on-chip, and hence requires no off-chip bandwidth. The proposed systolic array maximizes data reuse for a typical convolutional layer. A 5-layer convolutional BW-SNN hardware is implemented in 90nm CMOS. Compared with state-of-the-art designs, the area cost and energy per classification are reduced by 7$\times$ and 23$\times$, respectively, while also achieving a higher accuracy on the MNIST benchmark. This is also a pioneering SNN hardware architecture that supports advanced CNN architectures.
Tasks Object Classification
Published 2020-03-12
URL https://arxiv.org/abs/2003.06310v1
PDF https://arxiv.org/pdf/2003.06310v1.pdf
PWC https://paperswithcode.com/paper/a-power-efficient-binary-weight-spiking
Repo
Framework

A General Framework for Learning Mean-Field Games

Title A General Framework for Learning Mean-Field Games
Authors Xin Guo, Anran Hu, Renyuan Xu, Junzi Zhang
Abstract This paper presents a general mean-field game (GMFG) framework for simultaneous learning and decision-making in stochastic games with a large population. It first establishes the existence of a unique Nash Equilibrium to this GMFG, and demonstrates that naively combining Q-learning with the fixed-point approach in classical MFGs yields unstable algorithms. It then proposes value-based and policy-based reinforcement learning algorithms (GMF-P and GMF-P respectively) with smoothed policies, with analysis of convergence property and computational complexity. The experiments on repeated Ad auction problems demonstrate that GMF-V-Q, a specific GMF-V algorithm based on Q-learning, is efficient and robust in terms of convergence and learning accuracy. Moreover, its performance is superior in convergence, stability, and learning ability, when compared with existing algorithms for multi-agent reinforcement learning.
Tasks Decision Making, Multi-agent Reinforcement Learning, Q-Learning
Published 2020-03-13
URL https://arxiv.org/abs/2003.06069v1
PDF https://arxiv.org/pdf/2003.06069v1.pdf
PWC https://paperswithcode.com/paper/a-general-framework-for-learning-mean-field
Repo
Framework

Fixed smooth convolutional layer for avoiding checkerboard artifacts in CNNs

Title Fixed smooth convolutional layer for avoiding checkerboard artifacts in CNNs
Authors Yuma Kinoshita, Hitoshi Kiya
Abstract In this paper, we propose a fixed convolutional layer with an order of smoothness not only for avoiding checkerboard artifacts in convolutional neural networks (CNNs) but also for enhancing the performance of CNNs, where the smoothness of its filter kernel can be controlled by a parameter. It is well-known that a number of CNNs generate checkerboard artifacts in both of two process: forward-propagation of upsampling layers and backward-propagation of strided convolutional layers. The proposed layer can perfectly prevent checkerboard artifacts caused by strided convolutional layers or upsampling layers including transposed convolutional layers. In an image-classification experiment with four CNNs: a simple CNN, VGG8, ResNet-18, and ResNet-101, applying the fixed layers to these CNNs is shown to improve the classification performance of all CNNs. In addition, the fixed layer are applied to generative adversarial networks (GANs), for the first time. From image-generation results, a smoother fixed convolutional layer is demonstrated to enable us to improve the quality of images generated with GANs.
Tasks Image Classification, Image Generation
Published 2020-02-06
URL https://arxiv.org/abs/2002.02117v1
PDF https://arxiv.org/pdf/2002.02117v1.pdf
PWC https://paperswithcode.com/paper/fixed-smooth-convolutional-layer-for-avoiding
Repo
Framework

Optimal Regularization Can Mitigate Double Descent

Title Optimal Regularization Can Mitigate Double Descent
Authors Preetum Nakkiran, Prayaag Venkat, Sham Kakade, Tengyu Ma
Abstract Recent empirical and theoretical studies have shown that many learning algorithms – from linear regression to neural networks – can have test performance that is non-monotonic in quantities such the sample size and model size. This striking phenomenon, often referred to as “double descent”, has raised questions of if we need to re-think our current understanding of generalization. In this work, we study whether the double-descent phenomenon can be avoided by using optimal regularization. Theoretically, we prove that for certain linear regression models with isotropic data distribution, optimally-tuned $\ell_2$ regularization achieves monotonic test performance as we grow either the sample size or the model size. We also demonstrate empirically that optimally-tuned $\ell_2$ regularization can mitigate double descent for more general models, including neural networks. Our results suggest that it may also be informative to study the test risk scalings of various algorithms in the context of appropriately tuned regularization.
Tasks
Published 2020-03-04
URL https://arxiv.org/abs/2003.01897v1
PDF https://arxiv.org/pdf/2003.01897v1.pdf
PWC https://paperswithcode.com/paper/optimal-regularization-can-mitigate-double
Repo
Framework

Long term planning of military aircraft flight and maintenance operations

Title Long term planning of military aircraft flight and maintenance operations
Authors Franco Peschiera, Olga Battaïa, Alain Haït, Nicolas Dupin
Abstract We present the Flight and Maintenance Planning (FMP) problem in its military variant and applied to long term planning. The problem has been previously studied for short- and medium-term horizons only. We compare its similarities and differences with previous work and prove its complexity. We generate scenarios inspired by the French Air Force fleet. We formulate an exact Mixed Integer Programming (MIP) model to solve the problem in these scenarios and we analyse the performance of the solving method under these circumstances. A heuristic was built to generate fast feasible solutions, that in some cases were shown to help warm-start the model.
Tasks
Published 2020-01-27
URL https://arxiv.org/abs/2001.09856v1
PDF https://arxiv.org/pdf/2001.09856v1.pdf
PWC https://paperswithcode.com/paper/long-term-planning-of-military-aircraft
Repo
Framework
comments powered by Disqus