April 1, 2020

2990 words 15 mins read

Paper Group ANR 448

A Hybrid Model-based and Data-driven Approach to Spectrum Sharing in mmWave Cellular Networks. Translating Diffusion, Wavelets, and Regularisation into Residual Networks. Generalized Hidden Parameter MDPs Transferable Model-based RL in a Handful of Trials. Handling the Positive-Definite Constraint in the Bayesian Learning Rule. A Tutorial on Learni …


Title	A Hybrid Model-based and Data-driven Approach to Spectrum Sharing in mmWave Cellular Networks
Authors	Hossein S. Ghadikolaei, Hadi Ghauch, Gabor Fodor, Mikael Skoglund, Carlo Fischione
Abstract	Inter-operator spectrum sharing in millimeter-wave bands has the potential of substantially increasing the spectrum utilization and providing a larger bandwidth to individual user equipment at the expense of increasing inter-operator interference. Unfortunately, traditional model-based spectrum sharing schemes make idealistic assumptions about inter-operator coordination mechanisms in terms of latency and protocol overhead, while being sensitive to missing channel state information. In this paper, we propose hybrid model-based and data-driven multi-operator spectrum sharing mechanisms, which incorporate model-based beamforming and user association complemented by data-driven model refinements. Our solution has the same computational complexity as a model-based approach but has the major advantage of having substantially less signaling overhead. We discuss how limited channel state information and quantized codebook-based beamforming affect the learning and the spectrum sharing performance. We show that the proposed hybrid sharing scheme significantly improves spectrum utilization under realistic assumptions on inter-operator coordination and channel state information acquisition.
Tasks
Published	2020-03-19
URL	https://arxiv.org/abs/2003.08611v1
PDF	https://arxiv.org/pdf/2003.08611v1.pdf
PWC	https://paperswithcode.com/paper/a-hybrid-model-based-and-data-driven-approach
Repo
Framework

Translating Diffusion, Wavelets, and Regularisation into Residual Networks


Title	Translating Diffusion, Wavelets, and Regularisation into Residual Networks
Authors	Tobias Alt, Joachim Weickert, Pascal Peter
Abstract	Convolutional neural networks (CNNs) often perform well, but their stability is poorly understood. To address this problem, we consider the simple prototypical problem of signal denoising, where classical approaches such as nonlinear diffusion, wavelet-based methods and regularisation offer provable stability guarantees. To transfer such guarantees to CNNs, we interpret numerical approximations of these classical methods as a specific residual network (ResNet) architecture. This leads to a dictionary which allows to translate diffusivities, shrinkage functions, and regularisers into activation functions, and enables a direct communication between the four research communities. On the CNN side, it does not only inspire new families of nonmonotone activation functions, but also introduces intrinsically stable architectures for an arbitrary number of layers.
Tasks	Denoising
Published	2020-02-07
URL	https://arxiv.org/abs/2002.02753v1
PDF	https://arxiv.org/pdf/2002.02753v1.pdf
PWC	https://paperswithcode.com/paper/translating-diffusion-wavelets-and
Repo
Framework

Generalized Hidden Parameter MDPs Transferable Model-based RL in a Handful of Trials


Title	Generalized Hidden Parameter MDPs Transferable Model-based RL in a Handful of Trials
Authors	Christian F. Perez, Felipe Petroski Such, Theofanis Karaletsos
Abstract	There is broad interest in creating RL agents that can solve many (related) tasks and adapt to new tasks and environments after initial training. Model-based RL leverages learned surrogate models that describe dynamics and rewards of individual tasks, such that planning in a good surrogate can lead to good control of the true system. Rather than solving each task individually from scratch, hierarchical models can exploit the fact that tasks are often related by (unobserved) causal factors of variation in order to achieve efficient generalization, as in learning how the mass of an item affects the force required to lift it can generalize to previously unobserved masses. We propose Generalized Hidden Parameter MDPs (GHP-MDPs) that describe a family of MDPs where both dynamics and reward can change as a function of hidden parameters that vary across tasks. The GHP-MDP augments model-based RL with latent variables that capture these hidden parameters, facilitating transfer across tasks. We also explore a variant of the model that incorporates explicit latent structure mirroring the causal factors of variation across tasks (for instance: agent properties, environmental factors, and goals). We experimentally demonstrate state-of-the-art performance and sample-efficiency on a new challenging MuJoCo task using reward and dynamics latent spaces, while beating a previous state-of-the-art baseline with $>10\times$ less data. Using test-time inference of the latent variables, our approach generalizes in a single episode to novel combinations of dynamics and reward, and to novel rewards.
Tasks
Published	2020-02-08
URL	https://arxiv.org/abs/2002.03072v1
PDF	https://arxiv.org/pdf/2002.03072v1.pdf
PWC	https://paperswithcode.com/paper/generalized-hidden-parameter-mdps
Repo
Framework

Handling the Positive-Definite Constraint in the Bayesian Learning Rule


Title	Handling the Positive-Definite Constraint in the Bayesian Learning Rule
Authors	Wu Lin, Mark Schmidt, Mohammad Emtiyaz Khan
Abstract	The Bayesian learning rule is a recently proposed variational inference method, which not only contains many existing learning algorithms as special cases but also enables the design of new algorithms. Unfortunately, when posterior parameters lie in an open constraint set, the rule may not satisfy the constraints and requires line-searches which could slow down the algorithm. In this paper, we fix this issue for the positive-definite constraint by proposing an improved rule that naturally handles the constraint. Our modification is obtained using Riemannian gradient methods, and is valid when the approximation attains a \emph{block-coordinate natural parameterization} (e.g., Gaussian distributions and their mixtures). Our method outperforms existing methods without any significant increase in computation. Our work makes it easier to apply the learning rule in the presence of positive-definite constraints in parameter spaces.
Tasks
Published	2020-02-24
URL	https://arxiv.org/abs/2002.10060v3
PDF	https://arxiv.org/pdf/2002.10060v3.pdf
PWC	https://paperswithcode.com/paper/handling-the-positive-definite-constraint-in
Repo
Framework

A Tutorial on Learning With Bayesian Networks


Title	A Tutorial on Learning With Bayesian Networks
Authors	David Heckerman
Abstract	A Bayesian network is a graphical model that encodes probabilistic relationships among variables of interest. When used in conjunction with statistical techniques, the graphical model has several advantages for data analysis. One, because the model encodes dependencies among all variables, it readily handles situations where some data entries are missing. Two, a Bayesian network can be used to learn causal relationships, and hence can be used to gain understanding about a problem domain and to predict the consequences of intervention. Three, because the model has both a causal and probabilistic semantics, it is an ideal representation for combining prior knowledge (which often comes in causal form) and data. Four, Bayesian statistical methods in conjunction with Bayesian networks offer an efficient and principled approach for avoiding the overfitting of data. In this paper, we discuss methods for constructing Bayesian networks from prior knowledge and summarize Bayesian statistical methods for using data to improve these models. With regard to the latter task, we describe methods for learning both the parameters and structure of a Bayesian network, including techniques for learning with incomplete data. In addition, we relate Bayesian-network methods for learning to techniques for supervised and unsupervised learning. We illustrate the graphical-modeling approach using a real-world case study.
Tasks
Published	2020-02-01
URL	https://arxiv.org/abs/2002.00269v1
PDF	https://arxiv.org/pdf/2002.00269v1.pdf
PWC	https://paperswithcode.com/paper/a-tutorial-on-learning-with-bayesian-networks
Repo
Framework

Rapid Whole Slide Imaging via Learning-based Two-shot Virtual Autofocusing


Title	Rapid Whole Slide Imaging via Learning-based Two-shot Virtual Autofocusing
Authors	Qiang Li, Xianming Liu, Kaige Han, Cheng Guo, Xiangyang Ji, Xiaolin Wu
Abstract	Whole slide imaging (WSI) is an emerging technology for digital pathology. The process of autofocusing is the main influence of the performance of WSI. Traditional autofocusing methods either are time-consuming due to repetitive mechanical motions, or require additional hardware and thus are not compatible to current WSI systems. In this paper, we propose the concept of \textit{virtual autofocusing}, which does not rely on mechanical adjustment to conduct refocusing but instead recovers in-focus images in an offline learning-based manner. With the initial focal position, we only perform two-shot imaging, in contrast traditional methods commonly need to conduct as many as 21 times image shooting in each tile scanning. Considering that the two captured out-of-focus images retain pieces of partial information about the underlying in-focus image, we propose a U-Net-inspired deep neural network based approach for fusing them into a recovered in-focus image. The proposed scheme is fast in tissue slides scanning, enabling a high-throughput generation of digital pathology images. Experimental results demonstrate that our scheme achieves satisfactory refocusing performance.
Tasks
Published	2020-03-14
URL	https://arxiv.org/abs/2003.06630v1
PDF	https://arxiv.org/pdf/2003.06630v1.pdf
PWC	https://paperswithcode.com/paper/rapid-whole-slide-imaging-via-learning-based
Repo
Framework

Improving Adversarial Robustness Through Progressive Hardening


Title	Improving Adversarial Robustness Through Progressive Hardening
Authors	Chawin Sitawarin, Supriyo Chakraborty, David Wagner
Abstract	Adversarial training (AT) has become a popular choice for training robust networks. However, by virtue of its formulation, AT tends to sacrifice clean accuracy heavily in favor of robustness. Furthermore, AT with a large perturbation budget can cause models to get stuck at poor local minima and behave like a constant function, always predicting the same class. To address the above concerns we propose Adversarial Training with Early Stopping (ATES). The design of ATES is guided by principles from curriculum learning that emphasizes on starting “easy” and gradually ramping up on the “difficulty” of training. We do so by early stopping the adversarial example generation step in AT, progressively increasing difficulty of the samples the network trains on. This stabilizes network training even for large perturbation budgets and allows the network to operate at a better clean accuracy versus robustness trade-off curve compared to AT. Functionally, this leads to a significant improvement in both clean accuracy and robustness for ATES models.
Tasks
Published	2020-03-18
URL	https://arxiv.org/abs/2003.09347v1
PDF	https://arxiv.org/pdf/2003.09347v1.pdf
PWC	https://paperswithcode.com/paper/improving-adversarial-robustness-through
Repo
Framework

Causal Interpretability for Machine Learning – Problems, Methods and Evaluation


Title	Causal Interpretability for Machine Learning – Problems, Methods and Evaluation
Authors	Raha Moraffah, Mansooreh Karami, Ruocheng Guo, Adrienne Raglin, Huan Liu
Abstract	Machine learning models have had discernible achievements in a myriad of applications. However, most of these models are black-boxes, and it is obscure how the decisions are made by them. This makes the models unreliable and untrustworthy. To provide insights into the decision making processes of these models, a variety of traditional interpretable models have been proposed. Moreover, to generate more human-friendly explanations, recent work on interpretability tries to answer questions related to causality such as “Why does this model makes such decisions?” or “Was it a specific feature that caused the decision made by the model?". In this work, models that aim to answer causal questions are referred to as causal interpretable models. The existing surveys have covered concepts and methodologies of traditional interpretability. In this work, we present a comprehensive survey on causal interpretable models from the aspects of the problems and methods. In addition, this survey provides in-depth insights into the existing evaluation metrics for measuring interpretability, which can help practitioners understand for what scenarios each evaluation metric is suitable.
Tasks	Decision Making
Published	2020-03-09
URL	https://arxiv.org/abs/2003.03934v3
PDF	https://arxiv.org/pdf/2003.03934v3.pdf
PWC	https://paperswithcode.com/paper/causal-interpretability-for-machine-learning
Repo
Framework

Variational Encoder-based Reliable Classification


Title	Variational Encoder-based Reliable Classification
Authors	Chitresh Bhushan, Zhaoyuan Yang, Nurali Virani, Naresh Iyer
Abstract	Machine learning models provide statistically impressive results which might be individually unreliable. To provide reliability, we propose an Epistemic Classifier (EC) that can provide justification of its belief using support from the training dataset as well as quality of reconstruction. Our approach is based on modified variational auto-encoders that can identify a semantically meaningful low-dimensional space where perceptually similar instances are close in $\ell_2$-distance too. Our results demonstrate improved reliability of predictions and robust identification of samples with adversarial attacks as compared to baseline of softmax-based thresholding.
Tasks
Published	2020-02-19
URL	https://arxiv.org/abs/2002.08289v1
PDF	https://arxiv.org/pdf/2002.08289v1.pdf
PWC	https://paperswithcode.com/paper/variational-encoder-based-reliable
Repo
Framework

1st Place Solutions for OpenImage2019 – Object Detection and Instance Segmentation


Title	1st Place Solutions for OpenImage2019 – Object Detection and Instance Segmentation
Authors	Yu Liu, Guanglu Song, Yuhang Zang, Yan Gao, Enze Xie, Junjie Yan, Chen Change Loy, Xiaogang Wang
Abstract	This article introduces the solutions of the two champion teams, `MMfruit' for the detection track and` MMfruitSeg’ for the segmentation track, in OpenImage Challenge 2019. It is commonly known that for an object detector, the shared feature at the end of the backbone is not appropriate for both classification and regression, which greatly limits the performance of both single stage detector and Faster RCNN \cite{ren2015faster} based detector. In this competition, we observe that even with a shared feature, different locations in one object has completely inconsistent performances for the two tasks. \textit{E.g. the features of salient locations are usually good for classification, while those around the object edge are good for regression.} Inspired by this, we propose the Decoupling Head (DH) to disentangle the object classification and regression via the self-learned optimal feature extraction, which leads to a great improvement. Furthermore, we adjust the soft-NMS algorithm to adj-NMS to obtain stable performance improvement. Finally, a well-designed ensemble strategy via voting the bounding box location and confidence is proposed. We will also introduce several training/inferencing strategies and a bag of tricks that give minor improvement. Given those masses of details, we train and aggregate 28 global models with various backbones, heads and 3+2 expert models, and achieves the 1st place on the OpenImage 2019 Object Detection Challenge on the both public and private leadboards. Given such good instance bounding box, we further design a simple instance-level semantic segmentation pipeline and achieve the 1st place on the segmentation challenge.
Tasks	Instance Segmentation, Object Classification, Object Detection, Semantic Segmentation
Published	2020-03-17
URL	https://arxiv.org/abs/2003.07557v1
PDF	https://arxiv.org/pdf/2003.07557v1.pdf
PWC	https://paperswithcode.com/paper/1st-place-solutions-for-openimage2019-object
Repo
Framework

A Power-Efficient Binary-Weight Spiking Neural Network Architecture for Real-Time Object Classification


Title	A Power-Efficient Binary-Weight Spiking Neural Network Architecture for Real-Time Object Classification
Authors	Pai-Yu Tan, Po-Yao Chuang, Yen-Ting Lin, Cheng-Wen Wu, Juin-Ming Lu
Abstract	Neural network hardware is considered an essential part of future edge devices. In this paper, we propose a binary-weight spiking neural network (BW-SNN) hardware architecture for low-power real-time object classification on edge platforms. This design stores a full neural network on-chip, and hence requires no off-chip bandwidth. The proposed systolic array maximizes data reuse for a typical convolutional layer. A 5-layer convolutional BW-SNN hardware is implemented in 90nm CMOS. Compared with state-of-the-art designs, the area cost and energy per classification are reduced by 7$\times$ and 23$\times$, respectively, while also achieving a higher accuracy on the MNIST benchmark. This is also a pioneering SNN hardware architecture that supports advanced CNN architectures.
Tasks	Object Classification
Published	2020-03-12
URL	https://arxiv.org/abs/2003.06310v1
PDF	https://arxiv.org/pdf/2003.06310v1.pdf
PWC	https://paperswithcode.com/paper/a-power-efficient-binary-weight-spiking
Repo
Framework

A General Framework for Learning Mean-Field Games


Title	A General Framework for Learning Mean-Field Games
Authors	Xin Guo, Anran Hu, Renyuan Xu, Junzi Zhang
Abstract	This paper presents a general mean-field game (GMFG) framework for simultaneous learning and decision-making in stochastic games with a large population. It first establishes the existence of a unique Nash Equilibrium to this GMFG, and demonstrates that naively combining Q-learning with the fixed-point approach in classical MFGs yields unstable algorithms. It then proposes value-based and policy-based reinforcement learning algorithms (GMF-P and GMF-P respectively) with smoothed policies, with analysis of convergence property and computational complexity. The experiments on repeated Ad auction problems demonstrate that GMF-V-Q, a specific GMF-V algorithm based on Q-learning, is efficient and robust in terms of convergence and learning accuracy. Moreover, its performance is superior in convergence, stability, and learning ability, when compared with existing algorithms for multi-agent reinforcement learning.
Tasks	Decision Making, Multi-agent Reinforcement Learning, Q-Learning
Published	2020-03-13
URL	https://arxiv.org/abs/2003.06069v1
PDF	https://arxiv.org/pdf/2003.06069v1.pdf
PWC	https://paperswithcode.com/paper/a-general-framework-for-learning-mean-field
Repo
Framework

Fixed smooth convolutional layer for avoiding checkerboard artifacts in CNNs


Title	Fixed smooth convolutional layer for avoiding checkerboard artifacts in CNNs
Authors	Yuma Kinoshita, Hitoshi Kiya
Abstract	In this paper, we propose a fixed convolutional layer with an order of smoothness not only for avoiding checkerboard artifacts in convolutional neural networks (CNNs) but also for enhancing the performance of CNNs, where the smoothness of its filter kernel can be controlled by a parameter. It is well-known that a number of CNNs generate checkerboard artifacts in both of two process: forward-propagation of upsampling layers and backward-propagation of strided convolutional layers. The proposed layer can perfectly prevent checkerboard artifacts caused by strided convolutional layers or upsampling layers including transposed convolutional layers. In an image-classification experiment with four CNNs: a simple CNN, VGG8, ResNet-18, and ResNet-101, applying the fixed layers to these CNNs is shown to improve the classification performance of all CNNs. In addition, the fixed layer are applied to generative adversarial networks (GANs), for the first time. From image-generation results, a smoother fixed convolutional layer is demonstrated to enable us to improve the quality of images generated with GANs.
Tasks	Image Classification, Image Generation
Published	2020-02-06
URL	https://arxiv.org/abs/2002.02117v1
PDF	https://arxiv.org/pdf/2002.02117v1.pdf
PWC	https://paperswithcode.com/paper/fixed-smooth-convolutional-layer-for-avoiding
Repo
Framework

Optimal Regularization Can Mitigate Double Descent


Title	Optimal Regularization Can Mitigate Double Descent
Authors	Preetum Nakkiran, Prayaag Venkat, Sham Kakade, Tengyu Ma
Abstract	Recent empirical and theoretical studies have shown that many learning algorithms – from linear regression to neural networks – can have test performance that is non-monotonic in quantities such the sample size and model size. This striking phenomenon, often referred to as “double descent”, has raised questions of if we need to re-think our current understanding of generalization. In this work, we study whether the double-descent phenomenon can be avoided by using optimal regularization. Theoretically, we prove that for certain linear regression models with isotropic data distribution, optimally-tuned $\ell_2$ regularization achieves monotonic test performance as we grow either the sample size or the model size. We also demonstrate empirically that optimally-tuned $\ell_2$ regularization can mitigate double descent for more general models, including neural networks. Our results suggest that it may also be informative to study the test risk scalings of various algorithms in the context of appropriately tuned regularization.
Tasks
Published	2020-03-04
URL	https://arxiv.org/abs/2003.01897v1
PDF	https://arxiv.org/pdf/2003.01897v1.pdf
PWC	https://paperswithcode.com/paper/optimal-regularization-can-mitigate-double
Repo
Framework

Long term planning of military aircraft flight and maintenance operations


Title	Long term planning of military aircraft flight and maintenance operations
Authors	Franco Peschiera, Olga Battaïa, Alain Haït, Nicolas Dupin
Abstract	We present the Flight and Maintenance Planning (FMP) problem in its military variant and applied to long term planning. The problem has been previously studied for short- and medium-term horizons only. We compare its similarities and differences with previous work and prove its complexity. We generate scenarios inspired by the French Air Force fleet. We formulate an exact Mixed Integer Programming (MIP) model to solve the problem in these scenarios and we analyse the performance of the solving method under these circumstances. A heuristic was built to generate fast feasible solutions, that in some cases were shown to help warm-start the model.
Tasks
Published	2020-01-27
URL	https://arxiv.org/abs/2001.09856v1
PDF	https://arxiv.org/pdf/2001.09856v1.pdf
PWC	https://paperswithcode.com/paper/long-term-planning-of-military-aircraft
Repo
Framework