January 26, 2020

3067 words 15 mins read

Paper Group ANR 1384

SMIX($λ$): Enhancing Centralized Value Functions for Cooperative Multi-Agent Reinforcement Learning. Neural Network-based Equalizer by Utilizing Coding Gain in Advance. Capturing Evolution Genes for Time Series Data. Medical Concept Representation Learning from Claims Data and Application to Health Plan Payment Risk Adjustment. Inference of a Multi …

SMIX($λ$): Enhancing Centralized Value Functions for Cooperative Multi-Agent Reinforcement Learning


Title	SMIX($λ$): Enhancing Centralized Value Functions for Cooperative Multi-Agent Reinforcement Learning
Authors	Xinghu Yao, Chao Wen, Yuhui Wang, Xiaoyang Tan
Abstract	Learning a stable and generalizable centralized value function (CVF) is a crucial but challenging task in multi-agent reinforcement learning (MARL), as it has to deal with the issue that the joint action space increases exponentially with the number of agents in such scenarios. This paper proposes an approach, named SMIX(${\lambda}$), that uses an off-policy training to achieve this by avoiding the greedy assumption commonly made in CVF learning. As importance sampling for such off-policy training is both computationally costly and numerically unstable, we proposed to use the ${\lambda}$-return as a proxy to compute the TD error. With this new loss function objective, we adopt a modified QMIX network structure as the base to train our model. By further connecting it with the ${Q(\lambda)}$ approach from an unified expectation correction viewpoint, we show that the proposed SMIX(${\lambda}$) is equivalent to ${Q(\lambda)}$ and hence shares its convergence properties, while without being suffered from the aforementioned curse of dimensionality problem inherent in MARL. Experiments on the StarCraft Multi-Agent Challenge (SMAC) benchmark demonstrate that our approach not only outperforms several state-of-the-art MARL methods by a large margin, but also can be used as a general tool to improve the overall performance of other CTDE-type algorithms by enhancing their CVFs.
Tasks	Multi-agent Reinforcement Learning, Starcraft
Published	2019-11-11
URL	https://arxiv.org/abs/1911.04094v3
PDF	https://arxiv.org/pdf/1911.04094v3.pdf
PWC	https://paperswithcode.com/paper/smix-enhancing-centralized-value-functions
Repo
Framework

Neural Network-based Equalizer by Utilizing Coding Gain in Advance


Title	Neural Network-based Equalizer by Utilizing Coding Gain in Advance
Authors	Chieh-Fang Teng, Han-Mo Ou, An-Yeu Wu
Abstract	Recently, deep learning has been exploited in many fields with revolutionary breakthroughs. In the light of this, deep learning-assisted communication systems have also attracted much attention in recent years and have potential to break down the conventional design rule for communication systems. In this work, we propose two kinds of neural network-based equalizers to exploit different characteristics between convolutional neural networks and recurrent neural networks. The equalizer in conventional block-based design may destroy the code structure and degrade the capacity of coding gain for decoder. On the contrary, our proposed approach not only eliminates channel fading, but also exploits the code structure with utilization of coding gain in advance, which can effectively increase the overall utilization of coding gain with more than 1.5 dB gain.
Tasks
Published	2019-07-11
URL	https://arxiv.org/abs/1907.04980v2
PDF	https://arxiv.org/pdf/1907.04980v2.pdf
PWC	https://paperswithcode.com/paper/neural-network-based-equalizer-by-utilizing
Repo
Framework

Capturing Evolution Genes for Time Series Data


Title	Capturing Evolution Genes for Time Series Data
Authors	Wenjie Hu, Yang Yang, Liang Wu, Zongtao Liu, Zhanlin Sun, Bingshen Yao
Abstract	The modeling of time series is becoming increasingly critical in a wide variety of applications. Overall, data evolves by following different patterns, which are generally caused by different user behaviors. Given a time series, we define the evolution gene to capture the latent user behaviors and to describe how the behaviors lead to the generation of time series. In particular, we propose a uniform framework that recognizes different evolution genes of segments by learning a classifier, and adopt an adversarial generator to implement the evolution gene by estimating the segments’ distribution. Experimental results based on a synthetic dataset and five real-world datasets show that our approach can not only achieve a good prediction results (e.g., averagely +10.56% in terms of F1), but is also able to provide explanations of the results.
Tasks	Time Series
Published	2019-05-10
URL	https://arxiv.org/abs/1905.05004v1
PDF	https://arxiv.org/pdf/1905.05004v1.pdf
PWC	https://paperswithcode.com/paper/190505004
Repo
Framework

Medical Concept Representation Learning from Claims Data and Application to Health Plan Payment Risk Adjustment


Title	Medical Concept Representation Learning from Claims Data and Application to Health Plan Payment Risk Adjustment
Authors	Qiu-Yue Zhong, Andrew H. Fairless, Jasmine M. McCammon, Farbod Rahmanian
Abstract	Risk adjustment has become an increasingly important tool in healthcare. It has been extensively applied to payment adjustment for health plans to reflect the expected cost of providing coverage for members. Risk adjustment models are typically estimated using linear regression, which does not fully exploit the information in claims data. Moreover, the development of such linear regression models requires substantial domain expert knowledge and computational effort for data preprocessing. In this paper, we propose a novel approach for risk adjustment that uses semantic embeddings to represent patient medical histories. Embeddings efficiently represent medical concepts learned from diagnostic, procedure, and prescription codes in patients’ medical histories. This approach substantially reduces the need for feature engineering. Our results show that models using embeddings had better performance than a commercial risk adjustment model on the task of prospective risk score prediction.
Tasks	Feature Engineering, Representation Learning
Published	2019-07-15
URL	https://arxiv.org/abs/1907.06600v1
PDF	https://arxiv.org/pdf/1907.06600v1.pdf
PWC	https://paperswithcode.com/paper/medical-concept-representation-learning-from-1
Repo
Framework

Inference of a Multi-Domain Machine Learning Model to Predict Mortality in Hospital Stays for Patients with Cancer upon Febrile Neutropenia Onset


Title	Inference of a Multi-Domain Machine Learning Model to Predict Mortality in Hospital Stays for Patients with Cancer upon Febrile Neutropenia Onset
Authors	Xinsong Du, Jae Min, Mattia Prosperi, Rohit Bishnoi, Dominick J. Lemas, Chintan P. Shah
Abstract	Febrile neutropenia (FN) has been associated with high mortality, especially among adults with cancer. Understanding the patient and provider level heterogeneity in FN hospital admissions has potential to inform personalized interventions focused on increasing survival of individuals with FN. We leverage machine learning techniques to disentangling the complex interactions among multi domain risk factors in a population with FN. Data from the Healthcare Cost and Utilization Project (HCUP) National Inpatient Sample and Nationwide Inpatient Sample (NIS) were used to build machine learning based models of mortality for adult cancer patients who were diagnosed with FN during a hospital admission. In particular, the importance of risk factors from different domains (including demographic, clinical, and hospital associated information) was studied. A set of more interpretable (decision tree, logistic regression) as well as more black box (random forest, gradient boosting, neural networks) models were analyzed and compared via multiple cross validation. Our results demonstrate that a linear prediction score of FN mortality among adults with cancer, based on admission information is effective in classifying high risk patients; clinical diagnoses is the domain with the highest predictive power. A number of the risk variables (e.g. sepsis, kidney failure, etc.) identified in this study are clinically actionable and may inform future studies looking at the patients prior medical history are warranted.
Tasks	Mortality Prediction
Published	2019-02-21
URL	https://arxiv.org/abs/1902.07839v3
PDF	https://arxiv.org/pdf/1902.07839v3.pdf
PWC	https://paperswithcode.com/paper/inference-of-a-multi-domain-machine-learning
Repo
Framework

EEG-Based Driver Drowsiness Estimation Using Feature Weighted Episodic Training


Title	EEG-Based Driver Drowsiness Estimation Using Feature Weighted Episodic Training
Authors	Yuqi Cuui, Yifan Xu, Dongrui Wu
Abstract	Drowsy driving is pervasive, and also a major cause of traffic accidents. Estimating a driver’s drowsiness level by monitoring the electroencephalogram (EEG) signal and taking preventative actions accordingly may improve driving safety. However, individual differences among different drivers make this task very challenging. A calibration session is usually required to collect some subject-specific data and tune the model parameters before applying it to a new subject, which is very inconvenient and not user-friendly. Many approaches have been proposed to reduce the calibration effort, but few can completely eliminate it. This paper proposes a novel approach, feature weighted episodic training (FWET), to completely eliminate the calibration requirement. It integrates two techniques: feature weighting to learn the importance of different features, and episodic training for domain generalization. Experiments on EEG-based driver drowsiness estimation demonstrated that both feature weighting and episodic training are effective, and their integration can further improve the generalization performance. FWET does not need any labelled or unlabelled calibration data from the new subject, and hence could be very useful in plug-and-play brain-computer interfaces.
Tasks	Calibration, Domain Generalization, EEG
Published	2019-09-25
URL	https://arxiv.org/abs/1909.11456v1
PDF	https://arxiv.org/pdf/1909.11456v1.pdf
PWC	https://paperswithcode.com/paper/eeg-based-driver-drowsiness-estimation-using
Repo
Framework

Improved Structural Discovery and Representation Learning of Multi-Agent Data


Title	Improved Structural Discovery and Representation Learning of Multi-Agent Data
Authors	Jennifer Hobbs, Matthew Holbrook, Nathan Frank, Long Sha, Patrick Lucey
Abstract	Central to all machine learning algorithms is data representation. For multi-agent systems, selecting a representation which adequately captures the interactions among agents is challenging due to the latent group structure which tends to vary depending on context. However, in multi-agent systems with strong group structure, we can simultaneously learn this structure and map a set of agents to a consistently ordered representation for further learning. In this paper, we present a dynamic alignment method which provides a robust ordering of structured multi-agent data enabling representation learning to occur in a fraction of the time of previous methods. We demonstrate the value of this approach using a large amount of soccer tracking data from a professional league.
Tasks	Representation Learning
Published	2019-12-30
URL	https://arxiv.org/abs/1912.13107v1
PDF	https://arxiv.org/pdf/1912.13107v1.pdf
PWC	https://paperswithcode.com/paper/improved-structural-discovery-and-1
Repo
Framework


Title	Vision-Depth Landmarks and Inertial Fusion for Navigation in Degraded Visual Environments
Authors	Shehryar Khattak, Christos Papachristos, Kostas Alexis
Abstract	This paper proposes a method for tight fusion of visual, depth and inertial data in order to extend robotic capabilities for navigation in GPS-denied, poorly illuminated, and texture-less environments. Visual and depth information are fused at the feature detection and descriptor extraction levels to augment one sensing modality with the other. These multimodal features are then further integrated with inertial sensor cues using an extended Kalman filter to estimate the robot pose, sensor bias terms, and landmark positions simultaneously as part of the filter state. As demonstrated through a set of hand-held and Micro Aerial Vehicle experiments, the proposed algorithm is shown to perform reliably in challenging visually-degraded environments using RGB-D information from a lightweight and low-cost sensor and data from an IMU.
Tasks
Published	2019-03-05
URL	http://arxiv.org/abs/1903.01659v1
PDF	http://arxiv.org/pdf/1903.01659v1.pdf
PWC	https://paperswithcode.com/paper/vision-depth-landmarks-and-inertial-fusion
Repo
Framework

Asymptotically unbiased estimation of physical observables with neural samplers


Title	Asymptotically unbiased estimation of physical observables with neural samplers
Authors	Kim A. Nicoli, Shinichi Nakajima, Nils Strodthoff, Wojciech Samek, Klaus-Robert Müller, Pan Kessel
Abstract	We propose a general framework for the estimation of observables with generative neural samplers focusing on modern deep generative neural networks that provide an exact sampling probability. In this framework, we present asymptotically unbiased estimators for generic observables, including those that explicitly depend on the partition function such as free energy or entropy, and derive corresponding variance estimators. We demonstrate their practical applicability by numerical experiments for the 2d Ising model which highlight the superiority over existing methods. Our approach greatly enhances the applicability of generative neural samplers to real-world physical systems.
Tasks
Published	2019-10-29
URL	https://arxiv.org/abs/1910.13496v2
PDF	https://arxiv.org/pdf/1910.13496v2.pdf
PWC	https://paperswithcode.com/paper/asymptotically-unbiased-generative-neural
Repo
Framework

BERT Rediscovers the Classical NLP Pipeline


Title	BERT Rediscovers the Classical NLP Pipeline
Authors	Ian Tenney, Dipanjan Das, Ellie Pavlick
Abstract	Pre-trained text encoders have rapidly advanced the state of the art on many NLP tasks. We focus on one such model, BERT, and aim to quantify where linguistic information is captured within the network. We find that the model represents the steps of the traditional NLP pipeline in an interpretable and localizable way, and that the regions responsible for each step appear in the expected sequence: POS tagging, parsing, NER, semantic roles, then coreference. Qualitative analysis reveals that the model can and often does adjust this pipeline dynamically, revising lower-level decisions on the basis of disambiguating information from higher-level representations.
Tasks	Named Entity Recognition
Published	2019-05-15
URL	https://arxiv.org/abs/1905.05950v2
PDF	https://arxiv.org/pdf/1905.05950v2.pdf
PWC	https://paperswithcode.com/paper/bert-rediscovers-the-classical-nlp-pipeline
Repo
Framework

Dis-entangling Mixture of Interventions on a Causal Bayesian Network Using Aggregate Observations


Title	Dis-entangling Mixture of Interventions on a Causal Bayesian Network Using Aggregate Observations
Authors	Gaurav Sinha, Ayush Chauhan, Aurghya Maiti, Naman Poddar, Pulkit Goel
Abstract	We study the problem of separating a mixture of distributions, all of which come from interventions on a known causal bayesian network. Given oracle access to marginals of all distributions resulting from interventions on the network, and estimates of marginals from the mixture distribution, we want to recover the mixing proportions of different mixture components. We show that in the worst case, mixing proportions cannot be identified using marginals only. If exact marginals of the mixture distribution were known, under a simple assumption of excluding a few distributions from the mixture, we show that the mixing proportions become identifiable. Our identifiability proof is constructive and gives an efficient algorithm recovering the mixing proportions exactly. When exact marginals are not available, we design an optimization framework to estimate the mixing proportions. Our problem is motivated from a real-world scenario of an e-commerce business, where multiple interventions occur at a given time, leading to deviations in expected metrics. We conduct experiments on the well known publicly available ALARM network and on a proprietary dataset from a large e-commerce company validating the performance of our method.
Tasks
Published	2019-11-30
URL	https://arxiv.org/abs/1912.00163v2
PDF	https://arxiv.org/pdf/1912.00163v2.pdf
PWC	https://paperswithcode.com/paper/dis-entangling-mixture-of-interventions-on-a
Repo
Framework

BackPACK: Packing more into backprop


Title	BackPACK: Packing more into backprop
Authors	Felix Dangel, Frederik Kunstner, Philipp Hennig
Abstract	Automatic differentiation frameworks are optimized for exactly one thing: computing the average mini-batch gradient. Yet, other quantities such as the variance of the mini-batch gradients or many approximations to the Hessian can, in theory, be computed efficiently, and at the same time as the gradient. While these quantities are of great interest to researchers and practitioners, current deep-learning software does not support their automatic calculation. Manually implementing them is burdensome, inefficient if done naively, and the resulting code is rarely shared. This hampers progress in deep learning, and unnecessarily narrows research to focus on gradient descent and its variants; it also complicates replication studies and comparisons between newly developed methods that require those quantities, to the point of impossibility. To address this problem, we introduce BackPACK, an efficient framework built on top of PyTorch, that extends the backpropagation algorithm to extract additional information from first- and second-order derivatives. Its capabilities are illustrated by benchmark reports for computing additional quantities on deep neural networks, and an example application by testing several recent curvature approximations for optimization.
Tasks
Published	2019-12-23
URL	https://arxiv.org/abs/1912.10985v2
PDF	https://arxiv.org/pdf/1912.10985v2.pdf
PWC	https://paperswithcode.com/paper/backpack-packing-more-into-backprop
Repo
Framework

New Computational and Statistical Aspects of Regularized Regression with Application to Rare Feature Selection and Aggregation


Title	New Computational and Statistical Aspects of Regularized Regression with Application to Rare Feature Selection and Aggregation
Authors	Amin Jalali, Adel Javanmard, Maryam Fazel
Abstract	Prior knowledge on properties of a target model often come as discrete or combinatorial descriptions. This work provides a unified computational framework for defining norms that promote such structures. More specifically, we develop associated tools for optimization involving such norms given only the orthogonal projection oracle onto the non-convex set of desired models. As an example, we study a norm, which we term the doubly-sparse norm, for promoting vectors with few nonzero entries taking only a few distinct values. We further discuss how the K-means algorithm can serve as the underlying projection oracle in this case and how it can be efficiently represented as a quadratically constrained quadratic program. Our motivation for the study of this norm is regularized regression in the presence of rare features which poses a challenge to various methods within high-dimensional statistics, and in machine learning in general. The proposed estimation procedure is designed to perform automatic feature selection and aggregation for which we develop statistical bounds. The bounds are general and offer a statistical framework for norm-based regularization. The bounds rely on novel geometric quantities on which we attempt to elaborate as well.
Tasks	Feature Selection
Published	2019-04-10
URL	http://arxiv.org/abs/1904.05338v1
PDF	http://arxiv.org/pdf/1904.05338v1.pdf
PWC	https://paperswithcode.com/paper/new-computational-and-statistical-aspects-of
Repo
Framework

Shaping the learning landscape in neural networks around wide flat minima


Title	Shaping the learning landscape in neural networks around wide flat minima
Authors	Carlo Baldassi, Fabrizio Pittorino, Riccardo Zecchina
Abstract	Learning in Deep Neural Networks (DNN) takes place by minimizing a non-convex high-dimensional loss function, typically by a stochastic gradient descent (SGD) strategy. The learning process is observed to be able to find good minimizers without getting stuck in local critical points, and that such minimizers are often satisfactory at avoiding overfitting. How these two features can be kept under control in nonlinear devices composed of millions of tunable connections is a profound and far reaching open question. In this paper we study basic non-convex one- and two-layer neural network models which learn random patterns, and derive a number of basic geometrical and algorithmic features which suggest some answers. We first show that the error loss function presents few extremely wide flat minima (WFM) which coexist with narrower minima and critical points. We then show that the minimizers of the cross-entropy loss function overlap with the WFM of the error loss. We also show examples of learning devices for which WFM do not exist. From the algorithmic perspective we derive entropy driven greedy and message passing algorithms which focus their search on wide flat regions of minimizers. In the case of SGD and cross-entropy loss, we show that a slow reduction of the norm of the weights along the learning process also leads to WFM. We corroborate the results by a numerical study of the correlations between the volumes of the minimizers, their Hessian and their generalization performance on real data.
Tasks
Published	2019-05-20
URL	https://arxiv.org/abs/1905.07833v4
PDF	https://arxiv.org/pdf/1905.07833v4.pdf
PWC	https://paperswithcode.com/paper/shaping-the-learning-landscape-in-neural
Repo
Framework

Bin-wise Temperature Scaling (BTS): Improvement in Confidence Calibration Performance through Simple Scaling Techniques


Title	Bin-wise Temperature Scaling (BTS): Improvement in Confidence Calibration Performance through Simple Scaling Techniques
Authors	Byeongmoon Ji, Hyemin Jung, Jihyeun Yoon, Kyungyul Kim, Younghak Shin
Abstract	The prediction reliability of neural networks is important in many applications. Specifically, in safety-critical domains, such as cancer prediction or autonomous driving, a reliable confidence of model’s prediction is critical for the interpretation of the results. Modern deep neural networks have achieved a significant improvement in performance for many different image classification tasks. However, these networks tend to be poorly calibrated in terms of output confidence. Temperature scaling is an efficient post-processing-based calibration scheme and obtains well calibrated results. In this study, we leverage the concept of temperature scaling to build a sophisticated bin-wise scaling. Furthermore, we adopt augmentation of validation samples for elaborated scaling. The proposed methods consistently improve calibration performance with various datasets and deep convolutional neural network models.
Tasks	Autonomous Driving, Calibration, Image Classification
Published	2019-08-30
URL	https://arxiv.org/abs/1908.11528v2
PDF	https://arxiv.org/pdf/1908.11528v2.pdf
PWC	https://paperswithcode.com/paper/bin-wise-temperature-scaling-bts-improvement
Repo
Framework