January 30, 2020

2817 words 14 mins read

Paper Group ANR 258

A Framework for End-to-End Deep Learning-Based Anomaly Detection in Transportation Networks. Infochain: A Decentralized System for Truthful Information Elicitation. Placement Optimization of Aerial Base Stations with Deep Reinforcement Learning. Precision Matrix Estimation with Noisy and Missing Data. Deep Verifier Networks: Verification of Deep Di …

A Framework for End-to-End Deep Learning-Based Anomaly Detection in Transportation Networks


Title	A Framework for End-to-End Deep Learning-Based Anomaly Detection in Transportation Networks
Authors	Neema Davis, Gaurav Raina, Krishna Jagannathan
Abstract	We develop an end-to-end deep learning-based anomaly detection model for temporal data in transportation networks. The proposed EVT-LSTM model is derived from the popular LSTM (Long Short-Term Memory) network and adopts an objective function that is based on fundamental results from EVT (Extreme Value Theory). We compare the EVT-LSTM model with some established statistical, machine learning, and hybrid deep learning baselines. Experiments on seven diverse real-world data sets demonstrate the superior anomaly detection performance of our proposed model over the other models considered in the comparison study.
Tasks	Anomaly Detection
Published	2019-11-20
URL	https://arxiv.org/abs/1911.08793v1
PDF	https://arxiv.org/pdf/1911.08793v1.pdf
PWC	https://paperswithcode.com/paper/a-framework-for-end-to-end-deep-learning
Repo
Framework

Infochain: A Decentralized System for Truthful Information Elicitation


Title	Infochain: A Decentralized System for Truthful Information Elicitation
Authors	Cyril van Schreven, Naman Goel, Boi Faltings
Abstract	Incentive mechanisms play a pivotal role in collecting correct and reliable information from self-interested agents. Peer-prediction mechanisms are game-theoretic mechanisms that incentivize agents for reporting the information truthfully, even when the information is unverifiable in nature. Traditionally, a trusted third party implements these mechanisms. We built Infochain, a decentralized system for information elicitation. Infochain ensures transparent, trustless and cost-efficient collection of information from self-interested agents without compromising the game-theoretical guarantees of the peer-prediction mechanisms. In this paper, we address various non-trivial challenges in implementing these mechanisms in Ethereum and provide experimental analysis.
Tasks
Published	2019-08-27
URL	https://arxiv.org/abs/1908.10258v1
PDF	https://arxiv.org/pdf/1908.10258v1.pdf
PWC	https://paperswithcode.com/paper/infochain-a-decentralized-system-for-truthful
Repo
Framework

Placement Optimization of Aerial Base Stations with Deep Reinforcement Learning


Title	Placement Optimization of Aerial Base Stations with Deep Reinforcement Learning
Authors	Jin Qiu, Jiangbin Lyu, Liqun Fu
Abstract	Unmanned aerial vehicles (UAVs) can be utilized as aerial base stations (ABSs) to assist terrestrial infrastructure for keeping wireless connectivity in various emergency scenarios. To maximize the coverage rate of N ground users (GUs) by jointly placing multiple ABSs with limited coverage range is known to be a NP-hard problem with exponential complexity in N. The problem is further complicated when the coverage range becomes irregular due to site-specific blockage (e.g., buildings) on the air-ground channel in the 3-dimensional (3D) space. To tackle this challenging problem, this paper applies the Deep Reinforcement Learning (DRL) method by 1) representing the state by a coverage bitmap to capture the spatial correlation of GUs/ABSs, whose dimension and associated neural network complexity is invariant with arbitrarily large N; and 2) designing the action and reward for the DRL agent to effectively learn from the dynamic interactions with the complicated propagation environment represented by a 3D Terrain Map. Specifically, a novel two-level design approach is proposed, consisting of a preliminary design based on the dominant line-of-sight (LoS) channel model, and an advanced design to further refine the ABS positions based on site-specific LoS/non-LoS channel states. The double deep Q-network (DQN) with Prioritized Experience Replay (Prioritized Replay DDQN) algorithm is applied to train the policy of multi-ABS placement decision. Numerical results show that the proposed approach significantly improves the coverage rate in complex environment, compared to the benchmark DQN and K-means algorithms.
Tasks
Published	2019-11-19
URL	https://arxiv.org/abs/1911.08111v2
PDF	https://arxiv.org/pdf/1911.08111v2.pdf
PWC	https://paperswithcode.com/paper/placement-optimization-of-aerial-base
Repo
Framework

Precision Matrix Estimation with Noisy and Missing Data


Title	Precision Matrix Estimation with Noisy and Missing Data
Authors	Roger Fan, Byoungwook Jang, Yuekai Sun, Shuheng Zhou
Abstract	Estimating conditional dependence graphs and precision matrices are some of the most common problems in modern statistics and machine learning. When data are fully observed, penalized maximum likelihood-type estimators have become standard tools for estimating graphical models under sparsity conditions. Extensions of these methods to more complex settings where data are contaminated with additive or multiplicative noise have been developed in recent years. In these settings, however, the relative performance of different methods is not well understood and algorithmic gaps still exist. In particular, in high-dimensional settings these methods require using non-positive semidefinite matrices as inputs, presenting novel optimization challenges. We develop an alternating direction method of multipliers (ADMM) algorithm for these problems, providing a feasible algorithm to estimate precision matrices with indefinite input and potentially nonconvex penalties. We compare this method with existing alternative solutions and empirically characterize the tradeoffs between them. Finally, we use this method to explore the networks among US senators estimated from voting records data.
Tasks
Published	2019-04-07
URL	http://arxiv.org/abs/1904.03548v1
PDF	http://arxiv.org/pdf/1904.03548v1.pdf
PWC	https://paperswithcode.com/paper/precision-matrix-estimation-with-noisy-and
Repo
Framework

Deep Verifier Networks: Verification of Deep Discriminative Models with Deep Generative Models


Title	Deep Verifier Networks: Verification of Deep Discriminative Models with Deep Generative Models
Authors	Tong Che, Xiaofeng Liu, Site Li, Yubin Ge, Ruixiang Zhang, Caiming Xiong, Yoshua Bengio
Abstract	AI Safety is a major concern in many deep learning applications such as autonomous driving. Given a trained deep learning model, an important natural problem is how to reliably verify the model’s prediction. In this paper, we propose a novel framework — deep verifier networks (DVN) to verify the inputs and outputs of deep discriminative models with deep generative models. Our proposed model is based on conditional variational auto-encoders with disentanglement constraints. We give both intuitive and theoretical justifications of the model. Our verifier network is trained independently with the prediction model, which eliminates the need of retraining the verifier network for a new model. We test the verifier network on out-of-distribution detection and adversarial example detection problems, as well as anomaly detection problems in structured prediction tasks such as image caption generation. We achieve state-of-the-art results in all of these problems.
Tasks	Anomaly Detection, Autonomous Driving, Out-of-Distribution Detection, Structured Prediction
Published	2019-11-18
URL	https://arxiv.org/abs/1911.07421v2
PDF	https://arxiv.org/pdf/1911.07421v2.pdf
PWC	https://paperswithcode.com/paper/deep-verifier-networks-verification-of-deep
Repo
Framework

Learning in Gated Neural Networks


Title	Learning in Gated Neural Networks
Authors	Ashok Vardhan Makkuva, Sewoong Oh, Sreeram Kannan, Pramod Viswanath
Abstract	Gating is a key feature in modern neural networks including LSTMs, GRUs and sparsely-gated deep neural networks. The backbone of such gated networks is a mixture-of-experts layer, where several experts make regression decisions and gating controls how to weigh the decisions in an input-dependent manner. Despite having such a prominent role in both modern and classical machine learning, very little is understood about parameter recovery of mixture-of-experts since gradient descent and EM algorithms are known to be stuck in local optima in such models. In this paper, we perform a careful analysis of the optimization landscape and show that with appropriately designed loss functions, gradient descent can indeed learn the parameters accurately. A key idea underpinning our results is the design of two {\em distinct} loss functions, one for recovering the expert parameters and another for recovering the gating parameters. We demonstrate the first sample complexity results for parameter recovery in this model for any algorithm and demonstrate significant performance gains over standard loss functions in numerical experiments.
Tasks
Published	2019-06-06
URL	https://arxiv.org/abs/1906.02777v1
PDF	https://arxiv.org/pdf/1906.02777v1.pdf
PWC	https://paperswithcode.com/paper/learning-in-gated-neural-networks
Repo
Framework

Adaptative Inference Cost With Convolutional Neural Mixture Models


Title	Adaptative Inference Cost With Convolutional Neural Mixture Models
Authors	Adria Ruiz, Jakob Verbeek
Abstract	Despite the outstanding performance of convolutional neural networks (CNNs) for many vision tasks, the required computational cost during inference is problematic when resources are limited. In this context, we propose Convolutional Neural Mixture Models (CNMMs), a probabilistic model embedding a large number of CNNs that can be jointly trained and evaluated in an efficient manner. Within the proposed framework, we present different mechanisms to prune subsets of CNNs from the mixture, allowing to easily adapt the computational cost required for inference. Image classification and semantic segmentation experiments show that our method achieve excellent accuracy-compute trade-offs. Moreover, unlike most of previous approaches, a single CNMM provides a large range of operating points along this trade-off, without any re-training.
Tasks	Image Classification, Semantic Segmentation
Published	2019-08-19
URL	https://arxiv.org/abs/1908.06694v1
PDF	https://arxiv.org/pdf/1908.06694v1.pdf
PWC	https://paperswithcode.com/paper/adaptative-inference-cost-with-convolutional
Repo
Framework

Fitting A Mixture Distribution to Data: Tutorial


Title	Fitting A Mixture Distribution to Data: Tutorial
Authors	Benyamin Ghojogh, Aydin Ghojogh, Mark Crowley, Fakhri Karray
Abstract	This paper is a step-by-step tutorial for fitting a mixture distribution to data. It merely assumes the reader has the background of calculus and linear algebra. Other required background is briefly reviewed before explaining the main algorithm. In explaining the main algorithm, first, fitting a mixture of two distributions is detailed and examples of fitting two Gaussians and Poissons, respectively for continuous and discrete cases, are introduced. Thereafter, fitting several distributions in general case is explained and examples of several Gaussians (Gaussian Mixture Model) and Poissons are again provided. Model-based clustering, as one of the applications of mixture distributions, is also introduced. Numerical simulations are also provided for both Gaussian and Poisson examples for the sake of better clarification.
Tasks
Published	2019-01-20
URL	http://arxiv.org/abs/1901.06708v1
PDF	http://arxiv.org/pdf/1901.06708v1.pdf
PWC	https://paperswithcode.com/paper/fitting-a-mixture-distribution-to-data
Repo
Framework

Personalized Colorectal Cancer Survivability Prediction with Machine Learning Methods


Title	Personalized Colorectal Cancer Survivability Prediction with Machine Learning Methods
Authors	Samuel Li, Talayeh Razzaghi
Abstract	In this work, we investigate the importance of ethnicity in colorectal cancer survivability prediction using machine learning techniques and the SEER cancer incidence database. We compare model performances for 2-year survivability prediction and feature importance rankings between Hispanic, White, and mixed patient populations. Our models consistently perform better on single-ethnicity populations and provide different feature importance rankings when trained in different populations. Additionally, we show our models achieve higher Area Under Curve (AUC) score than the best reported in the literature. We also apply imbalanced classification techniques to improve classification performance when the number of patients who have survived from colorectal cancer is much larger than who have not. These results provide evidence in favor for increased consideration of patient ethnicity in cancer survivability prediction, and for more personalized medicine in general.
Tasks	Feature Importance
Published	2019-01-12
URL	http://arxiv.org/abs/1901.03896v1
PDF	http://arxiv.org/pdf/1901.03896v1.pdf
PWC	https://paperswithcode.com/paper/personalized-colorectal-cancer-survivability
Repo
Framework

A Cycle-GAN Approach to Model Natural Perturbations in Speech for ASR Applications


Title	A Cycle-GAN Approach to Model Natural Perturbations in Speech for ASR Applications
Authors	Sri Harsha Dumpala, Imran Sheikh, Rupayan Chakraborty, Sunil Kumar Kopparapu
Abstract	Naturally introduced perturbations in audio signal, caused by emotional and physical states of the speaker, can significantly degrade the performance of Automatic Speech Recognition (ASR) systems. In this paper, we propose a front-end based on Cycle-Consistent Generative Adversarial Network (CycleGAN) which transforms naturally perturbed speech into normal speech, and hence improves the robustness of an ASR system. The CycleGAN model is trained on non-parallel examples of perturbed and normal speech. Experiments on spontaneous laughter-speech and creaky-speech datasets show that the performance of four different ASR systems improve by using speech obtained from CycleGAN based front-end, as compared to directly using the original perturbed speech. Visualization of the features of the laughter perturbed speech and those generated by the proposed front-end further demonstrates the effectiveness of our approach.
Tasks	Speech Recognition
Published	2019-12-18
URL	https://arxiv.org/abs/1912.11151v1
PDF	https://arxiv.org/pdf/1912.11151v1.pdf
PWC	https://paperswithcode.com/paper/a-cycle-gan-approach-to-model-natural
Repo
Framework

Structure of Deep Neural Networks with a Priori Information in Wireless Tasks


Title	Structure of Deep Neural Networks with a Priori Information in Wireless Tasks
Authors	Jia Guo, Chenyang Yang
Abstract	Deep neural networks (DNNs) have been employed for designing wireless networks in many aspects, such as transceiver optimization, resource allocation, and information prediction. Existing works either use fully-connected DNN or the DNNs with specific structures that are designed in other domains. In this paper, we show that a priori information widely existed in wireless tasks is permutation invariant. For these tasks, we propose a DNN with special structure, where the weight matrices between layers of the DNN only consist of two smaller sub-matrices. By such way of parameter sharing, the number of model parameters reduces, giving rise to low sample and computational complexity for training a DNN. We take predictive resource allocation as an example to show how the designed DNN can be applied for learning the optimal policy with unsupervised learning. Simulations results validate our analysis and show dramatic gain of the proposed structure in terms of reducing training complexity.
Tasks
Published	2019-10-30
URL	https://arxiv.org/abs/1910.13728v2
PDF	https://arxiv.org/pdf/1910.13728v2.pdf
PWC	https://paperswithcode.com/paper/structure-of-deep-neural-networks-with-a
Repo
Framework

Nearly Minimax-Optimal Regret for Linearly Parameterized Bandits


Title	Nearly Minimax-Optimal Regret for Linearly Parameterized Bandits
Authors	Yingkai Li, Yining Wang, Yuan Zhou
Abstract	We study the linear contextual bandit problem with finite action sets. When the problem dimension is $d$, the time horizon is $T$, and there are $n \leq 2^{d/2}$ candidate actions per time period, we (1) show that the minimax expected regret is $\Omega(\sqrt{dT \log T \log n})$ for every algorithm, and (2) introduce a Variable-Confidence-Level (VCL) SupLinUCB algorithm whose regret matches the lower bound up to iterated logarithmic factors. Our algorithmic result saves two $\sqrt{\log T}$ factors from previous analysis, and our information-theoretical lower bound also improves previous results by one $\sqrt{\log T}$ factor, revealing a regret scaling quite different from classical multi-armed bandits in which no logarithmic $T$ term is present in minimax regret. Our proof techniques include variable confidence levels and a careful analysis of layer sizes of SupLinUCB on the upper bound side, and delicately constructed adversarial sequences showing the tightness of elliptical potential lemmas on the lower bound side.
Tasks	Multi-Armed Bandits
Published	2019-03-30
URL	http://arxiv.org/abs/1904.00242v1
PDF	http://arxiv.org/pdf/1904.00242v1.pdf
PWC	https://paperswithcode.com/paper/nearly-minimax-optimal-regret-for-linearly
Repo
Framework

Unifying Structure Analysis and Surrogate-driven Function Regression for Glaucoma OCT Image Screening


Title	Unifying Structure Analysis and Surrogate-driven Function Regression for Glaucoma OCT Image Screening
Authors	Xi Wang, Hao Chen, Luyang Luo, An-ran Ran, Poemen P. Chan, Clement C. Tham, Carol Y. Cheung, Pheng-Ann Heng
Abstract	Optical Coherence Tomography (OCT) imaging plays an important role in glaucoma diagnosis in clinical practice. Early detection and timely treatment can prevent glaucoma patients from permanent vision loss. However, only a dearth of automated methods has been developed based on OCT images for glaucoma study. In this paper, we present a novel framework to effectively classify glaucoma OCT images from normal ones. A semi-supervised learning strategy with smoothness assumption is applied for surrogate assignment of missing function regression labels. Besides, the proposed multi-task learning network is capable of exploring the structure and function relationship from the OCT image and visual field measurement simultaneously, which contributes to classification performance boosting. Essentially, we are the first to unify the structure analysis and function regression for glaucoma screening. It is also worth noting that we build the largest glaucoma OCT image dataset involving 4877 volumes to develop and evaluate the proposed method. Extensive experiments demonstrate that our framework outperforms the baseline methods and two glaucoma experts by a large margin, achieving 93.2%, 93.2% and 97.8% on accuracy, F1 score and AUC, respectively.
Tasks	Multi-Task Learning
Published	2019-07-26
URL	https://arxiv.org/abs/1907.12927v1
PDF	https://arxiv.org/pdf/1907.12927v1.pdf
PWC	https://paperswithcode.com/paper/unifying-structure-analysis-and-surrogate
Repo
Framework

Minimax Optimal Online Stochastic Learning for Sequences of Convex Functions under Sub-Gradient Observation Failures


Title	Minimax Optimal Online Stochastic Learning for Sequences of Convex Functions under Sub-Gradient Observation Failures
Authors	Hakan Gokcesu, Suleyman S. Kozat
Abstract	We study online convex optimization under stochastic sub-gradient observation faults, where we introduce adaptive algorithms with minimax optimal regret guarantees. We specifically study scenarios where our sub-gradient observations can be noisy or even completely missing in a stochastic manner. To this end, we propose algorithms based on sub-gradient descent method, which achieve tight minimax optimal regret bounds. When necessary, these algorithms utilize properties of the underlying stochastic settings to optimize their learning rates (step sizes). These optimizations are the main factor in providing the minimax optimal performance guarantees, especially when observations are stochastically missing. However, in real world scenarios, these properties of the underlying stochastic settings may not be revealed to the optimizer. For such a scenario, we propose a blind algorithm that estimates these properties empirically in a generally applicable manner. Through extensive experiments, we show that this empirical approach is a natural combination of regular stochastic gradient descent and the minimax optimal algorithms (which work best for randomized and adversarial function sequences, respectively).
Tasks
Published	2019-04-19
URL	http://arxiv.org/abs/1904.09369v1
PDF	http://arxiv.org/pdf/1904.09369v1.pdf
PWC	https://paperswithcode.com/paper/190409369
Repo
Framework

Perturbed-History Exploration in Stochastic Multi-Armed Bandits


Title	Perturbed-History Exploration in Stochastic Multi-Armed Bandits
Authors	Branislav Kveton, Csaba Szepesvari, Mohammad Ghavamzadeh, Craig Boutilier
Abstract	We propose an online algorithm for cumulative regret minimization in a stochastic multi-armed bandit. The algorithm adds $O(t)$ i.i.d. pseudo-rewards to its history in round $t$ and then pulls the arm with the highest average reward in its perturbed history. Therefore, we call it perturbed-history exploration (PHE). The pseudo-rewards are carefully designed to offset potentially underestimated mean rewards of arms with a high probability. We derive near-optimal gap-dependent and gap-free bounds on the $n$-round regret of PHE. The key step in our analysis is a novel argument that shows that randomized Bernoulli rewards lead to optimism. Finally, we empirically evaluate PHE and show that it is competitive with state-of-the-art baselines.
Tasks	Multi-Armed Bandits
Published	2019-02-26
URL	https://arxiv.org/abs/1902.10089v2
PDF	https://arxiv.org/pdf/1902.10089v2.pdf
PWC	https://paperswithcode.com/paper/perturbed-history-exploration-in-stochastic
Repo
Framework