January 25, 2020

3237 words 16 mins read

Paper Group ANR 1681

Learning ASR-Robust Contextualized Embeddings for Spoken Language Understanding. Autonomous Highway Driving using Deep Reinforcement Learning. Accelerometer-Based Gait Segmentation: Simultaneously User and Adversary Identification. A Commentary on “Breaking Row and Column Symmetries in Matrix Models”. Transfer Learning-Based Outdoor Position Recove …

Learning ASR-Robust Contextualized Embeddings for Spoken Language Understanding


Title	Learning ASR-Robust Contextualized Embeddings for Spoken Language Understanding
Authors	Chao-Wei Huang, Yun-Nung Chen
Abstract	Employing pre-trained language models (LM) to extract contextualized word representations has achieved state-of-the-art performance on various NLP tasks. However, applying this technique to noisy transcripts generated by automatic speech recognizer (ASR) is concerned. Therefore, this paper focuses on making contextualized representations more ASR-robust. We propose a novel confusion-aware fine-tuning method to mitigate the impact of ASR errors to pre-trained LMs. Specifically, we fine-tune LMs to produce similar representations for acoustically confusable words that are obtained from word confusion networks (WCNs) produced by ASR. Experiments on the benchmark ATIS dataset show that the proposed method significantly improves the performance of spoken language understanding when performing on ASR transcripts.
Tasks	Spoken Language Understanding
Published	2019-09-24
URL	https://arxiv.org/abs/1909.10861v1
PDF	https://arxiv.org/pdf/1909.10861v1.pdf
PWC	https://paperswithcode.com/paper/learning-asr-robust-contextualized-embeddings
Repo
Framework

Autonomous Highway Driving using Deep Reinforcement Learning


Title	Autonomous Highway Driving using Deep Reinforcement Learning
Authors	Subramanya Nageshrao, Eric Tseng, Dimitar Filev
Abstract	The operational space of an autonomous vehicle (AV) can be diverse and vary significantly. This may lead to a scenario that was not postulated in the design phase. Due to this, formulating a rule based decision maker for selecting maneuvers may not be ideal. Similarly, it may not be effective to design an a-priori cost function and then solve the optimal control problem in real-time. In order to address these issues and to avoid peculiar behaviors when encountering unforeseen scenario, we propose a reinforcement learning (RL) based method, where the ego car, i.e., an autonomous vehicle, learns to make decisions by directly interacting with simulated traffic. The decision maker for AV is implemented as a deep neural network providing an action choice for a given system state. In a critical application such as driving, an RL agent without explicit notion of safety may not converge or it may need extremely large number of samples before finding a reliable policy. To best address the issue, this paper incorporates reinforcement learning with an additional short horizon safety check (SC). In a critical scenario, the safety check will also provide an alternate safe action to the agent provided if it exists. This leads to two novel contributions. First, it generalizes the states that could lead to undesirable “near-misses” or “collisions “. Second, inclusion of safety check can provide a safe and stable training environment. This significantly enhances learning efficiency without inhibiting meaningful exploration to ensure safe and optimal learned behavior. We demonstrate the performance of the developed algorithm in highway driving scenario where the trained AV encounters varying traffic density in a highway setting.
Tasks
Published	2019-03-29
URL	http://arxiv.org/abs/1904.00035v1
PDF	http://arxiv.org/pdf/1904.00035v1.pdf
PWC	https://paperswithcode.com/paper/autonomous-highway-driving-using-deep
Repo
Framework

Accelerometer-Based Gait Segmentation: Simultaneously User and Adversary Identification


Title	Accelerometer-Based Gait Segmentation: Simultaneously User and Adversary Identification
Authors	Yujia Ding, Weiqing Gu
Abstract	In this paper, we introduce a new gait segmentation method based on accelerometer data and develop a new distance function between two time series, showing novel and effectiveness in simultaneously identifying user and adversary. Comparing with the normally used Neural Network methods, our approaches use geometric features to extract walking cycles more precisely and employ a new similarity metric to conduct user-adversary identification. This new technology for simultaneously identify user and adversary contributes to cybersecurity beyond user-only identification. In particular, the new technology is being applied to cell phone recorded walking data and performs an accuracy of $98.79%$ for 6 classes classification (user-adversary identification) and $99.06%$ for binary classification (user only identification). In addition to walking signal, our approach works on walking up, walking down and mixed walking signals. This technology is feasible for both large and small data set, overcoming the current challenges facing to Neural Networks such as tuning large number of hyper-parameters for large data sets and lacking of training data for small data sets. In addition, the new distance function developed here can be applied in any signal analysis.
Tasks	Time Series
Published	2019-10-11
URL	https://arxiv.org/abs/1910.06149v1
PDF	https://arxiv.org/pdf/1910.06149v1.pdf
PWC	https://paperswithcode.com/paper/accelerometer-based-gait-segmentation
Repo
Framework

A Commentary on “Breaking Row and Column Symmetries in Matrix Models”


Title	A Commentary on “Breaking Row and Column Symmetries in Matrix Models”
Authors	Alan M. Frisch, Brahim Hnich, Zeynep Kiziltan, Ian Miguel, Toby Walsh
Abstract	The CP 2002 paper entitled “Breaking Row and Column Symmetries in Matrix Models” by Flener et al. (https://link.springer.com/chapter/10.1007%2F3-540-46135-3_31) describes some of the first work for identifying and analyzing row and column symmetry in matrix models and for efficiently and effectively dealing with such symmetry using static symmetry-breaking ordering constraints. This commentary provides a retrospective on that work and highlights some of the subsequent work on the topic.
Tasks
Published	2019-10-03
URL	https://arxiv.org/abs/1910.01423v1
PDF	https://arxiv.org/pdf/1910.01423v1.pdf
PWC	https://paperswithcode.com/paper/a-commentary-on-breaking-row-and-column
Repo
Framework

Transfer Learning-Based Outdoor Position Recovery with Telco Data


Title	Transfer Learning-Based Outdoor Position Recovery with Telco Data
Authors	Yige Zhang, Aaron Yi Ding, Jorg Ott, Mingxuan Yuan, Jia Zeng, Kun Zhang, Weixiong Rao
Abstract	Telecommunication (Telco) outdoor position recovery aims to localize outdoor mobile devices by leveraging measurement report (MR) data. Unfortunately, Telco position recovery requires sufficient amount of MR samples across different areas and suffers from high data collection cost. For an area with scarce MR samples, it is hard to achieve good accuracy. In this paper, by leveraging the recently developed transfer learning techniques, we design a novel Telco position recovery framework, called TLoc, to transfer good models in the carefully selected source domains (those fine-grained small subareas) to a target one which originally suffers from poor localization accuracy. Specifically, TLoc introduces three dedicated components: 1) a new coordinate space to divide an area of interest into smaller domains, 2) a similarity measurement to select best source domains, and 3) an adaptation of an existing transfer learning approach. To the best of our knowledge, TLoc is the first framework that demonstrates the efficacy of applying transfer learning in the Telco outdoor position recovery. To exemplify, on the 2G GSM and 4G LTE MR datasets in Shanghai, TLoc outperforms a nontransfer approach by 27.58% and 26.12% less median errors, and further leads to 47.77% and 49.22% less median errors than a recent fingerprinting approach NBL.
Tasks	Transfer Learning
Published	2019-12-10
URL	https://arxiv.org/abs/1912.04521v1
PDF	https://arxiv.org/pdf/1912.04521v1.pdf
PWC	https://paperswithcode.com/paper/transfer-learning-based-outdoor-position
Repo
Framework

From Reinforcement Learning to Optimal Control: A unified framework for sequential decisions


Title	From Reinforcement Learning to Optimal Control: A unified framework for sequential decisions
Authors	Warren B Powell
Abstract	There are over 15 distinct communities that work in the general area of sequential decisions and information, often referred to as decisions under uncertainty or stochastic optimization. We focus on two of the most important fields: stochastic optimal control, with its roots in deterministic optimal control, and reinforcement learning, with its roots in Markov decision processes. Building on prior work, we describe a unified framework that covers all 15 different communities, and note the strong parallels with the modeling framework of stochastic optimal control. By contrast, we make the case that the modeling framework of reinforcement learning, inherited from discrete Markov decision processes, is quite limited. Our framework (and that of stochastic control) is based on the core problem of optimizing over policies. We describe four classes of policies that we claim are universal, and show that each of these two fields have, in their own way, evolved to include examples of each of these four classes.
Tasks	Stochastic Optimization
Published	2019-12-07
URL	https://arxiv.org/abs/1912.03513v2
PDF	https://arxiv.org/pdf/1912.03513v2.pdf
PWC	https://paperswithcode.com/paper/from-reinforcement-learning-to-optimal
Repo
Framework

Veritatem Dies Aperit- Temporally Consistent Depth Prediction Enabled by a Multi-Task Geometric and Semantic Scene Understanding Approach


Title	Veritatem Dies Aperit- Temporally Consistent Depth Prediction Enabled by a Multi-Task Geometric and Semantic Scene Understanding Approach
Authors	Amir Atapour-Abarghouei, Toby P. Breckon
Abstract	Robust geometric and semantic scene understanding is ever more important in many real-world applications such as autonomous driving and robotic navigation. In this paper, we propose a multi-task learning-based approach capable of jointly performing geometric and semantic scene understanding, namely depth prediction (monocular depth estimation and depth completion) and semantic scene segmentation. Within a single temporally constrained recurrent network, our approach uniquely takes advantage of a complex series of skip connections, adversarial training and the temporal constraint of sequential frame recurrence to produce consistent depth and semantic class labels simultaneously. Extensive experimental evaluation demonstrates the efficacy of our approach compared to other contemporary state-of-the-art techniques.
Tasks	Autonomous Driving, Depth Completion, Depth Estimation, Monocular Depth Estimation, Multi-Task Learning, Scene Segmentation, Scene Understanding
Published	2019-03-26
URL	https://arxiv.org/abs/1903.10764v2
PDF	https://arxiv.org/pdf/1903.10764v2.pdf
PWC	https://paperswithcode.com/paper/veritatem-dies-aperit-temporally-consistent
Repo
Framework

Deep Reinforcement Learning for Optimal Critical Care Pain Management with Morphine using Dueling Double-Deep Q Networks


Title	Deep Reinforcement Learning for Optimal Critical Care Pain Management with Morphine using Dueling Double-Deep Q Networks
Authors	Daniel Lopez-Martinez, Patrick Eschenfeldt, Sassan Ostvar, Myles Ingram, Chin Hur, Rosalind Picard
Abstract	Opioids are the preferred medications for the treatment of pain in the intensive care unit. While undertreatment leads to unrelieved pain and poor clinical outcomes, excessive use of opioids puts patients at risk of experiencing multiple adverse effects. In this work, we present a sequential decision making framework for opioid dosing based on deep reinforcement learning. It provides real-time clinically interpretable dosing recommendations, personalized according to each patient’s evolving pain and physiological condition. We focus on morphine, one of the most commonly prescribed opioids. To train and evaluate the model, we used retrospective data from the publicly available MIMIC-3 database. Our results demonstrate that reinforcement learning may be used to aid decision making in the intensive care setting by providing personalized pain management interventions.
Tasks	Decision Making
Published	2019-04-25
URL	http://arxiv.org/abs/1904.11115v1
PDF	http://arxiv.org/pdf/1904.11115v1.pdf
PWC	https://paperswithcode.com/paper/deep-reinforcement-learning-for-optimal-1
Repo
Framework

Classical Music Generation in Distinct Dastgahs with AlimNet ACGAN


Title	Classical Music Generation in Distinct Dastgahs with AlimNet ACGAN
Authors	Saber Malekzadeh, Maryam Samami, Shahla RezazadehAzar, Maryam Rayegan
Abstract	In this paper AlimNet (With respect to great musician, Alim Qasimov) an auxiliary generative adversarial deep neural network (ACGAN) for generating music categorically, is used. This proposed network is a conditional ACGAN to condition the generation process on music tracks which has a hybrid architecture, composing of different kind of layers of neural networks. The employed music dataset is MICM which contains 1137 music samples (506 violins and 631 straw) with seven types of classical music Dastgah labels. To extract both temporal and spectral features, Short-Time Fourier Transform (STFT) is applied to convert input audio signals from time domain to time-frequency domain. GANs are composed of a generator for generating new samples and a discriminator to help generator making better samples. Samples in time-frequency domain are used to train discriminator in fourteen classes (seven Dastgahs and two instruments). The outputs of the conditional ACGAN are also artificial music samples in those mentioned scales in time-frequency domain. Then the output of the generator is transformed by Inverse STFT (ISTFT). Finally, randomly ten generated music samples (five violin and five straw samples) are given to ten musicians to rate how exact the samples are and the overall result was 76.5%.
Tasks	Music Generation
Published	2019-01-15
URL	http://arxiv.org/abs/1901.04696v1
PDF	http://arxiv.org/pdf/1901.04696v1.pdf
PWC	https://paperswithcode.com/paper/classical-music-generation-in-distinct
Repo
Framework

A Modern Introduction to Online Learning


Title	A Modern Introduction to Online Learning
Authors	Francesco Orabona
Abstract	In this monograph, I introduce the basic concepts of Online Learning through a modern view of Online Convex Optimization. Here, online learning refers to the framework of regret minimization under worst-case assumptions. I present first-order and second-order algorithms for online learning with convex losses, in Euclidean and non-Euclidean settings. All the algorithms are clearly presented as instantiation of Online Mirror Descent or Follow-The-Regularized-Leader and their variants. Particular attention is given to the issue of tuning the parameters of the algorithms and learning in unbounded domains, through adaptive and parameter-free online learning algorithms. Non-convex losses are dealt through convex surrogate losses and through randomization. The bandit setting is also briefly discussed, touching on the problem of adversarial and stochastic multi-armed bandits. These notes do not require prior knowledge of convex analysis and all the required mathematical tools are rigorously explained. Moreover, all the proofs have been carefully chosen to be as simple and as short as possible.
Tasks	Multi-Armed Bandits
Published	2019-12-31
URL	https://arxiv.org/abs/1912.13213v1
PDF	https://arxiv.org/pdf/1912.13213v1.pdf
PWC	https://paperswithcode.com/paper/a-modern-introduction-to-online-learning
Repo
Framework

Oriented Objects as pairs of Middle Lines


Title	Oriented Objects as pairs of Middle Lines
Authors	Haoran Wei, Lin Zhou, Yue Zhang, Hao Li, Rongxin Guo, Hongqi Wang
Abstract	The detection of oriented objects is frequently appeared in the field of natural scene text detection as well as object detection in aerial images. Traditional detectors for oriented objects are common to rotate anchors on the basis of the RCNN frameworks, which will multiple the number of anchors with a variety of angles, coupled with rotating NMS algorithm, the computational complexities of these models are greatly increased. In this paper, we propose a novel model named Oriented Objects Detection Network O^2-DNet to detect oriented objects by predicting a pair of middle lines inside each target. O^2-DNet is an one-stage, anchor-free and NMS-free model. The target line segments of our model are defined as two corresponding middle lines of original rotating bounding box annotations which can be transformed directly instead of additional manual tagging. Experiments show that our O^2-DNet achieves excellent performance on ICDAR 2015 and DOTA datasets. It is noteworthy that the objects in COCO can be regard as a special form of oriented objects with an angle of 90 degrees. O^2-DNet can still achieve competitive results in these general natural object detection datasets.
Tasks	Object Detection, Object Detection In Aerial Images, Scene Text Detection
Published	2019-12-23
URL	https://arxiv.org/abs/1912.10694v2
PDF	https://arxiv.org/pdf/1912.10694v2.pdf
PWC	https://paperswithcode.com/paper/oriented-objects-as-pairs-of-middle-lines
Repo
Framework

Joint Regularization on Activations and Weights for Efficient Neural Network Pruning


Title	Joint Regularization on Activations and Weights for Efficient Neural Network Pruning
Authors	Qing Yang, Wei Wen, Zuoguan Wang, Hai Li
Abstract	With the rapid scaling up of deep neural networks (DNNs), extensive research studies on network model compression such as weight pruning have been performed for improving deployment efficiency. This work aims to advance the compression beyond the weights to neuron activations. We propose the joint regularization technique which simultaneously regulates the distribution of weights and activations. By distinguishing and leveraging the significance difference among neuron responses and connections during learning, the jointly pruned network, namely \textit{JPnet}, optimizes the sparsity of activations and weights for improving execution efficiency. The derived deep sparsification of JPnet reveals more optimization space for the existing DNN accelerators dedicated for sparse matrix operations. We thoroughly evaluate the effectiveness of joint regularization through various network models with different activation functions and on different datasets. With $0.4%$ degradation constraint on inference accuracy, a JPnet can save $72.3% \sim 98.8%$ of computation cost compared to the original dense models, with up to $5.2\times$ and $12.3\times$ reductions in activation and weight numbers, respectively.
Tasks	Model Compression, Network Pruning
Published	2019-06-19
URL	https://arxiv.org/abs/1906.07875v2
PDF	https://arxiv.org/pdf/1906.07875v2.pdf
PWC	https://paperswithcode.com/paper/joint-pruning-on-activations-and-weights-for
Repo
Framework

Full-Scale Continuous Synthetic Sonar Data Generation with Markov Conditional Generative Adversarial Networks


Title	Full-Scale Continuous Synthetic Sonar Data Generation with Markov Conditional Generative Adversarial Networks
Authors	Marija Jegorova, Antti Ilari Karjalainen, Jose Vazquez, Timothy Hospedales
Abstract	Deployment and operation of autonomous underwater vehicles is expensive and time-consuming. High-quality realistic sonar data simulation could be of benefit to multiple applications, including training of human operators for post-mission analysis, as well as tuning and validation of autonomous target recognition (ATR) systems for underwater vehicles. Producing realistic synthetic sonar imagery is a challenging problem as the model has to account for specific artefacts of real acoustic sensors, vehicle altitude, and a variety of environmental factors. We propose a novel method for generating realistic-looking sonar side-scans of full-length missions, called Markov Conditional pix2pix (MC-pix2pix). Quantitative assessment results confirm that the quality of the produced data is almost indistinguishable from real. Furthermore, we show that bootstrapping ATR systems with MC-pix2pix data can improve the performance. Synthetic data is generated 18 times faster than real acquisition speed, with full user control over the topography of the generated data.
Tasks
Published	2019-10-15
URL	https://arxiv.org/abs/1910.06750v2
PDF	https://arxiv.org/pdf/1910.06750v2.pdf
PWC	https://paperswithcode.com/paper/full-scale-continuous-synthetic-sonar-data
Repo
Framework

ChaosNet: A Chaos based Artificial Neural Network Architecture for Classification


Title	ChaosNet: A Chaos based Artificial Neural Network Architecture for Classification
Authors	Harikrishnan Nellippallil Balakrishnan, Aditi Kathpalia, Snehanshu Saha, Nithin Nagaraj
Abstract	Inspired by chaotic firing of neurons in the brain, we propose ChaosNet – a novel chaos based artificial neural network architecture for classification tasks. ChaosNet is built using layers of neurons, each of which is a 1D chaotic map known as the Generalized Luroth Series (GLS) which has been shown in earlier works to possess very useful properties for compression, cryptography and for computing XOR and other logical operations. In this work, we design a novel learning algorithm on ChaosNet that exploits the topological transitivity property of the chaotic GLS neurons. The proposed learning algorithm gives consistently good performance accuracy in a number of classification tasks on well known publicly available datasets with very limited training samples. Even with as low as 7 (or fewer) training samples/class (which accounts for less than 0.05% of the total available data), ChaosNet yields performance accuracies in the range 73.89 % - 98.33 %. We demonstrate the robustness of ChaosNet to additive parameter noise and also provide an example implementation of a 2-layer ChaosNet for enhancing classification accuracy. We envisage the development of several other novel learning algorithms on ChaosNet in the near future.
Tasks
Published	2019-10-06
URL	https://arxiv.org/abs/1910.02423v1
PDF	https://arxiv.org/pdf/1910.02423v1.pdf
PWC	https://paperswithcode.com/paper/chaosnet-a-chaos-based-artificial-neural
Repo
Framework

Effectiveness of LSTMs in Predicting Congestive Heart Failure Onset


Title	Effectiveness of LSTMs in Predicting Congestive Heart Failure Onset
Authors	Sunil Mallya, Marc Overhage, Navneet Srivastava, Tatsuya Arai, Cole Erdman
Abstract	In this paper we present a Recurrent neural networks (RNN) based architecture that achieves an AUCROC of 0.9147 for predicting the onset of Congestive Heart Failure (CHF) 15 months in advance using a 12-month observation window on a large cohort of 216,394 patients. We believe this to be the largest study in CHF onset prediction with respect to the number of CHF case patients in the cohort and the test set (3,332 CHF patients) on which the AUC metrics are reported. We explore the extent to which LSTM (Long Short Term Memory) based model, a variant of RNNs, can accurately predict the onset of CHF when compared to known linear baselines like Logistic Regression, Random Forests and deep learning based models such as Multi-Layer Perceptron and Convolutional Neural Networks. We utilize demographics, medical diagnosis and procedure data from 21,405 CHF and 194,989 control patients to as our features. We describe our feature embedding strategy for medical diagnosis codes that accommodates the sparse, irregular, longitudinal, and high-dimensional characteristics of EHR data. We empirically show that LSTMs can capture the longitudinal aspects of EHR data better than the proposed baselines. As an attempt to interpret the model, we present a temporal data analysis-based technique on false positives to attribute feature importance. A model capable of predicting the onset of congestive heart failure months in the future with this level of accuracy and precision can support efforts of practitioners to implement risk factor reduction strategies and researchers to begin to systematically evaluate interventions to potentially delay or avert development of the disease with high mortality, morbidity and significant costs.
Tasks	Feature Importance, Medical Diagnosis
Published	2019-02-07
URL	http://arxiv.org/abs/1902.02443v2
PDF	http://arxiv.org/pdf/1902.02443v2.pdf
PWC	https://paperswithcode.com/paper/effectiveness-of-lstms-in-predicting
Repo
Framework