January 28, 2020

2891 words 14 mins read

Paper Group ANR 833

A Novel Demodulation and Estimation Algorithm for Blackout Communication: Extract Principal Components with Deep Learning. Unsupervised Text Summarization via Mixed Model Back-Translation. Robust Reinforcement Learning for Continuous Control with Model Misspecification. Stochastic Neural Network with Kronecker Flow. ASCNet: Adaptive-Scale Convoluti …

A Novel Demodulation and Estimation Algorithm for Blackout Communication: Extract Principal Components with Deep Learning


Title	A Novel Demodulation and Estimation Algorithm for Blackout Communication: Extract Principal Components with Deep Learning
Authors	Haoyan Liu, Yanming Liu, Ming Yang, Xiaoping Li
Abstract	For reentry or near space communication, owing to the influence of the time-varying plasma sheath channel environment, the received IQ baseband signals are severely rotated on the constellation. Researches have shown that the frequency of electron density varies from 20kHz to 100 kHz which is on the same order as the symbol rate of most TT&C communication systems and a mass of bandwidth will be consumed to track the time-varying channel with traditional estimation. In this paper, motivated by principal curve analysis, we propose a deep learning (DL) algorithm which called symmetric manifold network (SMN) to extract the curves on the constellation and classify the signals based on the curves. The key advantage is that SMN can achieve joint optimization of demodulation and channel estimation. From our simulation results, the new algorithm significantly reduces the symbol error rate (SER) compared to existing algorithms and enables accurate estimation of fading with extremely high bandwith utilization rate.
Tasks
Published	2019-05-27
URL	https://arxiv.org/abs/1905.11229v2
PDF	https://arxiv.org/pdf/1905.11229v2.pdf
PWC	https://paperswithcode.com/paper/a-novel-demodulation-and-estimation-algorithm
Repo
Framework

Unsupervised Text Summarization via Mixed Model Back-Translation


Title	Unsupervised Text Summarization via Mixed Model Back-Translation
Authors	Yacine Jernite
Abstract	Back-translation based approaches have recently lead to significant progress in unsupervised sequence-to-sequence tasks such as machine translation or style transfer. In this work, we extend the paradigm to the problem of learning a sentence summarization system from unaligned data. We present several initial models which rely on the asymmetrical nature of the task to perform the first back-translation step, and demonstrate the value of combining the data created by these diverse initialization methods. Our system outperforms the current state-of-the-art for unsupervised sentence summarization from fully unaligned data by over 2 ROUGE, and matches the performance of recent semi-supervised approaches.
Tasks	Machine Translation, Style Transfer, Text Summarization
Published	2019-08-22
URL	https://arxiv.org/abs/1908.08566v1
PDF	https://arxiv.org/pdf/1908.08566v1.pdf
PWC	https://paperswithcode.com/paper/unsupervised-text-summarization-via-mixed
Repo
Framework

Robust Reinforcement Learning for Continuous Control with Model Misspecification


Title	Robust Reinforcement Learning for Continuous Control with Model Misspecification
Authors	Daniel J. Mankowitz, Nir Levine, Rae Jeong, Yuanyuan Shi, Jackie Kay, Abbas Abdolmaleki, Jost Tobias Springenberg, Timothy Mann, Todd Hester, Martin Riedmiller
Abstract	We provide a framework for incorporating robustness – to perturbations in the transition dynamics which we refer to as model misspecification – into continuous control Reinforcement Learning (RL) algorithms. We specifically focus on incorporating robustness into a state-of-the-art continuous control RL algorithm called Maximum a-posteriori Policy Optimization (MPO). We achieve this by learning a policy that optimizes for a worst case expected return objective and derive a corresponding robust entropy-regularized Bellman contraction operator. In addition, we introduce a less conservative, soft-robust, entropy-regularized objective with a corresponding Bellman operator. We show that both, robust and soft-robust policies, outperform their non-robust counterparts in nine Mujoco domains with environment perturbations. In addition, we show improved robust performance on a high-dimensional, simulated, dexterous robotic hand. Finally, we present multiple investigative experiments that provide a deeper insight into the robustness framework. This includes an adaptation to another continuous control RL algorithm as well as learning the uncertainty set from offline data. Performance videos can be found online at https://sites.google.com/view/robust-rl.
Tasks	Continuous Control
Published	2019-06-18
URL	https://arxiv.org/abs/1906.07516v2
PDF	https://arxiv.org/pdf/1906.07516v2.pdf
PWC	https://paperswithcode.com/paper/robust-reinforcement-learning-for-continuous
Repo
Framework

Stochastic Neural Network with Kronecker Flow


Title	Stochastic Neural Network with Kronecker Flow
Authors	Chin-Wei Huang, Ahmed Touati, Pascal Vincent, Gintare Karolina Dziugaite, Alexandre Lacoste, Aaron Courville
Abstract	Recent advances in variational inference enable the modelling of highly structured joint distributions, but are limited in their capacity to scale to the high-dimensional setting of stochastic neural networks. This limitation motivates a need for scalable parameterizations of the noise generation process, in a manner that adequately captures the dependencies among the various parameters. In this work, we address this need and present the Kronecker Flow, a generalization of the Kronecker product to invertible mappings designed for stochastic neural networks. We apply our method to variational Bayesian neural networks on predictive tasks, PAC-Bayes generalization bound estimation, and approximate Thompson sampling in contextual bandits. In all setups, our methods prove to be competitive with existing methods and better than the baselines.
Tasks	Multi-Armed Bandits
Published	2019-06-10
URL	https://arxiv.org/abs/1906.04282v2
PDF	https://arxiv.org/pdf/1906.04282v2.pdf
PWC	https://paperswithcode.com/paper/stochastic-neural-network-with-kronecker-flow
Repo
Framework

ASCNet: Adaptive-Scale Convolutional Neural Networks for Multi-Scale Feature Learning


Title	ASCNet: Adaptive-Scale Convolutional Neural Networks for Multi-Scale Feature Learning
Authors	Mo Zhang, Jie Zhao, Xiang Li, Li Zhang, Quanzheng Li
Abstract	Extracting multi-scale information is key to semantic segmentation. However, the classic convolutional neural networks (CNNs) encounter difficulties in achieving multi-scale information extraction: expanding convolutional kernel incurs the high computational cost and using maximum pooling sacrifices image information. The recently developed dilated convolution solves these problems, but with the limitation that the dilation rates are fixed and therefore the receptive field cannot fit for all objects with different sizes in the image. We propose an adaptivescale convolutional neural network (ASCNet), which introduces a 3-layer convolution structure in the end-to-end training, to adaptively learn an appropriate dilation rate for each pixel in the image. Such pixel-level dilation rates produce optimal receptive fields so that the information of objects with different sizes can be extracted at the corresponding scale. We compare the segmentation results using the classic CNN, the dilated CNN and the proposed ASCNet on two types of medical images (The Herlev dataset and SCD RBC dataset). The experimental results show that ASCNet achieves the highest accuracy. Moreover, the automatically generated dilation rates are positively correlated to the sizes of the objects, confirming the effectiveness of the proposed method.
Tasks	Semantic Segmentation
Published	2019-07-07
URL	https://arxiv.org/abs/1907.03241v1
PDF	https://arxiv.org/pdf/1907.03241v1.pdf
PWC	https://paperswithcode.com/paper/ascnet-adaptive-scale-convolutional-neural
Repo
Framework

P2L: Predicting Transfer Learning for Images and Semantic Relations


Title	P2L: Predicting Transfer Learning for Images and Semantic Relations
Authors	Bishwaranjan Bhattacharjee, Noel Codella, John R. Kender, Siyu Huo, Patrick Watson, Michael R. Glass, Parijat Dube, Matthew Hill, Brian Belgodere
Abstract	Transfer learning enhances learning across tasks, by leveraging previously learned representations – if they are properly chosen. We describe an efficient method to accurately estimate the appropriateness of a previously trained model for use in a new learning task. We use this measure, which we call “Predict To Learn” (“P2L”), in the two very different domains of images and semantic relations, where it predicts, from a set of “source” models, the one model most likely to produce effective transfer for training a given “target” model. We validate our approach thoroughly, by assembling a collection of candidate source models, then fine-tuning each candidate to perform each of a collection of target tasks, and finally measuring how well transfer has been enhanced. Across 95 tasks within multiple domains (images classification and semantic relations), the P2L approach was able to select the best transfer learning model on average, while the heuristic of choosing model trained with the largest data set selected the best model in only 55 cases. These results suggest that P2L captures important information in common between source and target tasks, and that this shared informational structure contributes to successful transfer learning more than simple data size.
Tasks	Transfer Learning
Published	2019-08-20
URL	https://arxiv.org/abs/1908.07630v1
PDF	https://arxiv.org/pdf/1908.07630v1.pdf
PWC	https://paperswithcode.com/paper/190807630
Repo
Framework

Neural Network Predictive Controller for Grid-Connected Virtual Synchronous Generator


Title	Neural Network Predictive Controller for Grid-Connected Virtual Synchronous Generator
Authors	Sepehr Saadatmand, Mohammad Saleh Sanjarinia, Pourya Shamsi, Mehdi Ferdowsi, Donald C. Wunsch
Abstract	In this paper, a neural network predictive controller is proposed to regulate the active and the reactive power delivered to the grid generated by a three-phase virtual inertia-based inverter. The concept of the conventional virtual synchronous generator (VSG) is discussed, and it is shown that when the inverter is connected to non-inductive grids, the conventional PI-based VSGs are unable to perform acceptable tracking. The concept of the neural network predictive controller is also discussed to replace the traditional VSGs. This replacement enables inverters to perform in both inductive and non-inductive grids. The simulation results confirm that a well-trained neural network predictive controller illustrates can adapt to any grid impedance angle, compared to the traditional PI-based virtual inertia controllers.
Tasks
Published	2019-08-14
URL	https://arxiv.org/abs/1908.05199v1
PDF	https://arxiv.org/pdf/1908.05199v1.pdf
PWC	https://paperswithcode.com/paper/neural-network-predictive-controller-for-grid
Repo
Framework

A Longitudinal Framework for Predicting Nonresponse in Panel Surveys


Title	A Longitudinal Framework for Predicting Nonresponse in Panel Surveys
Authors	Christoph Kern, Bernd Weiss, Jan-Philipp Kolb
Abstract	Nonresponse in panel studies can lead to a substantial loss in data quality due to its potential to introduce bias and distort survey estimates. Recent work investigates the usage of machine learning to predict nonresponse in advance, such that predicted nonresponse propensities can be used to inform the data collection process. However, predicting nonresponse in panel studies requires accounting for the longitudinal data structure in terms of model building, tuning, and evaluation. This study proposes a longitudinal framework for predicting nonresponse with machine learning and multiple panel waves and illustrates its application. With respect to model building, this approach utilizes information from multiple waves by introducing features that aggregate previous (non)response patterns. Concerning model tuning and evaluation, temporal cross-validation is employed by iterating through pairs of panel waves such that the training and test sets move in time. Implementing this approach with data from a German probability-based mixed-mode panel shows that aggregating information over multiple panel waves can be used to build prediction models with competitive and robust performance over all test waves.
Tasks
Published	2019-09-29
URL	https://arxiv.org/abs/1909.13361v2
PDF	https://arxiv.org/pdf/1909.13361v2.pdf
PWC	https://paperswithcode.com/paper/a-longitudinal-framework-for-predicting
Repo
Framework

Asynchronous Federated Optimization


Title	Asynchronous Federated Optimization
Authors	Cong Xie, Sanmi Koyejo, Indranil Gupta
Abstract	Federated learning enables training on a massive number of edge devices. To improve flexibility and scalability, we propose a new asynchronous federated optimization algorithm. We prove that the proposed approach has near-linear convergence to a global optimum, for both strongly and non-strongly convex problems, as well as a restricted family of non-convex problems. Empirical results show that the proposed algorithm converges fast and tolerates staleness.
Tasks
Published	2019-03-10
URL	https://arxiv.org/abs/1903.03934v4
PDF	https://arxiv.org/pdf/1903.03934v4.pdf
PWC	https://paperswithcode.com/paper/asynchronous-federated-optimization
Repo
Framework

Macro F1 and Macro F1


Title	Macro F1 and Macro F1
Authors	Juri Opitz, Sebastian Burst
Abstract	The ‘macro F1’ metric is frequently used to evaluate binary, multi-class and multi-label classification problems. Yet, we find that there exist two different formulas to calculate this quantity. In this note, we show that only under rare circumstances, the two computations can be considered equivalent. More specifically, one formula well ‘rewards’ classifiers which produce a skewed error type distribution. In fact, the difference in outcome of the two computations can be as high as 0.5. Finally, we show that the two computations may not only diverge in their scalar result but also lead to different classifier rankings.
Tasks	Multi-Label Classification
Published	2019-11-08
URL	https://arxiv.org/abs/1911.03347v2
PDF	https://arxiv.org/pdf/1911.03347v2.pdf
PWC	https://paperswithcode.com/paper/macro-f1-and-macro-f1
Repo
Framework

Fine-grained Recognition: Accounting for Subtle Differences between Similar Classes


Title	Fine-grained Recognition: Accounting for Subtle Differences between Similar Classes
Authors	Guolei Sun, Hisham Cholakkal, Salman Khan, Fahad Shahbaz Khan, Ling Shao
Abstract	The main requisite for fine-grained recognition task is to focus on subtle discriminative details that make the subordinate classes different from each other. We note that existing methods implicitly address this requirement and leave it to a data-driven pipeline to figure out what makes a subordinate class different from the others. This results in two major limitations: First, the network focuses on the most obvious distinctions between classes and overlooks more subtle inter-class variations. Second, the chance of misclassifying a given sample in any of the negative classes is considered equal, while in fact, confusions generally occur among only the most similar classes. Here, we propose to explicitly force the network to find the subtle differences among closely related classes. In this pursuit, we introduce two key novelties that can be easily plugged into existing end-to-end deep learning pipelines. On one hand, we introduce diversification block which masks the most salient features for an input to force the network to use more subtle cues for its correct classification. Concurrently, we introduce a gradient-boosting loss function that focuses only on the confusing classes for each sample and therefore moves swiftly along the direction on the loss surface that seeks to resolve these ambiguities. The synergy between these two blocks helps the network to learn more effective feature representations. Comprehensive experiments are performed on five challenging datasets. Our approach outperforms existing methods using similar experimental setting on all five datasets.
Tasks	Fine-Grained Image Classification
Published	2019-12-14
URL	https://arxiv.org/abs/1912.06842v1
PDF	https://arxiv.org/pdf/1912.06842v1.pdf
PWC	https://paperswithcode.com/paper/fine-grained-recognition-accounting-for
Repo
Framework

Rewarding Smatch: Transition-Based AMR Parsing with Reinforcement Learning


Title	Rewarding Smatch: Transition-Based AMR Parsing with Reinforcement Learning
Authors	Tahira Naseem, Abhishek Shah, Hui Wan, Radu Florian, Salim Roukos, Miguel Ballesteros
Abstract	Our work involves enriching the Stack-LSTM transition-based AMR parser (Ballesteros and Al-Onaizan, 2017) by augmenting training with Policy Learning and rewarding the Smatch score of sampled graphs. In addition, we also combined several AMR-to-text alignments with an attention mechanism and we supplemented the parser with pre-processed concept identification, named entities and contextualized embeddings. We achieve a highly competitive performance that is comparable to the best published results. We show an in-depth study ablating each of the new components of the parser
Tasks	Amr Parsing
Published	2019-05-31
URL	https://arxiv.org/abs/1905.13370v1
PDF	https://arxiv.org/pdf/1905.13370v1.pdf
PWC	https://paperswithcode.com/paper/rewarding-smatch-transition-based-amr-parsing
Repo
Framework

Automatic Accuracy Prediction for AMR Parsing


Title	Automatic Accuracy Prediction for AMR Parsing
Authors	Juri Opitz, Anette Frank
Abstract	Abstract Meaning Representation (AMR) represents sentences as directed, acyclic and rooted graphs, aiming at capturing their meaning in a machine readable format. AMR parsing converts natural language sentences into such graphs. However, evaluating a parser on new data by means of comparison to manually created AMR graphs is very costly. Also, we would like to be able to detect parses of questionable quality, or preferring results of alternative systems by selecting the ones for which we can assess good quality. We propose AMR accuracy prediction as the task of predicting several metrics of correctness for an automatically generated AMR parse - in absence of the corresponding gold parse. We develop a neural end-to-end multi-output regression model and perform three case studies: firstly, we evaluate the model’s capacity of predicting AMR parse accuracies and test whether it can reliably assign high scores to gold parses. Secondly, we perform parse selection based on predicted parse accuracies of candidate parses from alternative systems, with the aim of improving overall results. Finally, we predict system ranks for submissions from two AMR shared tasks on the basis of their predicted parse accuracy averages. All experiments are carried out across two different domains and show that our method is effective.
Tasks	Amr Parsing
Published	2019-04-17
URL	http://arxiv.org/abs/1904.08301v1
PDF	http://arxiv.org/pdf/1904.08301v1.pdf
PWC	https://paperswithcode.com/paper/automatic-accuracy-prediction-for-amr-parsing
Repo
Framework

Limited View and Sparse Photoacoustic Tomography for Neuroimaging with Deep Learning


Title	Limited View and Sparse Photoacoustic Tomography for Neuroimaging with Deep Learning
Authors	Steven Guan, Amir A. Khan, Siddhartha Sikdar, Parag V. Chitnis
Abstract	Photoacoustic tomography (PAT) is a nonionizing imaging modality capable of acquiring high contrast and resolution images of optical absorption at depths greater than traditional optical imaging techniques. Practical considerations with instrumentation and geometry limit the number of available acoustic sensors and their view of the imaging target, which result in significant image reconstruction artifacts degrading image quality. Iterative reconstruction methods can be used to reduce artifacts but are computationally expensive. In this work, we propose a novel deep learning approach termed pixelwise deep learning (PixelDL) that first employs pixelwise interpolation governed by the physics of photoacoustic wave propagation and then uses a convolution neural network to directly reconstruct an image. Simulated photoacoustic data from synthetic vasculature phantom and mouse-brain vasculature were used for training and testing, respectively. Results demonstrated that PixelDL achieved comparable performance to iterative methods and outperformed other CNN-based approaches for correcting artifacts. PixelDL is a computationally efficient approach that enables for realtime PAT rendering and for improved image quality, quantification, and interpretation.
Tasks	Image Reconstruction
Published	2019-11-11
URL	https://arxiv.org/abs/1911.04357v1
PDF	https://arxiv.org/pdf/1911.04357v1.pdf
PWC	https://paperswithcode.com/paper/limited-view-and-sparse-photoacoustic
Repo
Framework

P2SGrad: Refined Gradients for Optimizing Deep Face Models


Title	P2SGrad: Refined Gradients for Optimizing Deep Face Models
Authors	Xiao Zhang, Rui Zhao, Junjie Yan, Mengya Gao, Yu Qiao, Xiaogang Wang, Hongsheng Li
Abstract	Cosine-based softmax losses significantly improve the performance of deep face recognition networks. However, these losses always include sensitive hyper-parameters which can make training process unstable, and it is very tricky to set suitable hyper parameters for a specific dataset. This paper addresses this challenge by directly designing the gradients for adaptively training deep neural networks. We first investigate and unify previous cosine softmax losses by analyzing their gradients. This unified view inspires us to propose a novel gradient called P2SGrad (Probability-to-Similarity Gradient), which leverages a cosine similarity instead of classification probability to directly update the testing metrics for updating neural network parameters. P2SGrad is adaptive and hyper-parameter free, which makes the training process more efficient and faster. We evaluate our P2SGrad on three face recognition benchmarks, LFW, MegaFace, and IJB-C. The results show that P2SGrad is stable in training, robust to noise, and achieves state-of-the-art performance on all the three benchmarks.
Tasks	Face Recognition
Published	2019-05-07
URL	https://arxiv.org/abs/1905.02479v1
PDF	https://arxiv.org/pdf/1905.02479v1.pdf
PWC	https://paperswithcode.com/paper/p2sgrad-refined-gradients-for-optimizing-deep
Repo
Framework