January 31, 2020

2800 words 14 mins read

Paper Group ANR 139

Designing a Symbolic Intermediate Representation for Neural Surface Realization. Adversarial Pulmonary Pathology Translation for Pairwise Chest X-ray Data Augmentation. Efficient Continual Learning in Neural Networks with Embedding Regularization. CoachAI: A Project for Microscopic Badminton Match Data Collection and Tactical Analysis. Design and U …

Designing a Symbolic Intermediate Representation for Neural Surface Realization


Title	Designing a Symbolic Intermediate Representation for Neural Surface Realization
Authors	Henry Elder, Jennifer Foster, James Barry, Alexander O’Connor
Abstract	Generated output from neural NLG systems often contain errors such as hallucination, repetition or contradiction. This work focuses on designing a symbolic intermediate representation to be used in multi-stage neural generation with the intention of reducing the frequency of failed outputs. We show that surface realization from this intermediate representation is of high quality and when the full system is applied to the E2E dataset it outperforms the winner of the E2E challenge. Furthermore, by breaking out the surface realization step from typically end-to-end neural systems, we also provide a framework for non-neural content selection and planning systems to potentially take advantage of semi-supervised pretraining of neural surface realization models.
Tasks
Published	2019-05-24
URL	https://arxiv.org/abs/1905.10486v1
PDF	https://arxiv.org/pdf/1905.10486v1.pdf
PWC	https://paperswithcode.com/paper/designing-a-symbolic-intermediate
Repo
Framework

Adversarial Pulmonary Pathology Translation for Pairwise Chest X-ray Data Augmentation


Title	Adversarial Pulmonary Pathology Translation for Pairwise Chest X-ray Data Augmentation
Authors	Yunyan Xing, Zongyuan Ge, Rui Zeng, Dwarikanath Mahapatra, Jarrel Seah, Meng Law, Tom Drummond
Abstract	Recent works show that Generative Adversarial Networks (GANs) can be successfully applied to chest X-ray data augmentation for lung disease recognition. However, the implausible and distorted pathology features generated from the less than perfect generator may lead to wrong clinical decisions. Why not keep the original pathology region? We proposed a novel approach that allows our generative model to generate high quality plausible images that contain undistorted pathology areas. The main idea is to design a training scheme based on an image-to-image translation network to introduce variations of new lung features around the pathology ground-truth area. Moreover, our model is able to leverage both annotated disease images and unannotated healthy lung images for the purpose of generation. We demonstrate the effectiveness of our model on two tasks: (i) we invite certified radiologists to assess the quality of the generated synthetic images against real and other state-of-the-art generative models, and (ii) data augmentation to improve the performance of disease localisation.
Tasks	Data Augmentation, Image-to-Image Translation
Published	2019-10-11
URL	https://arxiv.org/abs/1910.04961v2
PDF	https://arxiv.org/pdf/1910.04961v2.pdf
PWC	https://paperswithcode.com/paper/adversarial-pulmonary-pathology-translation
Repo
Framework

Efficient Continual Learning in Neural Networks with Embedding Regularization


Title	Efficient Continual Learning in Neural Networks with Embedding Regularization
Authors	Jary Pomponi, Simone Scardapane, Vincenzo Lomonaco, Aurelio Uncini
Abstract	Continual learning of deep neural networks is a key requirement for scaling them up to more complex applicative scenarios and for achieving real lifelong learning of these architectures. Previous approaches to the problem have considered either the progressive increase in the size of the networks, or have tried to regularize the network behavior to equalize it with respect to previously observed tasks. In the latter case, it is essential to understand what type of information best represents this past behavior. Common techniques include regularizing the past outputs, gradients, or individual weights. In this work, we propose a new, relatively simple and efficient method to perform continual learning by regularizing instead the network internal embeddings. To make the approach scalable, we also propose a dynamic sampling strategy to reduce the memory footprint of the required external storage. We show that our method performs favorably with respect to state-of-the-art approaches in the literature, while requiring significantly less space in memory and computational time. In addition, inspired inspired by to recent works, we evaluate the impact of selecting a more flexible model for the activation functions inside the network, evaluating the impact of catastrophic forgetting on the activation functions themselves.
Tasks	Continual Learning
Published	2019-09-09
URL	https://arxiv.org/abs/1909.03742v2
PDF	https://arxiv.org/pdf/1909.03742v2.pdf
PWC	https://paperswithcode.com/paper/efficient-continual-learning-in-neural
Repo
Framework

CoachAI: A Project for Microscopic Badminton Match Data Collection and Tactical Analysis


Title	CoachAI: A Project for Microscopic Badminton Match Data Collection and Tactical Analysis
Authors	Tzu-Han Hsu, Ching-Hsuan Chen, Nyan Ping Ju, Tsì-Uí İk, Wen-Chih Peng, Chih-Chuan Wang, Yu-Shuen Wang, Yuan-Hsiang Lin, Yu-Chee Tseng, Jiun-Long Huang, Yu-Tai Ching
Abstract	Computer vision based object tracking has been used to annotate and augment sports video. For sports learning and training, video replay is often used in post-match review and training review for tactical analysis and movement analysis. For automatically and systematically competition data collection and tactical analysis, a project called CoachAI has been supported by the Ministry of Science and Technology, Taiwan. The proposed project also includes research of data visualization, connected training auxiliary devices, and data warehouse. Deep learning techniques will be used to develop video-based real-time microscopic competition data collection based on broadcast competition video. Machine learning techniques will be used to develop a tactical analysis. To reveal data in more understandable forms and to help in pre-match training, AR/VR techniques will be used to visualize data, tactics, and so on. In addition, training auxiliary devices including smart badminton rackets and connected serving machines will be developed based on the IoT technology to further utilize competition data and tactical data and boost training efficiency. Especially, the connected serving machines will be developed to perform specified tactics and to interact with players in their training.
Tasks	Object Tracking
Published	2019-07-12
URL	https://arxiv.org/abs/1907.12888v1
PDF	https://arxiv.org/pdf/1907.12888v1.pdf
PWC	https://paperswithcode.com/paper/coachai-a-project-for-microscopic-badminton
Repo
Framework

Design and Use of Loop-Transformation Pragmas


Title	Design and Use of Loop-Transformation Pragmas
Authors	Michael Kruse, Hal Finkel
Abstract	Adding a pragma directive into the source code is arguably easier than rewriting it, for instance for loop unrolling. Moreover, if the application is maintained for multiple platforms, their difference in performance characteristics may require different code transformations. Code transformation directives allow replacing the directives depending on the platform, i.e. separation of code semantics and its performance optimization. In this paper, we explore the design space (syntax and semantics) of adding such directive into a future OpenMP specification. Using a prototype implementation in Clang, we demonstrate the usefulness of such directives on a few benchmarks.
Tasks
Published	2019-10-06
URL	https://arxiv.org/abs/1910.02375v1
PDF	https://arxiv.org/pdf/1910.02375v1.pdf
PWC	https://paperswithcode.com/paper/design-and-use-of-loop-transformation-pragmas
Repo
Framework

Communication without Interception: Defense against Deep-Learning-based Modulation Detection


Title	Communication without Interception: Defense against Deep-Learning-based Modulation Detection
Authors	Muhammad Zaid Hameed, Andras Gyorgy, Deniz Gunduz
Abstract	We consider a communication scenario, in which an intruder, employing a deep neural network (DNN), tries to determine the modulation scheme of the intercepted signal. Our aim is to minimize the accuracy of the intruder, while guaranteeing that the intended receiver can still recover the underlying message with the highest reliability. This is achieved by constellation perturbation at the encoder, similarly to adversarial attacks against DNN-based classifiers. In the latter perturbation is limited to be imperceptible to a human observer, while in our case perturbation is constrained so that the message can still be reliably decoded by the legitimate receiver which is oblivious to the perturbation. Simulation results demonstrate the viability of our approach to make wireless communication secure against DNN-based intruders with minimal sacrifice in the communication performance.
Tasks
Published	2019-02-27
URL	http://arxiv.org/abs/1902.10674v1
PDF	http://arxiv.org/pdf/1902.10674v1.pdf
PWC	https://paperswithcode.com/paper/communication-without-interception-defense
Repo
Framework

What Kind of Language Is Hard to Language-Model?


Title	What Kind of Language Is Hard to Language-Model?
Authors	Sabrina J. Mielke, Ryan Cotterell, Kyle Gorman, Brian Roark, Jason Eisner
Abstract	How language-agnostic are current state-of-the-art NLP tools? Are there some types of language that are easier to model with current methods? In prior work (Cotterell et al., 2018) we attempted to address this question for language modeling, and observed that recurrent neural network language models do not perform equally well over all the high-resource European languages found in the Europarl corpus. We speculated that inflectional morphology may be the primary culprit for the discrepancy. In this paper, we extend these earlier experiments to cover 69 languages from 13 language families using a multilingual Bible corpus. Methodologically, we introduce a new paired-sample multiplicative mixed-effects model to obtain language difficulty coefficients from at-least-pairwise parallel corpora. In other words, the model is aware of inter-sentence variation and can handle missing data. Exploiting this model, we show that “translationese” is not any easier to model than natively written language in a fair comparison. Trying to answer the question of what features difficult languages have in common, we try and fail to reproduce our earlier (Cotterell et al., 2018) observation about morphological complexity and instead reveal far simpler statistics of the data that seem to drive complexity in a much larger sample.
Tasks	Language Modelling
Published	2019-06-11
URL	https://arxiv.org/abs/1906.04726v2
PDF	https://arxiv.org/pdf/1906.04726v2.pdf
PWC	https://paperswithcode.com/paper/what-kind-of-language-is-hard-to-language
Repo
Framework

Towards Deep Federated Defenses Against Malware in Cloud Ecosystems


Title	Towards Deep Federated Defenses Against Malware in Cloud Ecosystems
Authors	Josh Payne, Ashish Kundu
Abstract	In cloud computing environments with many virtual machines, containers, and other systems, an epidemic of malware can be highly threatening to business processes. In this vision paper, we introduce a hierarchical approach to performing malware detection and analysis using several recent advances in machine learning on graphs, hypergraphs, and natural language. We analyze individual systems and their logs, inspecting and understanding their behavior with attentional sequence models. Given a feature representation of each system’s logs using this procedure, we construct an attributed network of the cloud with systems and other components as vertices and propose an analysis of malware with inductive graph and hypergraph learning models. With this foundation, we consider the multicloud case, in which multiple clouds with differing privacy requirements cooperate against the spread of malware, proposing the use of federated learning to perform inference and training while preserving privacy. Finally, we discuss several open problems that remain in defending cloud computing environments against malware related to designing robust ecosystems, identifying cloud-specific optimization problems for response strategy, action spaces for malware containment and eradication, and developing priors and transfer learning tasks for machine learning models in this area.
Tasks	Malware Detection, Transfer Learning
Published	2019-12-27
URL	https://arxiv.org/abs/1912.12370v1
PDF	https://arxiv.org/pdf/1912.12370v1.pdf
PWC	https://paperswithcode.com/paper/towards-deep-federated-defenses-against
Repo
Framework

Selective Synthetic Augmentation with Quality Assurance


Title	Selective Synthetic Augmentation with Quality Assurance
Authors	Yuan Xue, Jiarong Ye, Rodney Long, Sameer Antani, Zhiyun Xue, Xiaolei Huang
Abstract	Supervised training of an automated medical image analysis system often requires a large amount of expert annotations that are hard to collect. Moreover, the proportions of data available across different classes may be highly imbalanced for rare diseases. To mitigate these issues, we investigate a novel data augmentation pipeline that selectively adds new synthetic images generated by conditional Adversarial Networks (cGANs), rather than extending directly the training set with synthetic images. The selection mechanisms that we introduce to the synthetic augmentation pipeline are motivated by the observation that, although cGAN-generated images can be visually appealing, they are not guaranteed to contain essential features for classification performance improvement. By selecting synthetic images based on the confidence of their assigned labels and their feature similarity to real labeled images, our framework provides quality assurance to synthetic augmentation by ensuring that adding the selected synthetic images to the training set will improve performance. We evaluate our model on a medical histopathology dataset, and two natural image classification benchmarks, CIFAR10 and SVHN. Results on these datasets show significant and consistent improvements in classification performance (with 6.8%, 3.9%, 1.6% higher accuracy, respectively) by leveraging cGAN generated images with selective augmentation.
Tasks	Data Augmentation, Image Classification
Published	2019-12-09
URL	https://arxiv.org/abs/1912.03837v1
PDF	https://arxiv.org/pdf/1912.03837v1.pdf
PWC	https://paperswithcode.com/paper/selective-synthetic-augmentation-with-quality
Repo
Framework

Self-reinforcing Unsupervised Matching


Title	Self-reinforcing Unsupervised Matching
Authors	Jiang Lu, Lei Li, Changshui Zhang
Abstract	Remarkable gains in deep learning usually rely on tremendous supervised data. Ensuring the modality diversity for one object in training set is critical for the generalization of cutting-edge deep models, but it burdens human with heavy manual labor on data collection and annotation. In addition, some rare or unexpected modalities are new for the current model, causing reduced performance under such emerging modalities. Inspired by the achievements in speech recognition, psychology and behavioristics, we present a practical solution, self-reinforcing unsupervised matching (SUM), to annotate the images with 2D structure-preserving property in an emerging modality by cross-modality matching. This approach requires no any supervision in emerging modality and only one template in seen modality, providing a possible route towards continual learning.
Tasks	Continual Learning, Speech Recognition
Published	2019-08-23
URL	https://arxiv.org/abs/1909.04138v1
PDF	https://arxiv.org/pdf/1909.04138v1.pdf
PWC	https://paperswithcode.com/paper/self-reinforcing-unsupervised-matching
Repo
Framework

Greedy Strategy Works for $k$-Center Clustering with Outliers and Coreset Construction


Title	Greedy Strategy Works for $k$-Center Clustering with Outliers and Coreset Construction
Authors	Hu Ding, Haikuo Yu, Zixiu Wang
Abstract	We study the problem of $k$-center clustering with outliers in arbitrary metrics and Euclidean space. Though a number of methods have been developed in the past decades, it is still quite challenging to design quality guaranteed algorithm with low complexity for this problem. Our idea is inspired by the greedy method, Gonzalez’s algorithm, for solving the problem of ordinary $k$-center clustering. Based on some novel observations, we show that this greedy strategy actually can handle $k$-center clustering with outliers efficiently, in terms of clustering quality and time complexity. We further show that the greedy approach yields small coreset for the problem in doubling metrics, so as to reduce the time complexity significantly. Our algorithms are easy to implement in practice. We test our method on both synthetic and real datasets. The experimental results suggest that our algorithms can achieve near optimal solutions and yield lower running times comparing with existing methods.
Tasks
Published	2019-01-24
URL	http://arxiv.org/abs/1901.08219v2
PDF	http://arxiv.org/pdf/1901.08219v2.pdf
PWC	https://paperswithcode.com/paper/greedy-strategy-works-for-clustering-with
Repo
Framework

Sensor Aware Lidar Odometry


Title	Sensor Aware Lidar Odometry
Authors	Dmitri Kovalenko, Mikhail Korobkin, Andrey Minin
Abstract	A lidar odometry method, integrating into the computation the knowledge about the physics of the sensor, is proposed. A model of measurement error enables higher precision in estimation of the point normal covariance. Adjacent laser beams are used in an outlier correspondence rejection scheme. The method is ranked in the KITTI’s leaderboard with 1.37% positioning error. 3.67% is achieved in comparison with the LOAM method on the internal dataset.
Tasks
Published	2019-07-22
URL	https://arxiv.org/abs/1907.09167v3
PDF	https://arxiv.org/pdf/1907.09167v3.pdf
PWC	https://paperswithcode.com/paper/sensor-aware-lidar-odometry
Repo
Framework

Cued@wmt19:ewc&lms


Title	Cued@wmt19:ewc&lms
Authors	Felix Stahlberg, Danielle Saunders, Adria de Gispert, Bill Byrne
Abstract	Two techniques provide the fabric of the Cambridge University Engineering Department’s (CUED) entry to the WMT19 evaluation campaign: elastic weight consolidation (EWC) and different forms of language modelling (LMs). We report substantial gains by fine-tuning very strong baselines on former WMT test sets using a combination of checkpoint averaging and EWC. A sentence-level Transformer LM and a document-level LM based on a modified Transformer architecture yield further gains. As in previous years, we also extract $n$-gram probabilities from SMT lattices which can be seen as a source-conditioned $n$-gram LM.
Tasks	Language Modelling
Published	2019-06-11
URL	https://arxiv.org/abs/1906.05447v1
PDF	https://arxiv.org/pdf/1906.05447v1.pdf
PWC	https://paperswithcode.com/paper/cuedwmt19ewclms
Repo
Framework

Continual Learning by Asymmetric Loss Approximation with Single-Side Overestimation


Title	Continual Learning by Asymmetric Loss Approximation with Single-Side Overestimation
Authors	Dongmin Park, Seokil Hong, Bohyung Han, Kyoung Mu Lee
Abstract	Catastrophic forgetting is a critical challenge in training deep neural networks. Although continual learning has been investigated as a countermeasure to the problem, it often suffers from the requirements of additional network components and the limited scalability to a large number of tasks. We propose a novel approach to continual learning by approximating a true loss function using an asymmetric quadratic function with one of its sides overestimated. Our algorithm is motivated by the empirical observation that the network parameter updates affect the target loss functions asymmetrically. In the proposed continual learning framework, we estimate an asymmetric loss function for the tasks considered in the past through a proper overestimation of its unobserved sides in training new tasks, while deriving the accurate model parameter for the observable sides. In contrast to existing approaches, our method is free from the side effects and achieves the state-of-the-art accuracy that is even close to the upper-bound performance on several challenging benchmark datasets.
Tasks	Continual Learning
Published	2019-08-08
URL	https://arxiv.org/abs/1908.02984v2
PDF	https://arxiv.org/pdf/1908.02984v2.pdf
PWC	https://paperswithcode.com/paper/continual-learning-by-asymmetric-loss
Repo
Framework

Channel Locality Block: A Variant of Squeeze-and-Excitation


Title	Channel Locality Block: A Variant of Squeeze-and-Excitation
Authors	Huayu Li
Abstract	Attention mechanism is a hot spot in deep learning field. Using channel attention model is an effective method for improving the performance of the convolutional neural network. Squeeze-and-Excitation block takes advantage of the channel dependence, selectively emphasizing the important channels and compressing the relatively useless channel. In this paper, we proposed a variant of SE block based on channel locality. Instead of using full connection layers to explore the global channel dependence, we adopt convolutional layers to learn the correlation between the nearby channels. We term this new algorithm Channel Locality(C-Local) block. We evaluate SE block and C-Local block by applying them to different CNNs architectures on cifar-10 dataset. We observed that our C-Local block got higher accuracy than SE block did.
Tasks
Published	2019-01-06
URL	http://arxiv.org/abs/1901.01493v1
PDF	http://arxiv.org/pdf/1901.01493v1.pdf
PWC	https://paperswithcode.com/paper/channel-locality-block-a-variant-of-squeeze
Repo
Framework