October 19, 2019

2951 words 14 mins read

Paper Group ANR 378

Security Matters: A Survey on Adversarial Machine Learning. Analyzing Language Learned by an Active Question Answering Agent. Hybrid Retrieval-Generation Reinforced Agent for Medical Image Report Generation. Transfer Learning from LDA to BiLSTM-CNN for Offensive Language Detection in Twitter. Machine learning regression on hyperspectral data to est …

Security Matters: A Survey on Adversarial Machine Learning


Title	Security Matters: A Survey on Adversarial Machine Learning
Authors	Guofu Li, Pengjia Zhu, Jin Li, Zhemin Yang, Ning Cao, Zhiyi Chen
Abstract	Adversarial machine learning is a fast growing research area, which considers the scenarios when machine learning systems may face potential adversarial attackers, who intentionally synthesize input data to make a well-trained model to make mistake. It always involves a defending side, usually a classifier, and an attacking side that aims to cause incorrect output. The earliest studies on the adversarial examples for machine learning algorithms start from the information security area, which considers a much wider varieties of attacking methods. But recent research focus that popularized by the deep learning community places strong emphasis on how the “imperceivable” perturbations on the normal inputs may cause dramatic mistakes by the deep learning with supposed super-human accuracy. This paper serves to give a comprehensive introduction to a range of aspects of the adversarial deep learning topic, including its foundations, typical attacking and defending strategies, and some extended studies.
Tasks
Published	2018-10-16
URL	http://arxiv.org/abs/1810.07339v2
PDF	http://arxiv.org/pdf/1810.07339v2.pdf
PWC	https://paperswithcode.com/paper/security-matters-a-survey-on-adversarial
Repo
Framework

Analyzing Language Learned by an Active Question Answering Agent


Title	Analyzing Language Learned by an Active Question Answering Agent
Authors	Christian Buck, Jannis Bulian, Massimiliano Ciaramita, Wojciech Gajewski, Andrea Gesmundo, Neil Houlsby, Wei Wang
Abstract	We analyze the language learned by an agent trained with reinforcement learning as a component of the ActiveQA system [Buck et al., 2017]. In ActiveQA, question answering is framed as a reinforcement learning task in which an agent sits between the user and a black box question-answering system. The agent learns to reformulate the user’s questions to elicit the optimal answers. It probes the system with many versions of a question that are generated via a sequence-to-sequence question reformulation model, then aggregates the returned evidence to find the best answer. This process is an instance of \emph{machine-machine} communication. The question reformulation model must adapt its language to increase the quality of the answers returned, matching the language of the question answering system. We find that the agent does not learn transformations that align with semantic intuitions but discovers through learning classical information retrieval techniques such as tf-idf re-weighting and stemming.
Tasks	Information Retrieval, Question Answering
Published	2018-01-23
URL	http://arxiv.org/abs/1801.07537v1
PDF	http://arxiv.org/pdf/1801.07537v1.pdf
PWC	https://paperswithcode.com/paper/analyzing-language-learned-by-an-active
Repo
Framework

Hybrid Retrieval-Generation Reinforced Agent for Medical Image Report Generation


Title	Hybrid Retrieval-Generation Reinforced Agent for Medical Image Report Generation
Authors	Christy Y. Li, Xiaodan Liang, Zhiting Hu, Eric P. Xing
Abstract	Generating long and coherent reports to describe medical images poses challenges to bridging visual patterns with informative human linguistic descriptions. We propose a novel Hybrid Retrieval-Generation Reinforced Agent (HRGR-Agent) which reconciles traditional retrieval-based approaches populated with human prior knowledge, with modern learning-based approaches to achieve structured, robust, and diverse report generation. HRGR-Agent employs a hierarchical decision-making procedure. For each sentence, a high-level retrieval policy module chooses to either retrieve a template sentence from an off-the-shelf template database, or invoke a low-level generation module to generate a new sentence. HRGR-Agent is updated via reinforcement learning, guided by sentence-level and word-level rewards. Experiments show that our approach achieves the state-of-the-art results on two medical report datasets, generating well-balanced structured sentences with robust coverage of heterogeneous medical report contents. In addition, our model achieves the highest detection accuracy of medical terminologies, and improved human evaluation performance.
Tasks	Decision Making
Published	2018-05-21
URL	http://arxiv.org/abs/1805.08298v2
PDF	http://arxiv.org/pdf/1805.08298v2.pdf
PWC	https://paperswithcode.com/paper/hybrid-retrieval-generation-reinforced-agent
Repo
Framework

Transfer Learning from LDA to BiLSTM-CNN for Offensive Language Detection in Twitter


Title	Transfer Learning from LDA to BiLSTM-CNN for Offensive Language Detection in Twitter
Authors	Gregor Wiedemann, Eugen Ruppert, Raghav Jindal, Chris Biemann
Abstract	We investigate different strategies for automatic offensive language classification on German Twitter data. For this, we employ a sequentially combined BiLSTM-CNN neural network. Based on this model, three transfer learning tasks to improve the classification performance with background knowledge are tested. We compare 1. Supervised category transfer: social media data annotated with near-offensive language categories, 2. Weakly-supervised category transfer: tweets annotated with emojis they contain, 3. Unsupervised category transfer: tweets annotated with topic clusters obtained by Latent Dirichlet Allocation (LDA). Further, we investigate the effect of three different strategies to mitigate negative effects of ‘catastrophic forgetting’ during transfer learning. Our results indicate that transfer learning in general improves offensive language detection. Best results are achieved from pre-training our model on the unsupervised topic clustering of tweets in combination with thematic user cluster information.
Tasks	Transfer Learning
Published	2018-11-07
URL	http://arxiv.org/abs/1811.02906v1
PDF	http://arxiv.org/pdf/1811.02906v1.pdf
PWC	https://paperswithcode.com/paper/transfer-learning-from-lda-to-bilstm-cnn-for
Repo
Framework

Machine learning regression on hyperspectral data to estimate multiple water parameters


Title	Machine learning regression on hyperspectral data to estimate multiple water parameters
Authors	Philipp M. Maier, Sina Keller
Abstract	In this paper, we present a regression framework involving several machine learning models to estimate water parameters based on hyperspectral data. Measurements from a multi-sensor field campaign, conducted on the River Elbe, Germany, represent the benchmark dataset. It contains hyperspectral data and the five water parameters chlorophyll a, green algae, diatoms, CDOM and turbidity. We apply a PCA for the high-dimensional data as a possible preprocessing step. Then, we evaluate the performance of the regression framework with and without this preprocessing step. The regression results of the framework clearly reveal the potential of estimating water parameters based on hyperspectral data with machine learning. The proposed framework provides the basis for further investigations, such as adapting the framework to estimate water parameters of different inland waters.
Tasks
Published	2018-05-03
URL	http://arxiv.org/abs/1805.01361v2
PDF	http://arxiv.org/pdf/1805.01361v2.pdf
PWC	https://paperswithcode.com/paper/machine-learning-regression-on-hyperspectral
Repo
Framework

Supervector Compression Strategies to Speed up I-Vector System Development


Title	Supervector Compression Strategies to Speed up I-Vector System Development
Authors	Ville Vestman, Tomi Kinnunen
Abstract	The front-end factor analysis (FEFA), an extension of principal component analysis (PPCA) tailored to be used with Gaussian mixture models (GMMs), is currently the prevalent approach to extract compact utterance-level features (i-vectors) for automatic speaker verification (ASV) systems. Little research has been conducted comparing FEFA to the conventional PPCA applied to maximum a posteriori (MAP) adapted GMM supervectors. We study several alternative methods, including PPCA, factor analysis (FA), and two supervised approaches, supervised PPCA (SPPCA) and the recently proposed probabilistic partial least squares (PPLS), to compress MAP-adapted GMM supervectors. The resulting i-vectors are used in ASV tasks with a probabilistic linear discriminant analysis (PLDA) back-end. We experiment on two different datasets, on the telephone condition of NIST SRE 2010 and on the recent VoxCeleb corpus collected from YouTube videos containing celebrity interviews recorded in various acoustical and technical conditions. The results suggest that, in terms of ASV accuracy, the supervector compression approaches are on a par with FEFA. The supervised approaches did not result in improved performance. In comparison to FEFA, we obtained more than hundred-fold (100x) speedups in the total variability model (TVM) training using the PPCA and FA supervector compression approaches.
Tasks	Speaker Verification
Published	2018-05-03
URL	http://arxiv.org/abs/1805.01156v1
PDF	http://arxiv.org/pdf/1805.01156v1.pdf
PWC	https://paperswithcode.com/paper/supervector-compression-strategies-to-speed
Repo
Framework

Learning Hybrid Sparsity Prior for Image Restoration: Where Deep Learning Meets Sparse Coding


Title	Learning Hybrid Sparsity Prior for Image Restoration: Where Deep Learning Meets Sparse Coding
Authors	Fangfang Wu, Weisheng Dong, Guangming Shi, Xin Li
Abstract	State-of-the-art approaches toward image restoration can be classified into model-based and learning-based. The former - best represented by sparse coding techniques - strive to exploit intrinsic prior knowledge about the unknown high-resolution images; while the latter - popularized by recently developed deep learning techniques - leverage external image prior from some training dataset. It is natural to explore their middle ground and pursue a hybrid image prior capable of achieving the best in both worlds. In this paper, we propose a systematic approach of achieving this goal called Structured Analysis Sparse Coding (SASC). Specifically, a structured sparse prior is learned from extrinsic training data via a deep convolutional neural network (in a similar way to previous learning-based approaches); meantime another structured sparse prior is internally estimated from the input observation image (similar to previous model-based approaches). Two structured sparse priors will then be combined to produce a hybrid prior incorporating the knowledge from both domains. To manage the computational complexity, we have developed a novel framework of implementing hybrid structured sparse coding processes by deep convolutional neural networks. Experimental results show that the proposed hybrid image restoration method performs comparably with and often better than the current state-of-the-art techniques.
Tasks	Image Restoration
Published	2018-07-18
URL	http://arxiv.org/abs/1807.06920v2
PDF	http://arxiv.org/pdf/1807.06920v2.pdf
PWC	https://paperswithcode.com/paper/learning-hybrid-sparsity-prior-for-image
Repo
Framework

Phonetic-and-Semantic Embedding of Spoken Words with Applications in Spoken Content Retrieval


Title	Phonetic-and-Semantic Embedding of Spoken Words with Applications in Spoken Content Retrieval
Authors	Yi-Chen Chen, Sung-Feng Huang, Chia-Hao Shen, Hung-yi Lee, Lin-shan Lee
Abstract	Word embedding or Word2Vec has been successful in offering semantics for text words learned from the context of words. Audio Word2Vec was shown to offer phonetic structures for spoken words (signal segments for words) learned from signals within spoken words. This paper proposes a two-stage framework to perform phonetic-and-semantic embedding on spoken words considering the context of the spoken words. Stage 1 performs phonetic embedding with speaker characteristics disentangled. Stage 2 then performs semantic embedding in addition. We further propose to evaluate the phonetic-and-semantic nature of the audio embeddings obtained in Stage 2 by parallelizing with text embeddings. In general, phonetic structure and semantics inevitably disturb each other. For example the words “brother” and “sister” are close in semantics but very different in phonetic structure, while the words “brother” and “bother” are in the other way around. But phonetic-and-semantic embedding is attractive, as shown in the initial experiments on spoken document retrieval. Not only spoken documents including the spoken query can be retrieved based on the phonetic structures, but spoken documents semantically related to the query but not including the query can also be retrieved based on the semantics.
Tasks
Published	2018-07-21
URL	http://arxiv.org/abs/1807.08089v4
PDF	http://arxiv.org/pdf/1807.08089v4.pdf
PWC	https://paperswithcode.com/paper/phonetic-and-semantic-embedding-of-spoken
Repo
Framework

Segmentation of histological images and fibrosis identification with a convolutional neural network


Title	Segmentation of histological images and fibrosis identification with a convolutional neural network
Authors	Xiaohang Fu, Tong Liu, Zhaohan Xiong, Bruce H. Smaill, Martin K. Stiles, Jichao Zhao
Abstract	Segmentation of histological images is one of the most crucial tasks for many biomedical analyses including quantification of certain tissue type. However, challenges are posed by high variability and complexity of structural features in such images, in addition to imaging artifacts. Further, the conventional approach of manual thresholding is labor-intensive, and highly sensitive to inter- and intra-image intensity variations. An accurate and robust automated segmentation method is of high interest. We propose and evaluate an elegant convolutional neural network (CNN) designed for segmentation of histological images, particularly those with Masson’s trichrome stain. The network comprises of 11 successive convolutional - rectified linear unit - batch normalization layers, and outperformed state-of-the-art CNNs on a dataset of cardiac histological images (labeling fibrosis, myocytes, and background) with a Dice similarity coefficient of 0.947. With 100 times fewer (only 300 thousand) trainable parameters, our CNN is less susceptible to overfitting, and is efficient. Additionally, it retains image resolution from input to output, captures fine-grained details, and can be trained end-to-end smoothly. To the best of our knowledge, this is the first deep CNN tailored for the problem of concern, and may be extended to solve similar segmentation tasks to facilitate investigations into pathology and clinical treatment.
Tasks
Published	2018-03-20
URL	http://arxiv.org/abs/1803.07301v1
PDF	http://arxiv.org/pdf/1803.07301v1.pdf
PWC	https://paperswithcode.com/paper/segmentation-of-histological-images-and
Repo
Framework

Bridging the Accuracy Gap for 2-bit Quantized Neural Networks (QNN)


Title	Bridging the Accuracy Gap for 2-bit Quantized Neural Networks (QNN)
Authors	Jungwook Choi, Pierce I-Jen Chuang, Zhuo Wang, Swagath Venkataramani, Vijayalakshmi Srinivasan, Kailash Gopalakrishnan
Abstract	Deep learning algorithms achieve high classification accuracy at the expense of significant computation cost. In order to reduce this cost, several quantization schemes have gained attention recently with some focusing on weight quantization, and others focusing on quantizing activations. This paper proposes novel techniques that target weight and activation quantizations separately resulting in an overall quantized neural network (QNN). The activation quantization technique, PArameterized Clipping acTivation (PACT), uses an activation clipping parameter $\alpha$ that is optimized during training to find the right quantization scale. The weight quantization scheme, statistics-aware weight binning (SAWB), finds the optimal scaling factor that minimizes the quantization error based on the statistical characteristics of the distribution of weights without the need for an exhaustive search. The combination of PACT and SAWB results in a 2-bit QNN that achieves state-of-the-art classification accuracy (comparable to full precision networks) across a range of popular models and datasets.
Tasks	Quantization
Published	2018-07-17
URL	http://arxiv.org/abs/1807.06964v1
PDF	http://arxiv.org/pdf/1807.06964v1.pdf
PWC	https://paperswithcode.com/paper/bridging-the-accuracy-gap-for-2-bit-quantized
Repo
Framework

Interpreting Complex Regression Models


Title	Interpreting Complex Regression Models
Authors	Noa Avigdor-Elgrabli, Alex Libov, Michael Viderman, Ran Wolff
Abstract	Interpretation of a machine learning induced models is critical for feature engineering, debugging, and, arguably, compliance. Yet, best of breed machine learning models tend to be very complex. This paper presents a method for model interpretation which has the main benefit that the simple interpretations it provides are always grounded in actual sets of learning examples. The method is validated on the task of interpreting a complex regression model in the context of both an academic problem – predicting the year in which a song was recorded and an industrial one – predicting mail user churn.
Tasks	Feature Engineering
Published	2018-02-26
URL	http://arxiv.org/abs/1802.09225v1
PDF	http://arxiv.org/pdf/1802.09225v1.pdf
PWC	https://paperswithcode.com/paper/interpreting-complex-regression-models
Repo
Framework

Using Deep Learning for price prediction by exploiting stationary limit order book features


Title	Using Deep Learning for price prediction by exploiting stationary limit order book features
Authors	Avraam Tsantekidis, Nikolaos Passalis, Anastasios Tefas, Juho Kanniainen, Moncef Gabbouj, Alexandros Iosifidis
Abstract	The recent surge in Deep Learning (DL) research of the past decade has successfully provided solutions to many difficult problems. The field of quantitative analysis has been slowly adapting the new methods to its problems, but due to problems such as the non-stationary nature of financial data, significant challenges must be overcome before DL is fully utilized. In this work a new method to construct stationary features, that allows DL models to be applied effectively, is proposed. These features are thoroughly tested on the task of predicting mid price movements of the Limit Order Book. Several DL models are evaluated, such as recurrent Long Short Term Memory (LSTM) networks and Convolutional Neural Networks (CNN). Finally a novel model that combines the ability of CNNs to extract useful features and the ability of LSTMs’ to analyze time series, is proposed and evaluated. The combined model is able to outperform the individual LSTM and CNN models in the prediction horizons that are tested.
Tasks	Time Series
Published	2018-10-23
URL	http://arxiv.org/abs/1810.09965v1
PDF	http://arxiv.org/pdf/1810.09965v1.pdf
PWC	https://paperswithcode.com/paper/using-deep-learning-for-price-prediction-by
Repo
Framework

Characterizing classification datasets: a study of meta-features for meta-learning


Title	Characterizing classification datasets: a study of meta-features for meta-learning
Authors	Adriano Rivolli, Luís P. F. Garcia, Carlos Soares, Joaquin Vanschoren, André C. P. L. F. de Carvalho
Abstract	Meta-learning is increasingly used to support the recommendation of machine learning algorithms and their configurations. Such recommendations are made based on meta-data, consisting of performance evaluations of algorithms on prior datasets, as well as characterizations of these datasets. These characterizations, also called meta-features, describe properties of the data which are predictive for the performance of machine learning algorithms trained on them. Unfortunately, despite being used in a large number of studies, meta-features are not uniformly described, organized and computed, making many empirical studies irreproducible and hard to compare. This paper aims to deal with this by systematizing and standardizing data characterization measures for classification datasets used in meta-learning. Moreover, it presents MFE, a new tool for extracting meta-features from datasets and identifying more subtle reproducibility issues in the literature, proposing guidelines for data characterization that strengthen reproducible empirical research in meta-learning.
Tasks	Meta-Learning
Published	2018-08-30
URL	https://arxiv.org/abs/1808.10406v2
PDF	https://arxiv.org/pdf/1808.10406v2.pdf
PWC	https://paperswithcode.com/paper/towards-reproducible-empirical-research-in
Repo
Framework

Evolutionary Neural Architecture Search for Image Restoration


Title	Evolutionary Neural Architecture Search for Image Restoration
Authors	Gerard Jacques van Wyk, Anna Sergeevna Bosman
Abstract	Convolutional neural network (CNN) architectures have traditionally been explored by human experts in a manual search process that is time-consuming and ineffectively explores the massive space of potential solutions. Neural architecture search (NAS) methods automatically search the space of neural network hyperparameters in order to find optimal task-specific architectures. NAS methods have discovered CNN architectures that achieve state-of-the-art performance in image classification among other tasks, however the application of NAS to image-to-image regression problems such as image restoration is sparse. This paper proposes a NAS method that performs computationally efficient evolutionary search of a minimally constrained network architecture search space. The performance of architectures discovered by the proposed method is evaluated on a variety of image restoration tasks applied to the ImageNet64x64 dataset, and compared with human-engineered CNN architectures. The best neural architectures discovered using only 2 GPU-hours of evolutionary search exhibit comparable performance to the human-engineered baseline architecture.
Tasks	Image Classification, Image Restoration, Neural Architecture Search
Published	2018-12-14
URL	http://arxiv.org/abs/1812.05866v2
PDF	http://arxiv.org/pdf/1812.05866v2.pdf
PWC	https://paperswithcode.com/paper/evolutionary-neural-architecture-search-for
Repo
Framework

Defense Against the Dark Arts: An overview of adversarial example security research and future research directions


Title	Defense Against the Dark Arts: An overview of adversarial example security research and future research directions
Authors	Ian Goodfellow
Abstract	This article presents a summary of a keynote lecture at the Deep Learning Security workshop at IEEE Security and Privacy 2018. This lecture summarizes the state of the art in defenses against adversarial examples and provides recommendations for future research directions on this topic.
Tasks
Published	2018-06-11
URL	http://arxiv.org/abs/1806.04169v1
PDF	http://arxiv.org/pdf/1806.04169v1.pdf
PWC	https://paperswithcode.com/paper/defense-against-the-dark-arts-an-overview-of
Repo
Framework