April 1, 2020

3020 words 15 mins read

Paper Group ANR 492

Learning to Generate Multiple Style Transfer Outputs for an Input Sentence. Regularized Optimal Transport is Ground Cost Adversarial. Co-occurrence Background Model with Superpixels for Robust Background Initialization. Excitation-based Voice Quality Analysis and Modification. Boosting Algorithms for Estimating Optimal Individualized Treatment Rule …

Learning to Generate Multiple Style Transfer Outputs for an Input Sentence


Title	Learning to Generate Multiple Style Transfer Outputs for an Input Sentence
Authors	Kevin Lin, Ming-Yu Liu, Ming-Ting Sun, Jan Kautz
Abstract	Text style transfer refers to the task of rephrasing a given text in a different style. While various methods have been proposed to advance the state of the art, they often assume the transfer output follows a delta distribution, and thus their models cannot generate different style transfer results for a given input text. To address the limitation, we propose a one-to-many text style transfer framework. In contrast to prior works that learn a one-to-one mapping that converts an input sentence to one output sentence, our approach learns a one-to-many mapping that can convert an input sentence to multiple different output sentences, while preserving the input content. This is achieved by applying adversarial training with a latent decomposition scheme. Specifically, we decompose the latent representation of the input sentence to a style code that captures the language style variation and a content code that encodes the language style-independent content. We then combine the content code with the style code for generating a style transfer output. By combining the same content code with a different style code, we generate a different style transfer output. Extensive experimental results with comparisons to several text style transfer approaches on multiple public datasets using a diverse set of performance metrics validate effectiveness of the proposed approach.
Tasks	Style Transfer, Text Style Transfer
Published	2020-02-16
URL	https://arxiv.org/abs/2002.06525v1
PDF	https://arxiv.org/pdf/2002.06525v1.pdf
PWC	https://paperswithcode.com/paper/learning-to-generate-multiple-style-transfer
Repo
Framework

Regularized Optimal Transport is Ground Cost Adversarial


Title	Regularized Optimal Transport is Ground Cost Adversarial
Authors	François-Pierre Paty, Marco Cuturi
Abstract	Regularizing Wasserstein distances has proved to be the key in the recent advances of optimal transport (OT) in machine learning. Most prominent is the entropic regularization of OT, which not only allows for fast computations and differentiation using Sinkhorn algorithm, but also improves stability with respect to data and accuracy in many numerical experiments. Theoretical understanding of these benefits remains unclear, although recent statistical works have shown that entropy-regularized OT mitigates classical OT’s curse of dimensionality. In this paper, we adopt a more geometrical point of view, and show using Fenchel duality that any convex regularization of OT can be interpreted as ground cost adversarial. This incidentally gives access to a robust dissimilarity measure on the ground space, which can in turn be used in other applications. We propose algorithms to compute this robust cost, and illustrate the interest of this approach empirically.
Tasks
Published	2020-02-10
URL	https://arxiv.org/abs/2002.03967v1
PDF	https://arxiv.org/pdf/2002.03967v1.pdf
PWC	https://paperswithcode.com/paper/regularized-optimal-transport-is-ground-cost
Repo
Framework

Co-occurrence Background Model with Superpixels for Robust Background Initialization


Title	Co-occurrence Background Model with Superpixels for Robust Background Initialization
Authors	Wenjun Zhou, Yuheng Deng, Bo Peng, Dong Liang, Shun’ichi Kaneko
Abstract	Background initialization is an important step in many high-level applications of video processing,ranging from video surveillance to video inpainting.However,this process is often affected by practical challenges such as illumination changes,background motion,camera jitter and intermittent movement,etc.In this paper,we develop a co-occurrence background model with superpixel segmentation for robust background initialization. We first introduce a novel co-occurrence background modeling method called as Co-occurrence Pixel-Block Pairs(CPB)to generate a reliable initial background model,and the superpixel segmentation is utilized to further acquire the spatial texture Information of foreground and background.Then,the initial background can be determined by combining the foreground extraction results with the superpixel segmentation information.Experimental results obtained from the dataset of the challenging benchmark(SBMnet)validate it’s performance under various challenges.
Tasks
Published	2020-03-29
URL	https://arxiv.org/abs/2003.12931v1
PDF	https://arxiv.org/pdf/2003.12931v1.pdf
PWC	https://paperswithcode.com/paper/co-occurrence-background-model-with
Repo
Framework

Excitation-based Voice Quality Analysis and Modification


Title	Excitation-based Voice Quality Analysis and Modification
Authors	Thomas Drugman, Thierry Dutoit, Baris Bozkurt
Abstract	This paper investigates the differences occuring in the excitation for different voice qualities. Its goal is two-fold. First a large corpus containing three voice qualities (modal, soft and loud) uttered by the same speaker is analyzed and significant differences in characteristics extracted from the excitation are observed. Secondly rules of modification derived from the analysis are used to build a voice quality transformation system applied as a post-process to HMM-based speech synthesis. The system is shown to effectively achieve the transformations while maintaining the delivered quality.
Tasks	Speech Synthesis
Published	2020-01-02
URL	https://arxiv.org/abs/2001.00582v1
PDF	https://arxiv.org/pdf/2001.00582v1.pdf
PWC	https://paperswithcode.com/paper/excitation-based-voice-quality-analysis-and
Repo
Framework

Boosting Algorithms for Estimating Optimal Individualized Treatment Rules


Title	Boosting Algorithms for Estimating Optimal Individualized Treatment Rules
Authors	Duzhe Wang, Haoda Fu, Po-Ling Loh
Abstract	We present nonparametric algorithms for estimating optimal individualized treatment rules. The proposed algorithms are based on the XGBoost algorithm, which is known as one of the most powerful algorithms in the machine learning literature. Our main idea is to model the conditional mean of clinical outcome or the decision rule via additive regression trees, and use the boosting technique to estimate each single tree iteratively. Our approaches overcome the challenge of correct model specification, which is required in current parametric methods. The major contribution of our proposed algorithms is providing efficient and accurate estimation of the highly nonlinear and complex optimal individualized treatment rules that often arise in practice. Finally, we illustrate the superior performance of our algorithms by extensive simulation studies and conclude with an application to the real data from a diabetes Phase III trial.
Tasks
Published	2020-01-31
URL	https://arxiv.org/abs/2002.00079v1
PDF	https://arxiv.org/pdf/2002.00079v1.pdf
PWC	https://paperswithcode.com/paper/boosting-algorithms-for-estimating-optimal
Repo
Framework

Massif: Interactive Interpretation of Adversarial Attacks on Deep Learning


Title	Massif: Interactive Interpretation of Adversarial Attacks on Deep Learning
Authors	Nilaksh Das, Haekyu Park, Zijie J. Wang, Fred Hohman, Robert Firstman, Emily Rogers, Duen Horng Chau
Abstract	Deep neural networks (DNNs) are increasingly powering high-stakes applications such as autonomous cars and healthcare; however, DNNs are often treated as “black boxes” in such applications. Recent research has also revealed that DNNs are highly vulnerable to adversarial attacks, raising serious concerns over deploying DNNs in the real world. To overcome these deficiencies, we are developing Massif, an interactive tool for deciphering adversarial attacks. Massif identifies and interactively visualizes neurons and their connections inside a DNN that are strongly activated or suppressed by an adversarial attack. Massif provides both a high-level, interpretable overview of the effect of an attack on a DNN, and a low-level, detailed description of the affected neurons. These tightly coupled views in Massif help people better understand which input features are most vulnerable or important for correct predictions.
Tasks	Adversarial Attack
Published	2020-01-21
URL	https://arxiv.org/abs/2001.07769v3
PDF	https://arxiv.org/pdf/2001.07769v3.pdf
PWC	https://paperswithcode.com/paper/massif-interactive-interpretation-of
Repo
Framework

Learn2Perturb: an End-to-end Feature Perturbation Learning to Improve Adversarial Robustness


Title	Learn2Perturb: an End-to-end Feature Perturbation Learning to Improve Adversarial Robustness
Authors	Ahmadreza Jeddi, Mohammad Javad Shafiee, Michelle Karg, Christian Scharfenberger, Alexander Wong
Abstract	While deep neural networks have been achieving state-of-the-art performance across a wide variety of applications, their vulnerability to adversarial attacks limits their widespread deployment for safety-critical applications. Alongside other adversarial defense approaches being investigated, there has been a very recent interest in improving adversarial robustness in deep neural networks through the introduction of perturbations during the training process. However, such methods leverage fixed, pre-defined perturbations and require significant hyper-parameter tuning that makes them very difficult to leverage in a general fashion. In this study, we introduce Learn2Perturb, an end-to-end feature perturbation learning approach for improving the adversarial robustness of deep neural networks. More specifically, we introduce novel perturbation-injection modules that are incorporated at each layer to perturb the feature space and increase uncertainty in the network. This feature perturbation is performed at both the training and the inference stages. Furthermore, inspired by the Expectation-Maximization, an alternating back-propagation training algorithm is introduced to train the network and noise parameters consecutively. Experimental results on CIFAR-10 and CIFAR-100 datasets show that the proposed Learn2Perturb method can result in deep neural networks which are $4-7%$ more robust on $l_{\infty}$ FGSM and PDG adversarial attacks and significantly outperforms the state-of-the-art against $l_2$ $C&W$ attack and a wide range of well-known black-box attacks.
Tasks	Adversarial Defense
Published	2020-03-02
URL	https://arxiv.org/abs/2003.01090v2
PDF	https://arxiv.org/pdf/2003.01090v2.pdf
PWC	https://paperswithcode.com/paper/learn2perturb-an-end-to-end-feature
Repo
Framework

Streamlined Empirical Bayes Fitting of Linear Mixed Models in Mobile Health


Title	Streamlined Empirical Bayes Fitting of Linear Mixed Models in Mobile Health
Authors	Marianne Menictas, Sabina Tomkins, Susan A Murphy
Abstract	To effect behavior change a successful algorithm must make high-quality decisions in real-time. For example, a mobile health (mHealth) application designed to increase physical activity must make contextually relevant suggestions to motivate users. While machine learning offers solutions for certain stylized settings, such as when batch data can be processed offline, there is a dearth of approaches which can deliver high-quality solutions under the specific constraints of mHealth. We propose an algorithm which provides users with contextualized and personalized physical activity suggestions. This algorithm is able to overcome a challenge critical to mHealth that complex models be trained efficiently. We propose a tractable streamlined empirical Bayes procedure which fits linear mixed effects models in large-data settings. Our procedure takes advantage of sparsity introduced by hierarchical random effects to efficiently learn the posterior distribution of a linear mixed effects model. A key contribution of this work is that we provide explicit updates in order to learn both fixed effects, random effects and hyper-parameter values. We demonstrate the success of this approach in a mobile health (mHealth) reinforcement learning application, a domain in which fast computations are crucial for real time interventions. Not only is our approach computationally efficient, it is also easily implemented with closed form matrix algebraic updates and we show improvements over state of the art approaches both in speed and accuracy of up to 99% and 56% respectively.
Tasks
Published	2020-03-28
URL	https://arxiv.org/abs/2003.12881v1
PDF	https://arxiv.org/pdf/2003.12881v1.pdf
PWC	https://paperswithcode.com/paper/streamlined-empirical-bayes-fitting-of-linear
Repo
Framework

Federated Residual Learning


Title	Federated Residual Learning
Authors	Alekh Agarwal, John Langford, Chen-Yu Wei
Abstract	We study a new form of federated learning where the clients train personalized local models and make predictions jointly with the server-side shared model. Using this new federated learning framework, the complexity of the central shared model can be minimized while still gaining all the performance benefits that joint training provides. Our framework is robust to data heterogeneity, addressing the slow convergence problem traditional federated learning methods face when the data is non-i.i.d. across clients. We test the theory empirically and find substantial performance gains over baselines.
Tasks
Published	2020-03-28
URL	https://arxiv.org/abs/2003.12880v1
PDF	https://arxiv.org/pdf/2003.12880v1.pdf
PWC	https://paperswithcode.com/paper/federated-residual-learning
Repo
Framework

ReLaText: Exploiting Visual Relationships for Arbitrary-Shaped Scene Text Detection with Graph Convolutional Networks


Title	ReLaText: Exploiting Visual Relationships for Arbitrary-Shaped Scene Text Detection with Graph Convolutional Networks
Authors	Chixiang Ma, Lei Sun, Zhuoyao Zhong, Qiang Huo
Abstract	We introduce a new arbitrary-shaped text detection approach named ReLaText by formulating text detection as a visual relationship detection problem. To demonstrate the effectiveness of this new formulation, we start from using a “link” relationship to address the challenging text-line grouping problem firstly. The key idea is to decompose text detection into two subproblems, namely detection of text primitives and prediction of link relationships between nearby text primitive pairs. Specifically, an anchor-free region proposal network based text detector is first used to detect text primitives of different scales from different feature maps of a feature pyramid network, from which a text primitive graph is constructed by linking each pair of nearby text primitives detected from a same feature map with an edge. Then, a Graph Convolutional Network (GCN) based link relationship prediction module is used to prune wrongly-linked edges in the text primitive graph to generate a number of disjoint subgraphs, each representing a detected text instance. As GCN can effectively leverage context information to improve link prediction accuracy, our GCN based text-line grouping approach can achieve better text detection accuracy than previous text-line grouping methods, especially when dealing with text instances with large inter-character or very small inter-line spacings. Consequently, the proposed ReLaText achieves state-of-the-art performance on five public text detection benchmarks, namely RCTW-17, MSRA-TD500, Total-Text, CTW1500 and DAST1500.
Tasks	Link Prediction, Scene Text Detection
Published	2020-03-16
URL	https://arxiv.org/abs/2003.06999v1
PDF	https://arxiv.org/pdf/2003.06999v1.pdf
PWC	https://paperswithcode.com/paper/relatext-exploiting-visual-relationships-for
Repo
Framework

Anomaly Detection with Density Estimation


Title	Anomaly Detection with Density Estimation
Authors	Benjamin Nachman, David Shih
Abstract	We leverage recent breakthroughs in neural density estimation to propose a new unsupervised anomaly detection technique (ANODE). By estimating the probability density of the data in a signal region and in sidebands, and interpolating the latter into the signal region, a likelihood ratio of data vs. background can be constructed. This likelihood ratio is broadly sensitive to overdensities in the data that could be due to localized anomalies. In addition, a unique potential benefit of the ANODE method is that the background can be directly estimated using the learned densities. Finally, ANODE is robust against systematic differences between signal region and sidebands, giving it broader applicability than other methods. We demonstrate the power of this new approach using the LHC Olympics 2020 R&D Dataset. We show how ANODE can enhance the significance of a dijet bump hunt by up to a factor of 7 with a 10% accuracy on the background prediction. While the LHC is used as the recurring example, the methods developed here have a much broader applicability to anomaly detection in physics and beyond.
Tasks	Anomaly Detection, Density Estimation, Unsupervised Anomaly Detection
Published	2020-01-14
URL	https://arxiv.org/abs/2001.04990v1
PDF	https://arxiv.org/pdf/2001.04990v1.pdf
PWC	https://paperswithcode.com/paper/anomaly-detection-with-density-estimation
Repo
Framework

On Last-Layer Algorithms for Classification: Decoupling Representation from Uncertainty Estimation


Title	On Last-Layer Algorithms for Classification: Decoupling Representation from Uncertainty Estimation
Authors	Nicolas Brosse, Carlos Riquelme, Alice Martin, Sylvain Gelly, Éric Moulines
Abstract	Uncertainty quantification for deep learning is a challenging open problem. Bayesian statistics offer a mathematically grounded framework to reason about uncertainties; however, approximate posteriors for modern neural networks still require prohibitive computational costs. We propose a family of algorithms which split the classification task into two stages: representation learning and uncertainty estimation. We compare four specific instances, where uncertainty estimation is performed via either an ensemble of Stochastic Gradient Descent or Stochastic Gradient Langevin Dynamics snapshots, an ensemble of bootstrapped logistic regressions, or via a number of Monte Carlo Dropout passes. We evaluate their performance in terms of \emph{selective} classification (risk-coverage), and their ability to detect out-of-distribution samples. Our experiments suggest there is limited value in adding multiple uncertainty layers to deep classifiers, and we observe that these simple methods strongly outperform a vanilla point-estimate SGD in some complex benchmarks like ImageNet.
Tasks	Representation Learning
Published	2020-01-22
URL	https://arxiv.org/abs/2001.08049v1
PDF	https://arxiv.org/pdf/2001.08049v1.pdf
PWC	https://paperswithcode.com/paper/on-last-layer-algorithms-for-classification
Repo
Framework

A Precisely Xtreme-Multi Channel Hybrid Approach For Roman Urdu Sentiment Analysis


Title	A Precisely Xtreme-Multi Channel Hybrid Approach For Roman Urdu Sentiment Analysis
Authors	Faiza Memood, Muhammad Usman Ghani, Muhammad Ali Ibrahim, Rehab Shehzadi, Muhammad Nabeel Asim
Abstract	In order to accelerate the performance of various Natural Language Processing tasks for Roman Urdu, this paper for the very first time provides 3 neural word embeddings prepared using most widely used approaches namely Word2vec, FastText, and Glove. The integrity of generated neural word embeddings is evaluated using intrinsic and extrinsic evaluation approaches. Considering the lack of publicly available benchmark datasets, it provides a first-ever Roman Urdu dataset which consists of 3241 sentiments annotated against positive, negative and neutral classes. To provide benchmark baseline performance over the presented dataset, we adapt diverse machine learning (Support Vector Machine Logistic Regression, Naive Bayes), deep learning (convolutional neural network, recurrent neural network), and hybrid approaches. Effectiveness of generated neural word embeddings is evaluated by comparing the performance of machine and deep learning based methodologies using 7, and 5 distinct feature representation approaches respectively. Finally, it proposes a novel precisely extreme multi-channel hybrid methodology which outperforms state-of-the-art adapted machine and deep learning approaches by the figure of 9%, and 4% in terms of F1-score. Roman Urdu Sentiment Analysis, Pretrain word embeddings for Roman Urdu, Word2Vec, Glove, Fast-Text
Tasks	Sentiment Analysis, Word Embeddings
Published	2020-03-11
URL	https://arxiv.org/abs/2003.05443v1
PDF	https://arxiv.org/pdf/2003.05443v1.pdf
PWC	https://paperswithcode.com/paper/a-precisely-xtreme-multi-channel-hybrid
Repo
Framework

Channel Pruning Guided by Classification Loss and Feature Importance


Title	Channel Pruning Guided by Classification Loss and Feature Importance
Authors	Jinyang Guo, Wanli Ouyang, Dong Xu
Abstract	In this work, we propose a new layer-by-layer channel pruning method called Channel Pruning guided by classification Loss and feature Importance (CPLI). In contrast to the existing layer-by-layer channel pruning approaches that only consider how to reconstruct the features from the next layer, our approach additionally take the classification loss into account in the channel pruning process. We also observe that some reconstructed features will be removed at the next pruning stage. So it is unnecessary to reconstruct these features. To this end, we propose a new strategy to suppress the influence of unimportant features (i.e., the features will be removed at the next pruning stage). Our comprehensive experiments on three benchmark datasets, i.e., CIFAR-10, ImageNet, and UCF-101, demonstrate the effectiveness of our CPLI method.
Tasks	Feature Importance
Published	2020-03-15
URL	https://arxiv.org/abs/2003.06757v1
PDF	https://arxiv.org/pdf/2003.06757v1.pdf
PWC	https://paperswithcode.com/paper/channel-pruning-guided-by-classification-loss
Repo
Framework

Exploiting the Full Capacity of Deep Neural Networks while Avoiding Overfitting by Targeted Sparsity Regularization


Title	Exploiting the Full Capacity of Deep Neural Networks while Avoiding Overfitting by Targeted Sparsity Regularization
Authors	Karim Huesmann, Soeren Klemm, Lars Linsen, Benjamin Risse
Abstract	Overfitting is one of the most common problems when training deep neural networks on comparatively small datasets. Here, we demonstrate that neural network activation sparsity is a reliable indicator for overfitting which we utilize to propose novel targeted sparsity visualization and regularization strategies. Based on these strategies we are able to understand and counteract overfitting caused by activation sparsity and filter correlation in a targeted layer-by-layer manner. Our results demonstrate that targeted sparsity regularization can efficiently be used to regularize well-known datasets and architectures with a significant increase in image classification performance while outperforming both dropout and batch normalization. Ultimately, our study reveals novel insights into the contradicting concepts of activation sparsity and network capacity by demonstrating that targeted sparsity regularization enables salient and discriminative feature learning while exploiting the full capacity of deep models without suffering from overfitting, even when trained excessively.
Tasks	Image Classification
Published	2020-02-21
URL	https://arxiv.org/abs/2002.09237v1
PDF	https://arxiv.org/pdf/2002.09237v1.pdf
PWC	https://paperswithcode.com/paper/exploiting-the-full-capacity-of-deep-neural
Repo
Framework