October 20, 2019

3193 words 15 mins read

Paper Group AWR 311

Deterministic Variational Inference for Robust Bayesian Neural Networks. One-Class Adversarial Nets for Fraud Detection. Black-box Adversarial Attacks with Limited Queries and Information. Learning Discriminative Features with Multiple Granularities for Person Re-Identification. Evaluating surgical skills from kinematic data using convolutional neu …

Deterministic Variational Inference for Robust Bayesian Neural Networks


Title	Deterministic Variational Inference for Robust Bayesian Neural Networks
Authors	Anqi Wu, Sebastian Nowozin, Edward Meeds, Richard E. Turner, José Miguel Hernández-Lobato, Alexander L. Gaunt
Abstract	Bayesian neural networks (BNNs) hold great promise as a flexible and principled solution to deal with uncertainty when learning from finite data. Among approaches to realize probabilistic inference in deep neural networks, variational Bayes (VB) is theoretically grounded, generally applicable, and computationally efficient. With wide recognition of potential advantages, why is it that variational Bayes has seen very limited practical use for BNNs in real applications? We argue that variational inference in neural networks is fragile: successful implementations require careful initialization and tuning of prior variances, as well as controlling the variance of Monte Carlo gradient estimates. We provide two innovations that aim to turn VB into a robust inference tool for Bayesian neural networks: first, we introduce a novel deterministic method to approximate moments in neural networks, eliminating gradient variance; second, we introduce a hierarchical prior for parameters and a novel Empirical Bayes procedure for automatically selecting prior variances. Combining these two innovations, the resulting method is highly efficient and robust. On the application of heteroscedastic regression we demonstrate good predictive performance over alternative approaches.
Tasks
Published	2018-10-09
URL	http://arxiv.org/abs/1810.03958v2
PDF	http://arxiv.org/pdf/1810.03958v2.pdf
PWC	https://paperswithcode.com/paper/deterministic-variational-inference-for
Repo	https://github.com/markovalexander/DVI
Framework	pytorch

One-Class Adversarial Nets for Fraud Detection


Title	One-Class Adversarial Nets for Fraud Detection
Authors	Panpan Zheng, Shuhan Yuan, Xintao Wu, Jun Li, Aidong Lu
Abstract	Many online applications, such as online social networks or knowledge bases, are often attacked by malicious users who commit different types of actions such as vandalism on Wikipedia or fraudulent reviews on eBay. Currently, most of the fraud detection approaches require a training dataset that contains records of both benign and malicious users. However, in practice, there are often no or very few records of malicious users. In this paper, we develop one-class adversarial nets (OCAN) for fraud detection using training data with only benign users. OCAN first uses LSTM-Autoencoder to learn the representations of benign users from their sequences of online activities. It then detects malicious users by training a discriminator with a complementary GAN model that is different from the regular GAN model. Experimental results show that our OCAN outperforms the state-of-the-art one-class classification models and achieves comparable performance with the latest multi-source LSTM model that requires both benign and malicious users in the training phase.
Tasks	Fraud Detection
Published	2018-03-05
URL	http://arxiv.org/abs/1803.01798v2
PDF	http://arxiv.org/pdf/1803.01798v2.pdf
PWC	https://paperswithcode.com/paper/one-class-adversarial-nets-for-fraud
Repo	https://github.com/PanpanZheng/OCAN
Framework	tf

Black-box Adversarial Attacks with Limited Queries and Information


Title	Black-box Adversarial Attacks with Limited Queries and Information
Authors	Andrew Ilyas, Logan Engstrom, Anish Athalye, Jessy Lin
Abstract	Current neural network-based classifiers are susceptible to adversarial examples even in the black-box setting, where the attacker only has query access to the model. In practice, the threat model for real-world systems is often more restrictive than the typical black-box model where the adversary can observe the full output of the network on arbitrarily many chosen inputs. We define three realistic threat models that more accurately characterize many real-world classifiers: the query-limited setting, the partial-information setting, and the label-only setting. We develop new attacks that fool classifiers under these more restrictive threat models, where previous methods would be impractical or ineffective. We demonstrate that our methods are effective against an ImageNet classifier under our proposed threat models. We also demonstrate a targeted black-box attack against a commercial classifier, overcoming the challenges of limited query access, partial information, and other practical issues to break the Google Cloud Vision API.
Tasks
Published	2018-04-23
URL	http://arxiv.org/abs/1804.08598v3
PDF	http://arxiv.org/pdf/1804.08598v3.pdf
PWC	https://paperswithcode.com/paper/black-box-adversarial-attacks-with-limited
Repo	https://github.com/mllab-adv-attack/lazy-attack
Framework	tf

Learning Discriminative Features with Multiple Granularities for Person Re-Identification


Title	Learning Discriminative Features with Multiple Granularities for Person Re-Identification
Authors	Guanshuo Wang, Yufeng Yuan, Xiong Chen, Jiwei Li, Xi Zhou
Abstract	The combination of global and partial features has been an essential solution to improve discriminative performances in person re-identification (Re-ID) tasks. Previous part-based methods mainly focus on locating regions with specific pre-defined semantics to learn local representations, which increases learning difficulty but not efficient or robust to scenarios with large variances. In this paper, we propose an end-to-end feature learning strategy integrating discriminative information with various granularities. We carefully design the Multiple Granularity Network (MGN), a multi-branch deep network architecture consisting of one branch for global feature representations and two branches for local feature representations. Instead of learning on semantic regions, we uniformly partition the images into several stripes, and vary the number of parts in different local branches to obtain local feature representations with multiple granularities. Comprehensive experiments implemented on the mainstream evaluation datasets including Market-1501, DukeMTMC-reid and CUHK03 indicate that our method has robustly achieved state-of-the-art performances and outperformed any existing approaches by a large margin. For example, on Market-1501 dataset in single query mode, we achieve a state-of-the-art result of Rank-1/mAP=96.6%/94.2% after re-ranking.
Tasks	Person Re-Identification
Published	2018-04-04
URL	http://arxiv.org/abs/1804.01438v3
PDF	http://arxiv.org/pdf/1804.01438v3.pdf
PWC	https://paperswithcode.com/paper/learning-discriminative-features-with
Repo	https://github.com/LcenArthas/Kaggle-Humpback-Whale-Identification
Framework	pytorch

Evaluating surgical skills from kinematic data using convolutional neural networks


Title	Evaluating surgical skills from kinematic data using convolutional neural networks
Authors	Hassan Ismail Fawaz, Germain Forestier, Jonathan Weber, Lhassane Idoumghar, Pierre-Alain Muller
Abstract	The need for automatic surgical skills assessment is increasing, especially because manual feedback from senior surgeons observing junior surgeons is prone to subjectivity and time consuming. Thus, automating surgical skills evaluation is a very important step towards improving surgical practice. In this paper, we designed a Convolutional Neural Network (CNN) to evaluate surgeon skills by extracting patterns in the surgeon motions performed in robotic surgery. The proposed method is validated on the JIGSAWS dataset and achieved very competitive results with 100% accuracy on the suturing and needle passing tasks. While we leveraged from the CNNs efficiency, we also managed to mitigate its black-box effect using class activation map. This feature allows our method to automatically highlight which parts of the surgical task influenced the skill prediction and can be used to explain the classification and to provide personalized feedback to the trainee.
Tasks	Surgical Skills Evaluation
Published	2018-06-07
URL	http://arxiv.org/abs/1806.02750v1
PDF	http://arxiv.org/pdf/1806.02750v1.pdf
PWC	https://paperswithcode.com/paper/evaluating-surgical-skills-from-kinematic
Repo	https://github.com/hfawaz/miccai18
Framework	tf

Training Classifiers with Natural Language Explanations


Title	Training Classifiers with Natural Language Explanations
Authors	Braden Hancock, Paroma Varma, Stephanie Wang, Martin Bringmann, Percy Liang, Christopher Ré
Abstract	Training accurate classifiers requires many labels, but each label provides only limited information (one bit for binary classification). In this work, we propose BabbleLabble, a framework for training classifiers in which an annotator provides a natural language explanation for each labeling decision. A semantic parser converts these explanations into programmatic labeling functions that generate noisy labels for an arbitrary amount of unlabeled data, which is used to train a classifier. On three relation extraction tasks, we find that users are able to train classifiers with comparable F1 scores from 5-100$\times$ faster by providing explanations instead of just labels. Furthermore, given the inherent imperfection of labeling functions, we find that a simple rule-based semantic parser suffices.
Tasks	Relation Extraction
Published	2018-05-10
URL	http://arxiv.org/abs/1805.03818v4
PDF	http://arxiv.org/pdf/1805.03818v4.pdf
PWC	https://paperswithcode.com/paper/training-classifiers-with-natural-language
Repo	https://github.com/HazyResearch/babble
Framework	none

Provable Subspace Tracking from Missing Data and Matrix Completion


Title	Provable Subspace Tracking from Missing Data and Matrix Completion
Authors	Praneeth Narayanamurthy, Vahid Daneshpajooh, Namrata Vaswani
Abstract	We study the problem of subspace tracking in the presence of missing data (ST-miss). In recent work, we studied a related problem called robust ST. In this work, we show that a simple modification of our robust ST solution also provably solves ST-miss and robust ST-miss. To our knowledge, our result is the first `complete’ guarantee for ST-miss. This means that we can prove that under assumptions on only the algorithm inputs, the output subspace estimates are close to the true data subspaces at all times. Our guarantees hold under mild and easily interpretable assumptions, and allow the underlying subspace to change with time in a piecewise constant fashion. In contrast, all existing guarantees for ST are partial results and assume a fixed unknown subspace. Extensive numerical experiments are shown to back up our theoretical claims. Finally, our solution can be interpreted as a provably correct mini-batch and memory-efficient solution to low-rank Matrix Completion (MC). \|
Tasks	Low-Rank Matrix Completion, Matrix Completion
Published	2018-10-06
URL	https://arxiv.org/abs/1810.03051v2
PDF	https://arxiv.org/pdf/1810.03051v2.pdf
PWC	https://paperswithcode.com/paper/subspace-tracking-from-missing-and-outlier
Repo	https://github.com/vdaneshpajooh/NORST-rmc
Framework	none

Discriminative Learning of Latent Features for Zero-Shot Recognition


Title	Discriminative Learning of Latent Features for Zero-Shot Recognition
Authors	Yan Li, Junge Zhang, Jianguo Zhang, Kaiqi Huang
Abstract	Zero-shot learning (ZSL) aims to recognize unseen image categories by learning an embedding space between image and semantic representations. For years, among existing works, it has been the center task to learn the proper mapping matrices aligning the visual and semantic space, whilst the importance to learn discriminative representations for ZSL is ignored. In this work, we retrospect existing methods and demonstrate the necessity to learn discriminative representations for both visual and semantic instances of ZSL. We propose an end-to-end network that is capable of 1) automatically discovering discriminative regions by a zoom network; and 2) learning discriminative semantic representations in an augmented space introduced for both user-defined and latent attributes. Our proposed method is tested extensively on two challenging ZSL datasets, and the experiment results show that the proposed method significantly outperforms state-of-the-art methods.
Tasks	Zero-Shot Learning
Published	2018-03-18
URL	http://arxiv.org/abs/1803.06731v1
PDF	http://arxiv.org/pdf/1803.06731v1.pdf
PWC	https://paperswithcode.com/paper/discriminative-learning-of-latent-features
Repo	https://github.com/MARMOTatZJU/ZSLPR-TIANCHI
Framework	pytorch

Predicting property damage from tornadoes with zero-inflated neural networks


Title	Predicting property damage from tornadoes with zero-inflated neural networks
Authors	Jeremy Diaz, Maxwell Joseph
Abstract	Tornadoes are the most violent of all atmospheric storms. In a typical year, the United States experiences hundreds of tornadoes with associated damages on the order of one billion dollars. Community preparation and resilience would benefit from accurate predictions of these economic losses, particularly as populations in tornado-prone areas increase in density and extent. Here, we use a zero-inflated modeling approach and artificial neural networks to predict tornado-induced property damage using publicly available data. We developed a neural network that predicts whether a tornado will cause property damage (out-of-sample accuracy = 0.821 and area under the receiver operating characteristic curve, AUROC, = 0.872). Conditional on a tornado causing damage, another neural network predicts the amount of damage (out-of-sample mean squared error = 0.0918 and R2 = 0.432). When used together, these two models function as a zero-inflated log-normal regression with hidden layers. From the best-performing models, we provide static and interactive gridded maps of monthly predicted probabilities of damage and property damages for the year 2019. Two primary weaknesses include (1) model fitting requires log-scale data which leads to large natural-scale residuals and (2) beginning tornado coordinates were utilized rather than tornado paths. Ultimately, this is the first known study to directly model tornado-induced property damages, and all data, code, and tools are publicly available. The predictive capacity of this model along with an interactive interface may provide an opportunity for science-informed tornado disaster planning.
Tasks
Published	2018-07-10
URL	https://arxiv.org/abs/1807.03456v3
PDF	https://arxiv.org/pdf/1807.03456v3.pdf
PWC	https://paperswithcode.com/paper/predicting-property-damage-from-tornadoes
Repo	https://github.com/YuqiYuqiMa/NatrualDisasterPredict
Framework	none

MOABB: Trustworthy algorithm benchmarking for BCIs


Title	MOABB: Trustworthy algorithm benchmarking for BCIs
Authors	Vinay Jayaram, Alexandre Barachant
Abstract	BCI algorithm development has long been hampered by two major issues: small sample sets and a lack of reproducibility. We offer a solution to both of these problems via a software suite that streamlines both the issues of finding and preprocessing data in a reliable manner, as well as that of using a consistent interface for machine learning methods. By building on recent advances in software for signal analysis implemented in the MNE toolkit, and the unified framework for machine learning offered by the scikit-learn project, we offer a system that can improve BCI algorithm development. This system is fully open-source under the BSD licence and available at https://github.com/NeuroTechX/moabb. To validate our efforts, we analyze a set of state-of-the-art decoding algorithms across 12 open access datasets, with over 250 subjects. Our analysis confirms that different datasets can result in very different results for identical processing pipelines, highlighting the need for trustworthy algorithm benchmarking in the field of BCIs, and further that many previously validated methods do not hold up when applied across different datasets, which has wide-reaching implications for practical BCIs.
Tasks
Published	2018-05-16
URL	http://arxiv.org/abs/1805.06427v1
PDF	http://arxiv.org/pdf/1805.06427v1.pdf
PWC	https://paperswithcode.com/paper/moabb-trustworthy-algorithm-benchmarking-for
Repo	https://github.com/Aadhvin02/https-github.com-meagmohit-EEG-Datasets
Framework	none

Iterative Deep Learning for Road Topology Extraction


Title	Iterative Deep Learning for Road Topology Extraction
Authors	Carles Ventura, Jordi Pont-Tuset, Sergi Caelles, Kevis-Kokitsi Maninis, Luc Van Gool
Abstract	This paper tackles the task of estimating the topology of road networks from aerial images. Building on top of a global model that performs a dense semantical classification of the pixels of the image, we design a Convolutional Neural Network (CNN) that predicts the local connectivity among the central pixel of an input patch and its border points. By iterating this local connectivity we sweep the whole image and infer the global topology of the road network, inspired by a human delineating a complex network with the tip of their finger. We perform an extensive and comprehensive qualitative and quantitative evaluation on the road network estimation task, and show that our method also generalizes well when moving to networks of retinal vessels.
Tasks
Published	2018-08-28
URL	http://arxiv.org/abs/1808.09814v1
PDF	http://arxiv.org/pdf/1808.09814v1.pdf
PWC	https://paperswithcode.com/paper/iterative-deep-learning-for-road-topology
Repo	https://github.com/carlesventura/iterative-deep-learning
Framework	pytorch

Greedy Layerwise Learning Can Scale to ImageNet


Title	Greedy Layerwise Learning Can Scale to ImageNet
Authors	Eugene Belilovsky, Michael Eickenberg, Edouard Oyallon
Abstract	Shallow supervised 1-hidden layer neural networks have a number of favorable properties that make them easier to interpret, analyze, and optimize than their deep counterparts, but lack their representational power. Here we use 1-hidden layer learning problems to sequentially build deep networks layer by layer, which can inherit properties from shallow networks. Contrary to previous approaches using shallow networks, we focus on problems where deep learning is reported as critical for success. We thus study CNNs on image classification tasks using the large-scale ImageNet dataset and the CIFAR-10 dataset. Using a simple set of ideas for architecture and training we find that solving sequential 1-hidden-layer auxiliary problems lead to a CNN that exceeds AlexNet performance on ImageNet. Extending this training methodology to construct individual layers by solving 2-and-3-hidden layer auxiliary problems, we obtain an 11-layer network that exceeds several members of the VGG model family on ImageNet, and can train a VGG-11 model to the same accuracy as end-to-end learning. To our knowledge, this is the first competitive alternative to end-to-end training of CNNs that can scale to ImageNet. We illustrate several interesting properties of these models theoretically and conduct a range of experiments to study the properties this training induces on the intermediate layers.
Tasks	Image Classification
Published	2018-12-29
URL	http://arxiv.org/abs/1812.11446v3
PDF	http://arxiv.org/pdf/1812.11446v3.pdf
PWC	https://paperswithcode.com/paper/greedy-layerwise-learning-can-scale-to
Repo	https://github.com/eugenium/layerCNN
Framework	pytorch

Improving GAN with neighbors embedding and gradient matching


Title	Improving GAN with neighbors embedding and gradient matching
Authors	Ngoc-Trung Tran, Tuan-Anh Bui, Ngai-Man Cheung
Abstract	We propose two new techniques for training Generative Adversarial Networks (GANs). Our objectives are to alleviate mode collapse in GAN and improve the quality of the generated samples. First, we propose neighbor embedding, a manifold learning-based regularization to explicitly retain local structures of latent samples in the generated samples. This prevents generator from producing nearly identical data samples from different latent samples, and reduces mode collapse. We propose an inverse t-SNE regularizer to achieve this. Second, we propose a new technique, gradient matching, to align the distributions of the generated samples and the real samples. As it is challenging to work with high-dimensional sample distributions, we propose to align these distributions through the scalar discriminator scores. We constrain the difference between the discriminator scores of the real samples and generated ones. We further constrain the difference between the gradients of these discriminator scores. We derive these constraints from Taylor approximations of the discriminator function. We perform experiments to demonstrate that our proposed techniques are computationally simple and easy to be incorporated in existing systems. When Gradient matching and Neighbour embedding are applied together, our GN-GAN achieves outstanding results on 1D/2D synthetic, CIFAR-10 and STL-10 datasets, e.g. FID score of $30.80$ for the STL-10 dataset. Our code is available at: https://github.com/tntrung/gan
Tasks
Published	2018-11-04
URL	http://arxiv.org/abs/1811.01333v1
PDF	http://arxiv.org/pdf/1811.01333v1.pdf
PWC	https://paperswithcode.com/paper/improving-gan-with-neighbors-embedding-and
Repo	https://github.com/tntrung/gan
Framework	tf

Summarizing Opinions: Aspect Extraction Meets Sentiment Prediction and They Are Both Weakly Supervised


Title	Summarizing Opinions: Aspect Extraction Meets Sentiment Prediction and They Are Both Weakly Supervised
Authors	Stefanos Angelidis, Mirella Lapata
Abstract	We present a neural framework for opinion summarization from online product reviews which is knowledge-lean and only requires light supervision (e.g., in the form of product domain labels and user-provided ratings). Our method combines two weakly supervised components to identify salient opinions and form extractive summaries from multiple reviews: an aspect extractor trained under a multi-task objective, and a sentiment predictor based on multiple instance learning. We introduce an opinion summarization dataset that includes a training set of product reviews from six diverse domains and human-annotated development and test sets with gold standard aspect annotations, salience labels, and opinion summaries. Automatic evaluation shows significant improvements over baselines, and a large-scale study indicates that our opinion summaries are preferred by human judges according to multiple criteria.
Tasks	Aspect Extraction, Multiple Instance Learning
Published	2018-08-27
URL	http://arxiv.org/abs/1808.08858v1
PDF	http://arxiv.org/pdf/1808.08858v1.pdf
PWC	https://paperswithcode.com/paper/summarizing-opinions-aspect-extraction-meets
Repo	https://github.com/stangelid/oposum
Framework	pytorch

Adaptive pooling operators for weakly labeled sound event detection


Title	Adaptive pooling operators for weakly labeled sound event detection
Authors	Brian McFee, Justin Salamon, Juan Pablo Bello
Abstract	Sound event detection (SED) methods are tasked with labeling segments of audio recordings by the presence of active sound sources. SED is typically posed as a supervised machine learning problem, requiring strong annotations for the presence or absence of each sound source at every time instant within the recording. However, strong annotations of this type are both labor- and cost-intensive for human annotators to produce, which limits the practical scalability of SED methods. In this work, we treat SED as a multiple instance learning (MIL) problem, where training labels are static over a short excerpt, indicating the presence or absence of sound sources but not their temporal locality. The models, however, must still produce temporally dynamic predictions, which must be aggregated (pooled) when comparing against static labels during training. To facilitate this aggregation, we develop a family of adaptive pooling operators—referred to as auto-pool—which smoothly interpolate between common pooling operators, such as min-, max-, or average-pooling, and automatically adapt to the characteristics of the sound sources in question. We evaluate the proposed pooling operators on three datasets, and demonstrate that in each case, the proposed methods outperform non-adaptive pooling operators for static prediction, and nearly match the performance of models trained with strong, dynamic annotations. The proposed method is evaluated in conjunction with convolutional neural networks, but can be readily applied to any differentiable model for time-series label prediction.
Tasks	Multiple Instance Learning, Sound Event Detection, Time Series
Published	2018-04-26
URL	http://arxiv.org/abs/1804.10070v2
PDF	http://arxiv.org/pdf/1804.10070v2.pdf
PWC	https://paperswithcode.com/paper/adaptive-pooling-operators-for-weakly-labeled
Repo	https://github.com/marl/autopool
Framework	none