January 27, 2020

2761 words 13 mins read

Paper Group ANR 1165

Batched Stochastic Bayesian Optimization via Combinatorial Constraints Design. Deep Sparse Band Selection for Hyperspectral Face Recognition. Visualizing and Understanding Generative Adversarial Networks (Extended Abstract). Interpretable BoW Networks for Adversarial Example Detection. Rescaling and other forms of unsupervised preprocessing introdu …

Batched Stochastic Bayesian Optimization via Combinatorial Constraints Design


Title	Batched Stochastic Bayesian Optimization via Combinatorial Constraints Design
Authors	Kevin K. Yang, Yuxin Chen, Alycia Lee, Yisong Yue
Abstract	In many high-throughput experimental design settings, such as those common in biochemical engineering, batched queries are more cost effective than one-by-one sequential queries. Furthermore, it is often not possible to directly choose items to query. Instead, the experimenter specifies a set of constraints that generates a library of possible items, which are then selected stochastically. Motivated by these considerations, we investigate \emph{Batched Stochastic Bayesian Optimization} (BSBO), a novel Bayesian optimization scheme for choosing the constraints in order to guide exploration towards items with greater utility. We focus on \emph{site-saturation mutagenesis}, a prototypical setting of BSBO in biochemical engineering, and propose a natural objective function for this problem. Importantly, we show that our objective function can be efficiently decomposed as a difference of submodular functions (DS), which allows us to employ DS optimization tools to greedily identify sets of constraints that increase the likelihood of finding items with high utility. Our experimental results show that our algorithm outperforms common heuristics on both synthetic and two real protein datasets.
Tasks
Published	2019-04-17
URL	http://arxiv.org/abs/1904.08102v1
PDF	http://arxiv.org/pdf/1904.08102v1.pdf
PWC	https://paperswithcode.com/paper/batched-stochastic-bayesian-optimization-via
Repo
Framework

Deep Sparse Band Selection for Hyperspectral Face Recognition


Title	Deep Sparse Band Selection for Hyperspectral Face Recognition
Authors	Fariborz Taherkhani, Jeremy Dawson, Nasser M. Nasrabadi
Abstract	Hyperspectral imaging systems collect and process information from specific wavelengths across the electromagnetic spectrum. The fusion of multi-spectral bands in the visible spectrum has been exploited to improve face recognition performance over all the conventional broad band face images. In this book chapter, we propose a new Convolutional Neural Network (CNN) framework which adopts a structural sparsity learning technique to select the optimal spectral bands to obtain the best face recognition performance over all of the spectral bands. Specifically, in this method, images from all bands are fed to a CNN, and the convolutional filters in the first layer of the CNN are then regularized by employing a group Lasso algorithm to zero out the redundant bands during the training of the network. Contrary to other methods which usually select the useful bands manually or in a greedy fashion, our method selects the optimal spectral bands automatically to achieve the best face recognition performance over all spectral bands. Moreover, experimental results demonstrate that our method outperforms state of the art band selection methods for face recognition on several publicly-available hyperspectral face image datasets.
Tasks	Face Recognition
Published	2019-08-15
URL	https://arxiv.org/abs/1908.09630v1
PDF	https://arxiv.org/pdf/1908.09630v1.pdf
PWC	https://paperswithcode.com/paper/deep-sparse-band-selection-for-hyperspectral
Repo
Framework

Visualizing and Understanding Generative Adversarial Networks (Extended Abstract)


Title	Visualizing and Understanding Generative Adversarial Networks (Extended Abstract)
Authors	David Bau, Jun-Yan Zhu, Hendrik Strobelt, Bolei Zhou, Joshua B. Tenenbaum, William T. Freeman, Antonio Torralba
Abstract	Generative Adversarial Networks (GANs) have achieved impressive results for many real-world applications. As an active research topic, many GAN variants have emerged with improvements in sample quality and training stability. However, visualization and understanding of GANs is largely missing. How does a GAN represent our visual world internally? What causes the artifacts in GAN results? How do architectural choices affect GAN learning? Answering such questions could enable us to develop new insights and better models. In this work, we present an analytic framework to visualize and understand GANs at the unit-, object-, and scene-level. We first identify a group of interpretable units that are closely related to concepts with a segmentation-based network dissection method. We quantify the causal effect of interpretable units by measuring the ability of interventions to control objects in the output. Finally, we examine the contextual relationship between these units and their surrounding by inserting the discovered concepts into new images. We show several practical applications enabled by our framework, from comparing internal representations across different layers, models, and datasets, to improving GANs by locating and removing artifact-causing units, to interactively manipulating objects in the scene. We will open source our interactive tools to help researchers and practitioners better understand their models.
Tasks
Published	2019-01-29
URL	http://arxiv.org/abs/1901.09887v1
PDF	http://arxiv.org/pdf/1901.09887v1.pdf
PWC	https://paperswithcode.com/paper/visualizing-and-understanding-generative
Repo
Framework

Interpretable BoW Networks for Adversarial Example Detection


Title	Interpretable BoW Networks for Adversarial Example Detection
Authors	Krishna Kanth Nakka, Mathieu Salzmann
Abstract	The standard approach to providing interpretability to deep convolutional neural networks (CNNs) consists of visualizing either their feature maps, or the image regions that contribute the most to the prediction. In this paper, we introduce an alternative strategy to interpret the results of a CNN. To this end, we leverage a Bag of visual Word representation within the network and associate a visual and semantic meaning to the corresponding codebook elements via the use of a generative adversarial network. The reason behind the prediction for a new sample can then be interpreted by looking at the visual representation of the most highly activated codeword. We then propose to exploit our interpretable BoW networks for adversarial example detection. To this end, we build upon the intuition that, while adversarial samples look very similar to real images, to produce incorrect predictions, they should activate codewords with a significantly different visual representation. We therefore cast the adversarial example detection problem as that of comparing the input image with the most highly activated visual codeword. As evidenced by our experiments, this allows us to outperform the state-of-the-art adversarial example detection methods on standard benchmarks, independently of the attack strategy.
Tasks
Published	2019-01-08
URL	http://arxiv.org/abs/1901.02229v1
PDF	http://arxiv.org/pdf/1901.02229v1.pdf
PWC	https://paperswithcode.com/paper/interpretable-bow-networks-for-adversarial
Repo
Framework

Rescaling and other forms of unsupervised preprocessing introduce bias into cross-validation


Title	Rescaling and other forms of unsupervised preprocessing introduce bias into cross-validation
Authors	Amit Moscovich, Saharon Rosset
Abstract	Cross-validation is the de-facto standard for model evaluation and selection. In proper use, it provides an unbiased estimate of a model’s predictive performance. However, data sets often undergo various forms of preprocessing, such as mean-centering, rescaling, dimensionality reduction and outlier removal, prior to cross-validation. It is widely believed that such preprocessing stages, if done in an unsupervised manner that does not involve the class labels or response values, has no effect on the validity of cross-validation. In this paper, we show that this belief is not true. Preliminary unsupervised preprocessing can introduce either a positive or negative bias into the estimates of model performance. Thus, it may lead to invalid inference and sub-optimal choices of model parameters. In light of this, the scientific community should re-examine the use of preprocessing prior to cross-validation across the various application domains. By default, the parameters of all data-dependent transformations should be learned only from the training samples.
Tasks	Dimensionality Reduction, Model Selection
Published	2019-01-25
URL	https://arxiv.org/abs/1901.08974v2
PDF	https://arxiv.org/pdf/1901.08974v2.pdf
PWC	https://paperswithcode.com/paper/rescaling-and-other-forms-of-unsupervised
Repo
Framework

Learning from Synthetic Animals


Title	Learning from Synthetic Animals
Authors	Jiteng Mu, Weichao Qiu, Gregory Hager, Alan Yuille
Abstract	Despite great success in human parsing, progress for parsing other deformable articulated objects, like animals, is still limited by the lack of labeled data. In this paper, we use synthetic images and ground truth generated from CAD animal models to address this challenge. To bridge the gap between real and synthetic images, we propose a novel consistency-constrained semi-supervised learning method (CC-SSL). Our method leverages both spatial and temporal consistencies, to bootstrap weak models trained on synthetic data with unlabeled real images. We demonstrate the effectiveness of our method on highly deformable animals, such as horses and tigers. Without using any real image label, our method allows for accurate keypoints prediction on real images. Moreover, we quantitatively show that models using synthetic data achieve better generalization performance than models trained on real images across different domains in the Visual Domain Adaptation Challenge dataset. Our synthetic dataset contains 10+ animals with diverse poses and rich ground truth, which enables us to use the multi-task learning strategy to further boost models’ performance.
Tasks	Domain Adaptation, Human Parsing, Multi-Task Learning
Published	2019-12-17
URL	https://arxiv.org/abs/1912.08265v1
PDF	https://arxiv.org/pdf/1912.08265v1.pdf
PWC	https://paperswithcode.com/paper/learning-from-synthetic-animals
Repo
Framework

Active Learning for High-Dimensional Binary Features


Title	Active Learning for High-Dimensional Binary Features
Authors	Ali Vahdat, Mouloud Belbahri, Vahid Partovi Nia
Abstract	Erbium-doped fiber amplifier (EDFA) is an optical amplifier/repeater device used to boost the intensity of optical signals being carried through a fiber optic communication system. A highly accurate EDFA model is important because of its crucial role in optical network management and optimization. The input channels of an EDFA device are treated as either on or off, hence the input features are binary. Labeled training data is very expensive to collect for EDFA devices, therefore we devise an active learning strategy suitable for binary variables to overcome this issue. We propose to take advantage of sparse linear models to simplify the predictive model. This approach simultaneously improves prediction and accelerates active learning query generation. We show the performance of our proposed active learning strategies on simulated data and real EDFA data.
Tasks	Active Learning
Published	2019-02-05
URL	https://arxiv.org/abs/1902.01923v2
PDF	https://arxiv.org/pdf/1902.01923v2.pdf
PWC	https://paperswithcode.com/paper/active-learning-for-high-dimensional-binary
Repo
Framework

Towards Fair Deep Clustering With Multi-State Protected Variables


Title	Towards Fair Deep Clustering With Multi-State Protected Variables
Authors	Bokun Wang, Ian Davidson
Abstract	Fair clustering under the disparate impact doctrine requires that population of each protected group should be approximately equal in every cluster. Previous work investigated a difficult-to-scale pre-processing step for $k$-center and $k$-median style algorithms for the special case of this problem when the number of protected groups is two. In this work, we consider a more general and practical setting where there can be many protected groups. To this end, we propose Deep Fair Clustering, which learns a discriminative but fair cluster assignment function. The experimental results on three public datasets with different types of protected attribute show that our approach can steadily improve the degree of fairness while only having minor loss in terms of clustering quality.
Tasks
Published	2019-01-29
URL	http://arxiv.org/abs/1901.10053v1
PDF	http://arxiv.org/pdf/1901.10053v1.pdf
PWC	https://paperswithcode.com/paper/towards-fair-deep-clustering-with-multi-state
Repo
Framework

BSDAR: Beam Search Decoding with Attention Reward in Neural Keyphrase Generation


Title	BSDAR: Beam Search Decoding with Attention Reward in Neural Keyphrase Generation
Authors	Iftitahu Ni’mah, Vlado Menkovski, Mykola Pechenizkiy
Abstract	This study mainly investigates two decoding problems in neural keyphrase generation: sequence length bias and beam diversity. We introduce an extension of beam search inference based on word-level and n-gram level attention score to adjust and constrain Seq2Seq prediction at test time. Results show that our proposed solution can overcome the algorithm bias to shorter and nearly identical sequences, resulting in a significant improvement of the decoding performance on generating keyphrases that are present and absent in source text.
Tasks
Published	2019-09-17
URL	https://arxiv.org/abs/1909.09485v1
PDF	https://arxiv.org/pdf/1909.09485v1.pdf
PWC	https://paperswithcode.com/paper/bsdar-beam-search-decoding-with-attention
Repo
Framework

Exploiting Parallel Audio Recordings to Enforce Device Invariance in CNN-based Acoustic Scene Classification


Title	Exploiting Parallel Audio Recordings to Enforce Device Invariance in CNN-based Acoustic Scene Classification
Authors	Paul Primus, Hamid Eghbal-zadeh, David Eitelsebner, Khaled Koutini, Andreas Arzt, Gerhard Widmer
Abstract	Distribution mismatches between the data seen at training and at application time remain a major challenge in all application areas of machine learning. We study this problem in the context of machine listening (Task 1b of the DCASE 2019 Challenge). We propose a novel approach to learn domain-invariant classifiers in an end-to-end fashion by enforcing equal hidden layer representations for domain-parallel samples, i.e. time-aligned recordings from different recording devices. No classification labels are needed for our domain adaptation (DA) method, which makes the data collection process cheaper.
Tasks	Acoustic Scene Classification, Domain Adaptation, Scene Classification
Published	2019-09-04
URL	https://arxiv.org/abs/1909.02869v1
PDF	https://arxiv.org/pdf/1909.02869v1.pdf
PWC	https://paperswithcode.com/paper/exploiting-parallel-audio-recordings-to
Repo
Framework

Discovering Invariances in Healthcare Neural Networks


Title	Discovering Invariances in Healthcare Neural Networks
Authors	Mohammad Taha Bahadori, Layne C. Price
Abstract	We study the invariance characteristics of pre-trained predictive models by empirically learning transformations on the input that leave the prediction function approximately unchanged. To learn invariant transformations, we minimize the Wasserstein distance between the predictive distribution conditioned on the data instances and the predictive distribution conditioned on the transformed data instances. To avoid finding degenerate or perturbative transformations, we add a similarity regularization to discourage similarity between the data and its transformed values. We theoretically analyze the correctness of the algorithm and the structure of the solutions. Applying the proposed technique to clinical time series data, we discover variables that commonly-used LSTM models do not rely on for their prediction, especially when the LSTM is trained to be adversarially robust. We also analyze the invariances of BioBERT on clinical notes and discover words that it is invariant to.
Tasks	Time Series
Published	2019-11-08
URL	https://arxiv.org/abs/1911.03295v3
PDF	https://arxiv.org/pdf/1911.03295v3.pdf
PWC	https://paperswithcode.com/paper/discovering-invariances-in-healthcare-neural
Repo
Framework

A Multilingual Encoding Method for Text Classification and Dialect Identification Using Convolutional Neural Network


Title	A Multilingual Encoding Method for Text Classification and Dialect Identification Using Convolutional Neural Network
Authors	Amr Adel Helmy
Abstract	This thesis presents a language-independent text classification model by introduced two new encoding methods “BUNOW” and “BUNOC” used for feeding the raw text data into a new CNN spatial architecture with vertical and horizontal convolutional process instead of commonly used methods like one hot vector or word representation (i.e. word2vec) with temporal CNN architecture. The proposed model can be classified as hybrid word-character model in its work methodology because it consumes less memory space by using a fewer neural network parameters as in character level representation, in addition to providing much faster computations with fewer network layers depth, as in word level representation. A promising result achieved compared to state of art models in two different morphological benchmarked dataset one for Arabic language and one for English language.
Tasks	Text Classification
Published	2019-03-18
URL	http://arxiv.org/abs/1903.07588v1
PDF	http://arxiv.org/pdf/1903.07588v1.pdf
PWC	https://paperswithcode.com/paper/a-multilingual-encoding-method-for-text
Repo
Framework

How to Make Swarms Open-Ended? Evolving Collective Intelligence Through a Constricted Exploration of Adjacent Possibles


Title	How to Make Swarms Open-Ended? Evolving Collective Intelligence Through a Constricted Exploration of Adjacent Possibles
Authors	Olaf Witkowski, Takashi Ikegami
Abstract	We propose an approach of open-ended evolution via the simulation of swarm dynamics. In nature, swarms possess remarkable properties, which allow many organisms, from swarming bacteria to ants and flocking birds, to form higher-order structures that enhance their behavior as a group. Swarm simulations highlight three important factors to create novelty and diversity: (a) communication generates combinatorial cooperative dynamics, (b) concurrency allows for separation of timescales, and (c) complexity and size increases push the system towards transitions in innovation. We illustrate these three components in a model computing the continuous evolution of a swarm of agents. The results, divided in three distinct applications, show how emergent structures are capable of filtering information through the bottleneck of their memory, to produce meaningful novelty and diversity within their simulated environment.
Tasks
Published	2019-03-19
URL	http://arxiv.org/abs/1903.08228v1
PDF	http://arxiv.org/pdf/1903.08228v1.pdf
PWC	https://paperswithcode.com/paper/how-to-make-swarms-open-ended-evolving
Repo
Framework

Streaming Bayesian Inference for Crowdsourced Classification


Title	Streaming Bayesian Inference for Crowdsourced Classification
Authors	Edoardo Manino, Long Tran-Thanh, Nicholas R. Jennings
Abstract	A key challenge in crowdsourcing is inferring the ground truth from noisy and unreliable data. To do so, existing approaches rely on collecting redundant information from the crowd, and aggregating it with some probabilistic method. However, oftentimes such methods are computationally inefficient, are restricted to some specific settings, or lack theoretical guarantees. In this paper, we revisit the problem of binary classification from crowdsourced data. Specifically we propose Streaming Bayesian Inference for Crowdsourcing (SBIC), a new algorithm that does not suffer from any of these limitations. First, SBIC has low complexity and can be used in a real-time online setting. Second, SBIC has the same accuracy as the best state-of-the-art algorithms in all settings. Third, SBIC has provable asymptotic guarantees both in the online and offline settings.
Tasks	Bayesian Inference
Published	2019-11-13
URL	https://arxiv.org/abs/1911.05712v1
PDF	https://arxiv.org/pdf/1911.05712v1.pdf
PWC	https://paperswithcode.com/paper/streaming-bayesian-inference-for-crowdsourced-1
Repo
Framework

Parameter-Free Locally Differentially Private Stochastic Subgradient Descent


Title	Parameter-Free Locally Differentially Private Stochastic Subgradient Descent
Authors	Kwang-Sung Jun, Francesco Orabona
Abstract	We consider the problem of minimizing a convex risk with stochastic subgradients guaranteeing $\epsilon$-locally differentially private ($\epsilon$-LDP). While it has been shown that stochastic optimization is possible with $\epsilon$-LDP via the standard SGD (Song et al., 2013), its convergence rate largely depends on the learning rate, which must be tuned via repeated runs. Further, tuning is detrimental to privacy loss since it significantly increases the number of gradient requests. In this work, we propose BANCO (Betting Algorithm for Noisy COins), the first $\epsilon$-LDP SGD algorithm that essentially matches the convergence rate of the tuned SGD without any learning rate parameter, reducing privacy loss and saving privacy budget.
Tasks	Stochastic Optimization
Published	2019-11-21
URL	https://arxiv.org/abs/1911.09564v1
PDF	https://arxiv.org/pdf/1911.09564v1.pdf
PWC	https://paperswithcode.com/paper/parameter-free-locally-differentially-private
Repo
Framework