October 21, 2019

2833 words 14 mins read

Paper Group AWR 62

Variable Prioritization in Nonlinear Black Box Methods: A Genetic Association Case Study. Fluctuation-dissipation relations for stochastic gradient descent. Learning Integral Representations of Gaussian Processes. HitNet: a neural network with capsules embedded in a Hit-or-Miss layer, extended with hybrid data augmentation and ghost capsules. Longi …

Variable Prioritization in Nonlinear Black Box Methods: A Genetic Association Case Study


Title	Variable Prioritization in Nonlinear Black Box Methods: A Genetic Association Case Study
Authors	Lorin Crawford, Seth R. Flaxman, Daniel E. Runcie, Mike West
Abstract	The central aim in this paper is to address variable selection questions in nonlinear and nonparametric regression. Motivated by statistical genetics, where nonlinear interactions are of particular interest, we introduce a novel and interpretable way to summarize the relative importance of predictor variables. Methodologically, we develop the “RelATive cEntrality” (RATE) measure to prioritize candidate genetic variants that are not just marginally important, but whose associations also stem from significant covarying relationships with other variants in the data. We illustrate RATE through Bayesian Gaussian process regression, but the methodological innovations apply to other “black box” methods. It is known that nonlinear models often exhibit greater predictive accuracy than linear models, particularly for phenotypes generated by complex genetic architectures. With detailed simulations and two real data association mapping studies, we show that applying RATE enables an explanation for this improved performance.
Tasks
Published	2018-01-22
URL	http://arxiv.org/abs/1801.07318v3
PDF	http://arxiv.org/pdf/1801.07318v3.pdf
PWC	https://paperswithcode.com/paper/variable-prioritization-in-nonlinear-black
Repo	https://github.com/lorinanthony/RATE
Framework	tf

Fluctuation-dissipation relations for stochastic gradient descent


Title	Fluctuation-dissipation relations for stochastic gradient descent
Authors	Sho Yaida
Abstract	The notion of the stationary equilibrium ensemble has played a central role in statistical mechanics. In machine learning as well, training serves as generalized equilibration that drives the probability distribution of model parameters toward stationarity. Here, we derive stationary fluctuation-dissipation relations that link measurable quantities and hyperparameters in the stochastic gradient descent algorithm. These relations hold exactly for any stationary state and can in particular be used to adaptively set training schedule. We can further use the relations to efficiently extract information pertaining to a loss-function landscape such as the magnitudes of its Hessian and anharmonicity. Our claims are empirically verified.
Tasks
Published	2018-09-28
URL	http://arxiv.org/abs/1810.00004v2
PDF	http://arxiv.org/pdf/1810.00004v2.pdf
PWC	https://paperswithcode.com/paper/fluctuation-dissipation-relations-for
Repo	https://github.com/cybertronai/pytorch-fd
Framework	pytorch

Learning Integral Representations of Gaussian Processes


Title	Learning Integral Representations of Gaussian Processes
Authors	Zilong Tan, Sayan Mukherjee
Abstract	We propose a representation of Gaussian processes (GPs) based on powers of the integral operator defined by a kernel function, we call these stochastic processes integral Gaussian processes (IGPs). Sample paths from IGPs are functions contained within the reproducing kernel Hilbert space (RKHS) defined by the kernel function, in contrast sample paths from the standard GP are not functions within the RKHS. We develop computationally efficient non-parametric regression models based on IGPs. The main innovation in our regression algorithm is the construction of a low dimensional subspace that captures the information most relevant to explaining variation in the response. We use ideas from supervised dimension reduction to compute this subspace. The result of using the construction we propose involves significant improvements in the computational complexity of estimating kernel hyper-parameters as well as reducing the prediction variance.
Tasks	Dimensionality Reduction, Gaussian Processes
Published	2018-02-21
URL	http://arxiv.org/abs/1802.07528v4
PDF	http://arxiv.org/pdf/1802.07528v4.pdf
PWC	https://paperswithcode.com/paper/learning-integral-representations-of-gaussian
Repo	https://github.com/ZilongTan/sigp
Framework	none

HitNet: a neural network with capsules embedded in a Hit-or-Miss layer, extended with hybrid data augmentation and ghost capsules


Title	HitNet: a neural network with capsules embedded in a Hit-or-Miss layer, extended with hybrid data augmentation and ghost capsules
Authors	Adrien Deliège, Anthony Cioppa, Marc Van Droogenbroeck
Abstract	Neural networks designed for the task of classification have become a commodity in recent years. Many works target the development of better networks, which results in a complexification of their architectures with more layers, multiple sub-networks, or even the combination of multiple classifiers. In this paper, we show how to redesign a simple network to reach excellent performances, which are better than the results reproduced with CapsNet on several datasets, by replacing a layer with a Hit-or-Miss layer. This layer contains activated vectors, called capsules, that we train to hit or miss a central capsule by tailoring a specific centripetal loss function. We also show how our network, named HitNet, is capable of synthesizing a representative sample of the images of a given class by including a reconstruction network. This possibility allows to develop a data augmentation step combining information from the data space and the feature space, resulting in a hybrid data augmentation process. In addition, we introduce the possibility for HitNet, to adopt an alternative to the true target when needed by using the new concept of ghost capsules, which is used here to detect potentially mislabeled images in the training data.
Tasks	Data Augmentation
Published	2018-06-18
URL	http://arxiv.org/abs/1806.06519v1
PDF	http://arxiv.org/pdf/1806.06519v1.pdf
PWC	https://paperswithcode.com/paper/hitnet-a-neural-network-with-capsules
Repo	https://github.com/bakirillov/capsules
Framework	pytorch

Longitudinal detection of radiological abnormalities with time-modulated LSTM


Title	Longitudinal detection of radiological abnormalities with time-modulated LSTM
Authors	Ruggiero Santeramo, Samuel Withey, Giovanni Montana
Abstract	Convolutional neural networks (CNNs) have been successfully employed in recent years for the detection of radiological abnormalities in medical images such as plain x-rays. To date, most studies use CNNs on individual examinations in isolation and discard previously available clinical information. In this study we set out to explore whether Long-Short-Term-Memory networks (LSTMs) can be used to improve classification performance when modelling the entire sequence of radiographs that may be available for a given patient, including their reports. A limitation of traditional LSTMs, though, is that they implicitly assume equally-spaced observations, whereas the radiological exams are event-based, and therefore irregularly sampled. Using both a simulated dataset and a large-scale chest x-ray dataset, we demonstrate that a simple modification of the LSTM architecture, which explicitly takes into account the time lag between consecutive observations, can boost classification performance. Our empirical results demonstrate improved detection of commonly reported abnormalities on chest x-rays such as cardiomegaly, consolidation, pleural effusion and hiatus hernia.
Tasks
Published	2018-07-16
URL	http://arxiv.org/abs/1807.06144v1
PDF	http://arxiv.org/pdf/1807.06144v1.pdf
PWC	https://paperswithcode.com/paper/longitudinal-detection-of-radiological
Repo	https://github.com/WMGDataScience/tLSTM
Framework	pytorch

Explaining Image Classifiers by Counterfactual Generation


Title	Explaining Image Classifiers by Counterfactual Generation
Authors	Chun-Hao Chang, Elliot Creager, Anna Goldenberg, David Duvenaud
Abstract	When an image classifier makes a prediction, which parts of the image are relevant and why? We can rephrase this question to ask: which parts of the image, if they were not seen by the classifier, would most change its decision? Producing an answer requires marginalizing over images that could have been seen but weren’t. We can sample plausible image in-fills by conditioning a generative model on the rest of the image. We then optimize to find the image regions that most change the classifier’s decision after in-fill. Our approach contrasts with ad-hoc in-filling approaches, such as blurring or injecting noise, which generate inputs far from the data distribution, and ignore informative relationships between different parts of the image. Our method produces more compact and relevant saliency maps, with fewer artifacts compared to previous methods.
Tasks	Image Classification
Published	2018-07-20
URL	http://arxiv.org/abs/1807.08024v3
PDF	http://arxiv.org/pdf/1807.08024v3.pdf
PWC	https://paperswithcode.com/paper/explaining-image-classifiers-by
Repo	https://github.com/zzzace2000/FIDO-saliency
Framework	pytorch

Whispered-to-voiced Alaryngeal Speech Conversion with Generative Adversarial Networks


Title	Whispered-to-voiced Alaryngeal Speech Conversion with Generative Adversarial Networks
Authors	Santiago Pascual, Antonio Bonafonte, Joan Serrà, Jose A. Gonzalez
Abstract	Most methods of voice restoration for patients suffering from aphonia either produce whispered or monotone speech. Apart from intelligibility, this type of speech lacks expressiveness and naturalness due to the absence of pitch (whispered speech) or artificial generation of it (monotone speech). Existing techniques to restore prosodic information typically combine a vocoder, which parameterises the speech signal, with machine learning techniques that predict prosodic information. In contrast, this paper describes an end-to-end neural approach for estimating a fully-voiced speech waveform from whispered alaryngeal speech. By adapting our previous work in speech enhancement with generative adversarial networks, we develop a speaker-dependent model to perform whispered-to-voiced speech conversion. Preliminary qualitative results show effectiveness in re-generating voiced speech, with the creation of realistic pitch contours.
Tasks	Speech Enhancement
Published	2018-08-31
URL	http://arxiv.org/abs/1808.10687v2
PDF	http://arxiv.org/pdf/1808.10687v2.pdf
PWC	https://paperswithcode.com/paper/whispered-to-voiced-alaryngeal-speech
Repo	https://github.com/rickyHong/segan-pytorch-repl
Framework	pytorch

On Generation of Adversarial Examples using Convex Programming


Title	On Generation of Adversarial Examples using Convex Programming
Authors	Emilio Rafael Balda, Arash Behboodi, Rudolf Mathar
Abstract	It has been observed that deep learning architectures tend to make erroneous decisions with high reliability for particularly designed adversarial instances. In this work, we show that the perturbation analysis of these architectures provides a framework for generating adversarial instances by convex programming which, for classification tasks, is able to recover variants of existing non-adaptive adversarial methods. The proposed framework can be used for the design of adversarial noise under various desirable constraints and different types of networks. Moreover, this framework is capable of explaining various existing adversarial methods and can be used to derive new algorithms as well. We make use of these results to obtain novel algorithms. The experiments show the competitive performance of the obtained solutions, in terms of fooling ratio, when benchmarked with well-known adversarial methods.
Tasks
Published	2018-03-09
URL	http://arxiv.org/abs/1803.03607v4
PDF	http://arxiv.org/pdf/1803.03607v4.pdf
PWC	https://paperswithcode.com/paper/on-generation-of-adversarial-examples-using
Repo	https://github.com/ebalda/adversarialconvex
Framework	tf

CompoNet: Learning to Generate the Unseen by Part Synthesis and Composition


Title	CompoNet: Learning to Generate the Unseen by Part Synthesis and Composition
Authors	Nadav Schor, Oren Katzir, Hao Zhang, Daniel Cohen-Or
Abstract	Data-driven generative modeling has made remarkable progress by leveraging the power of deep neural networks. A reoccurring challenge is how to enable a model to generate a rich variety of samples from the entire target distribution, rather than only from a distribution confined to the training data. In other words, we would like the generative model to go beyond the observed samples and learn to generate unseen'', yet still plausible, data. In our work, we present CompoNet, a generative neural network for 2D or 3D shapes that is based on a part-based prior, where the key idea is for the network to synthesize shapes by varying both the shape parts and their compositions. Treating a shape not as an unstructured whole, but as a (re-)composable set of deformable parts, adds a combinatorial dimension to the generative process to enrich the diversity of the output, encouraging the generator to venture more into the unseen’'. We show that our part-based model generates richer variety of plausible shapes compared with baseline generative models. To this end, we introduce two quantitative metrics to evaluate the diversity of a generative model and assess how well the generated data covers both the training data and unseen data from the same target distribution. Code is available at https://github.com/nschor/CompoNet.
Tasks
Published	2018-11-19
URL	https://arxiv.org/abs/1811.07441v4
PDF	https://arxiv.org/pdf/1811.07441v4.pdf
PWC	https://paperswithcode.com/paper/learning-to-generate-the-unseen-via-part
Repo	https://github.com/nschor/CompoNet
Framework	tf

SurfelMeshing: Online Surfel-Based Mesh Reconstruction


Title	SurfelMeshing: Online Surfel-Based Mesh Reconstruction
Authors	Thomas Schöps, Torsten Sattler, Marc Pollefeys
Abstract	We address the problem of mesh reconstruction from live RGB-D video, assuming a calibrated camera and poses provided externally (e.g., by a SLAM system). In contrast to most existing approaches, we do not fuse depth measurements in a volume but in a dense surfel cloud. We asynchronously (re)triangulate the smoothed surfels to reconstruct a surface mesh. This novel approach enables to maintain a dense surface representation of the scene during SLAM which can quickly adapt to loop closures. This is possible by deforming the surfel cloud and asynchronously remeshing the surface where necessary. The surfel-based representation also naturally supports strongly varying scan resolution. In particular, it reconstructs colors at the input camera’s resolution. Moreover, in contrast to many volumetric approaches, ours can reconstruct thin objects since objects do not need to enclose a volume. We demonstrate our approach in a number of experiments, showing that it produces reconstructions that are competitive with the state-of-the-art, and we discuss its advantages and limitations. The algorithm (excluding loop closure functionality) is available as open source at https://github.com/puzzlepaint/surfelmeshing .
Tasks
Published	2018-10-01
URL	https://arxiv.org/abs/1810.00729v2
PDF	https://arxiv.org/pdf/1810.00729v2.pdf
PWC	https://paperswithcode.com/paper/surfelmeshing-online-surfel-based-mesh
Repo	https://github.com/puzzlepaint/surfelmeshing
Framework	none

Scikit-Multiflow: A Multi-output Streaming Framework


Title	Scikit-Multiflow: A Multi-output Streaming Framework
Authors	Jacob Montiel, Jesse Read, Albert Bifet, Talel Abdessalem
Abstract	Scikit-multiflow is a multi-output/multi-label and stream data mining framework for the Python programming language. Conceived to serve as a platform to encourage democratization of stream learning research, it provides multiple state of the art methods for stream learning, stream generators and evaluators. scikit-multiflow builds upon popular open source frameworks including scikit-learn, MOA and MEKA. Development follows the FOSS principles and quality is enforced by complying with PEP8 guidelines and using continuous integration and automatic testing. The source code is publicly available at https://github.com/scikit-multiflow/scikit-multiflow.
Tasks
Published	2018-07-12
URL	http://arxiv.org/abs/1807.04662v1
PDF	http://arxiv.org/pdf/1807.04662v1.pdf
PWC	https://paperswithcode.com/paper/scikit-multiflow-a-multi-output-streaming
Repo	https://github.com/scikit-multiflow/scikit-multiflow
Framework	none

Efficient Bayesian Experimental Design for Implicit Models


Title	Efficient Bayesian Experimental Design for Implicit Models
Authors	Steven Kleinegesse, Michael Gutmann
Abstract	Bayesian experimental design involves the optimal allocation of resources in an experiment, with the aim of optimising cost and performance. For implicit models, where the likelihood is intractable but sampling from the model is possible, this task is particularly difficult and therefore largely unexplored. This is mainly due to technical difficulties associated with approximating posterior distributions and utility functions. We devise a novel experimental design framework for implicit models that improves upon previous work in two ways. First, we use the mutual information between parameters and data as the utility function, which has previously not been feasible. We achieve this by utilising Likelihood-Free Inference by Ratio Estimation (LFIRE) to approximate posterior distributions, instead of the traditional approximate Bayesian computation or synthetic likelihood methods. Secondly, we use Bayesian optimisation in order to solve the optimal design problem, as opposed to the typically used grid search or sampling-based methods. We find that this increases efficiency and allows us to consider higher design dimensions.
Tasks	Bayesian Optimisation
Published	2018-10-23
URL	http://arxiv.org/abs/1810.09912v2
PDF	http://arxiv.org/pdf/1810.09912v2.pdf
PWC	https://paperswithcode.com/paper/efficient-bayesian-experimental-design-for
Repo	https://github.com/stevenkleinegesse/bedimplicit
Framework	none

Hyperparameter Learning via Distributional Transfer


Title	Hyperparameter Learning via Distributional Transfer
Authors	Ho Chung Leon Law, Peilin Zhao, Lucian Chan, Junzhou Huang, Dino Sejdinovic
Abstract	Bayesian optimisation is a popular technique for hyperparameter learning but typically requires initial exploration even in cases where similar prior tasks have been solved. We propose to transfer information across tasks using learnt representations of training datasets used in those tasks. This results in a joint Gaussian process model on hyperparameters and data representations. Representations make use of the framework of distribution embeddings into reproducing kernel Hilbert spaces. The developed method has a faster convergence compared to existing baselines, in some cases requiring only a few evaluations of the target objective.
Tasks	Bayesian Optimisation
Published	2018-10-15
URL	https://arxiv.org/abs/1810.06305v3
PDF	https://arxiv.org/pdf/1810.06305v3.pdf
PWC	https://paperswithcode.com/paper/hyperparameter-learning-via-distributional
Repo	https://github.com/hcllaw/distBO
Framework	tf

Macro-Micro Adversarial Network for Human Parsing


Title	Macro-Micro Adversarial Network for Human Parsing
Authors	Yawei Luo, Zhedong Zheng, Liang Zheng, Tao Guan, Junqing Yu, Yi Yang
Abstract	In human parsing, the pixel-wise classification loss has drawbacks in its low-level local inconsistency and high-level semantic inconsistency. The introduction of the adversarial network tackles the two problems using a single discriminator. However, the two types of parsing inconsistency are generated by distinct mechanisms, so it is difficult for a single discriminator to solve them both. To address the two kinds of inconsistencies, this paper proposes the Macro-Micro Adversarial Net (MMAN). It has two discriminators. One discriminator, Macro D, acts on the low-resolution label map and penalizes semantic inconsistency, e.g., misplaced body parts. The other discriminator, Micro D, focuses on multiple patches of the high-resolution label map to address the local inconsistency, e.g., blur and hole. Compared with traditional adversarial networks, MMAN not only enforces local and semantic consistency explicitly, but also avoids the poor convergence problem of adversarial networks when handling high resolution images. In our experiment, we validate that the two discriminators are complementary to each other in improving the human parsing accuracy. The proposed framework is capable of producing competitive parsing performance compared with the state-of-the-art methods, i.e., mIoU=46.81% and 59.91% on LIP and PASCAL-Person-Part, respectively. On a relatively small dataset PPSS, our pre-trained model demonstrates impressive generalization ability. The code is publicly available at https://github.com/RoyalVane/MMAN.
Tasks	Human Parsing, Human Part Segmentation, Semantic Segmentation
Published	2018-07-22
URL	http://arxiv.org/abs/1807.08260v2
PDF	http://arxiv.org/pdf/1807.08260v2.pdf
PWC	https://paperswithcode.com/paper/macro-micro-adversarial-network-for-human
Repo	https://github.com/RoyalVane/MMAN
Framework	pytorch

Show, Attend and Read: A Simple and Strong Baseline for Irregular Text Recognition


Title	Show, Attend and Read: A Simple and Strong Baseline for Irregular Text Recognition
Authors	Hui Li, Peng Wang, Chunhua Shen, Guyu Zhang
Abstract	Recognizing irregular text in natural scene images is challenging due to the large variance in text appearance, such as curvature, orientation and distortion. Most existing approaches rely heavily on sophisticated model designs and/or extra fine-grained annotations, which, to some extent, increase the difficulty in algorithm implementation and data collection. In this work, we propose an easy-to-implement strong baseline for irregular scene text recognition, using off-the-shelf neural network components and only word-level annotations. It is composed of a $31$-layer ResNet, an LSTM-based encoder-decoder framework and a 2-dimensional attention module. Despite its simplicity, the proposed method is robust and achieves state-of-the-art performance on both regular and irregular scene text recognition benchmarks. Code is available at: https://tinyurl.com/ShowAttendRead
Tasks	Irregular Text Recognition, Scene Text Recognition
Published	2018-11-02
URL	http://arxiv.org/abs/1811.00751v2
PDF	http://arxiv.org/pdf/1811.00751v2.pdf
PWC	https://paperswithcode.com/paper/show-attend-and-read-a-simple-and-strong
Repo	https://github.com/15ROBO/ShowAttendRead
Framework	none