April 1, 2020

3382 words 16 mins read

Paper Group ANR 415

Paper Group ANR 415

A Simulation Study of Bandit Algorithms to Address External Validity of Software Fault Prediction. Efficient and Stable Graph Scattering Transforms via Pruning. Efficient sampling generation from explicit densities via Normalizing Flows. On Learning Vehicle Detection in Satellite Video. Combining data assimilation and machine learning to emulate a …

A Simulation Study of Bandit Algorithms to Address External Validity of Software Fault Prediction

Title A Simulation Study of Bandit Algorithms to Address External Validity of Software Fault Prediction
Authors Teruki Hayakawa, Masateru Tsunoda, Koji Toda, Keitaro Nakasai, Kenichi Matsumoto
Abstract Various software fault prediction models and techniques for building algorithms have been proposed. Many studies have compared and evaluated them to identify the most effective ones. However, in most cases, such models and techniques do not have the best performance on every dataset. This is because there is diversity of software development datasets, and therefore, there is a risk that the selected model or technique shows bad performance on a certain dataset. To avoid selecting a low accuracy model, we apply bandit algorithms to predict faults. Consider a case where player has 100 coins to bet on several slot machines. Ordinary usage of software fault prediction is analogous to the player betting all 100 coins in one slot machine. In contrast, bandit algorithms bet one coin on each machine (i.e., use prediction models) step-by-step to seek the best machine. In the experiment, we developed an artificial dataset that includes 100 modules, 15 of which include faults. Then, we developed various artificial fault prediction models and selected them dynamically using bandit algorithms. The Thomson sampling algorithm showed the best or second-best prediction performance compared with using only one prediction model.
Published 2020-03-11
URL https://arxiv.org/abs/2003.05094v2
PDF https://arxiv.org/pdf/2003.05094v2.pdf
PWC https://paperswithcode.com/paper/a-simulation-study-of-bandit-algorithms-to

Efficient and Stable Graph Scattering Transforms via Pruning

Title Efficient and Stable Graph Scattering Transforms via Pruning
Authors Vassilis N. Ioannidis, Siheng Chen, Georgios B. Giannakis
Abstract Graph convolutional networks (GCNs) have well-documented performance in various graph learning tasks, but their analysis is still at its infancy. Graph scattering transforms (GSTs) offer training-free deep GCN models that extract features from graph data, and are amenable to generalization and stability analyses. The price paid by GSTs is exponential complexity in space and time that increases with the number of layers. This discourages deployment of GSTs when a deep architecture is needed. The present work addresses the complexity limitation of GSTs by introducing an efficient so-termed pruned (p)GST approach. The resultant pruning algorithm is guided by a graph-spectrum-inspired criterion, and retains informative scattering features on-the-fly while bypassing the exponential complexity associated with GSTs. Stability of the novel pGSTs is also established when the input graph data or the network structure are perturbed. Furthermore, the sensitivity of pGST to random and localized signal perturbations is investigated analytically and experimentally. Numerical tests showcase that pGST performs comparably to the baseline GST at considerable computational savings. Furthermore, pGST achieves comparable performance to state-of-the-art GCNs in graph and 3D point cloud classification tasks. Upon analyzing the pGST pruning patterns, it is shown that graph data in different domains call for different network architectures, and that the pruning algorithm may be employed to guide the design choices for contemporary GCNs.
Published 2020-01-27
URL https://arxiv.org/abs/2001.09882v1
PDF https://arxiv.org/pdf/2001.09882v1.pdf
PWC https://paperswithcode.com/paper/efficient-and-stable-graph-scattering

Efficient sampling generation from explicit densities via Normalizing Flows

Title Efficient sampling generation from explicit densities via Normalizing Flows
Authors Sebastian Pina-Otey, Thorsten Lux, Federico Sánchez, Vicens Gaitan
Abstract For many applications, such as computing the expected value of different magnitudes, sampling from a known probability density function, the target density, is crucial but challenging through the inverse transform. In these cases, rejection and importance sampling require suitable proposal densities, which can be evaluated and sampled from efficiently. We will present a method based on normalizing flows, proposing a solution for the common problem of exploding reverse Kullback-Leibler divergence due to the target density having values of 0 in regions of the flow transformation. The performance of the method will be demonstrated using a multi-mode complex density function.
Published 2020-03-23
URL https://arxiv.org/abs/2003.10200v1
PDF https://arxiv.org/pdf/2003.10200v1.pdf
PWC https://paperswithcode.com/paper/efficient-sampling-generation-from-explicit

On Learning Vehicle Detection in Satellite Video

Title On Learning Vehicle Detection in Satellite Video
Authors Roman Pflugfelder, Axel Weissenfeld, Julian Wagner
Abstract Vehicle detection in aerial and satellite images is still challenging due to their tiny appearance in pixels compared to the overall size of remote sensing imagery. Classical methods of object detection very often fail in this scenario due to violation of implicit assumptions made such as rich texture, small to moderate ratios between image size and object size. Satellite video is a very new modality which introduces temporal consistency as inductive bias. Approaches for vehicle detection in satellite video use either background subtraction, frame differencing or subspace methods showing moderate performance (0.26 - 0.82 $F_1$ score). This work proposes to apply recent work on deep learning for wide-area motion imagery (WAMI) on satellite video. We show in a first approach comparable results (0.84 $F_1$) on Planet’s SkySat-1 LasVegas video with room for further improvement.
Tasks Object Detection
Published 2020-01-29
URL https://arxiv.org/abs/2001.10900v1
PDF https://arxiv.org/pdf/2001.10900v1.pdf
PWC https://paperswithcode.com/paper/on-learning-vehicle-detection-in-satellite

Combining data assimilation and machine learning to emulate a dynamical model from sparse and noisy observations: a case study with the Lorenz 96 model

Title Combining data assimilation and machine learning to emulate a dynamical model from sparse and noisy observations: a case study with the Lorenz 96 model
Authors Julien Brajard, Alberto Carassi, Marc Bocquet, Laurent Bertino
Abstract A novel method, based on the combination of data assimilation and machine learning is introduced. The new hybrid approach is designed for a two-fold scope: (i) emulating hidden, possibly chaotic, dynamics and (ii) predicting their future states. The method consists in applying iteratively a data assimilation step, here an ensemble Kalman filter, and a neural network. Data assimilation is used to optimally combine a surrogate model with sparse noisy data. The output analysis is spatially complete and is used as a training set by the neural network to update the surrogate model. The two steps are then repeated iteratively. Numerical experiments have been carried out using the chaotic 40-variables Lorenz 96 model, proving both convergence and statistical skills of the proposed hybrid approach. The surrogate model shows short-term forecast skills up to two Lyapunov times, the retrieval of positive Lyapunov exponents as well as the more energetic frequencies of the power density spectrum. The sensitivity of the method to critical setup parameters is also presented: forecast skills decrease smoothly with increased observational noise but drops abruptly if less than half of the model domain is observed. The successful synergy between data assimilation and machine learning, proven here with a low-dimensional system, encourages further investigation of such hybrids with more sophisticated dynamics.
Published 2020-01-06
URL https://arxiv.org/abs/2001.01520v1
PDF https://arxiv.org/pdf/2001.01520v1.pdf
PWC https://paperswithcode.com/paper/combining-data-assimilation-and-machine

Phase transitions in a decentralized graph-based approach to human language

Title Phase transitions in a decentralized graph-based approach to human language
Authors Javier Vera, Felipe Urbina, Wenceslao Palma
Abstract Zipf’s law establishes a scaling behavior for word-frequencies in large text corpora. The appearance of Zipfian properties in human language has been previously explained as an optimization problem for the interests of speakers and hearers. On the other hand, human-like vocabularies can be viewed as bipartite graphs. The aim here is double: within a bipartite-graph approach to human vocabularies, to propose a decentralized language game model for the formation of Zipfian properties. To do this, we define a language game, in which a population of artificial agents is involved in idealized linguistic interactions. Numerical simulations show the appearance of a phase transition from an initially disordered state to three possible phases for language formation. Our results suggest that Zipfian properties in language seem to arise partly from decentralized linguistic interactions between agents endowed with bipartite word-meaning mappings.
Published 2020-03-04
URL https://arxiv.org/abs/2003.02639v1
PDF https://arxiv.org/pdf/2003.02639v1.pdf
PWC https://paperswithcode.com/paper/phase-transitions-in-a-decentralized-graph

Playing to Learn Better: Repeated Games for Adversarial Learning with Multiple Classifiers

Title Playing to Learn Better: Repeated Games for Adversarial Learning with Multiple Classifiers
Authors Prithviraj Dasgupta, Joseph B. Collins, Michael McCarrick
Abstract We consider the problem of prediction by a machine learning algorithm, called learner, within an adversarial learning setting. The learner’s task is to correctly predict the class of data passed to it as a query. However, along with queries containing clean data, the learner could also receive malicious or adversarial queries from an adversary. The objective of the adversary is to evade the learner’s prediction mechanism by sending adversarial queries that result in erroneous class prediction by the learner, while the learner’s objective is to reduce the incorrect prediction of these adversarial queries without degrading the prediction quality of clean queries. We propose a game theory-based technique called a Repeated Bayesian Sequential Game where the learner interacts repeatedly with a model of the adversary using self play to determine the distribution of adversarial versus clean queries. It then strategically selects a classifier from a set of pre-trained classifiers that balances the likelihood of correct prediction for the query along with reducing the costs to use the classifier. We have evaluated our proposed technique using clean and adversarial text data with deep neural network-based classifiers and shown that the learner can select an appropriate classifier that is commensurate with the query type (clean or adversarial) while remaining aware of the cost to use the classifier.
Tasks Adversarial Text
Published 2020-02-10
URL https://arxiv.org/abs/2002.03924v1
PDF https://arxiv.org/pdf/2002.03924v1.pdf
PWC https://paperswithcode.com/paper/playing-to-learn-better-repeated-games-for

Supervised Learning: No Loss No Cry

Title Supervised Learning: No Loss No Cry
Authors Richard Nock, Aditya Krishna Menon
Abstract Supervised learning requires the specification of a loss function to minimise. While the theory of admissible losses from both a computational and statistical perspective is well-developed, these offer a panoply of different choices. In practice, this choice is typically made in an \emph{ad hoc} manner. In hopes of making this procedure more principled, the problem of \emph{learning the loss function} for a downstream task (e.g., classification) has garnered recent interest. However, works in this area have been generally empirical in nature. In this paper, we revisit the {\sc SLIsotron} algorithm of Kakade et al. (2011) through a novel lens, derive a generalisation based on Bregman divergences, and show how it provides a principled procedure for learning the loss. In detail, we cast {\sc SLIsotron} as learning a loss from a family of composite square losses. By interpreting this through the lens of \emph{proper losses}, we derive a generalisation of {\sc SLIsotron} based on Bregman divergences. The resulting {\sc BregmanTron} algorithm jointly learns the loss along with the classifier. It comes equipped with a simple guarantee of convergence for the loss it learns, and its set of possible outputs comes with a guarantee of agnostic approximability of Bayes rule. Experiments indicate that the {\sc BregmanTron} substantially outperforms the {\sc SLIsotron}, and that the loss it learns can be minimized by other algorithms for different tasks, thereby opening the interesting problem of \textit{loss transfer} between domains.
Published 2020-02-10
URL https://arxiv.org/abs/2002.03555v1
PDF https://arxiv.org/pdf/2002.03555v1.pdf
PWC https://paperswithcode.com/paper/supervised-learning-no-loss-no-cry

LC-GAN: Image-to-image Translation Based on Generative Adversarial Network for Endoscopic Images

Title LC-GAN: Image-to-image Translation Based on Generative Adversarial Network for Endoscopic Images
Authors Shan Lin, Fangbo Qin, Yangming Li, Randall A. Bly, Kris S. Moe, Blake Hannaford
Abstract The intelligent perception of endoscopic vision is appealing in many computer-assisted and robotic surgeries. Achieving good vision-based analysis with deep learning techniques requires large labeled datasets, but manual data labeling is expensive and time-consuming in medical problems. When applying a trained model to a different but relevant dataset, a new labeled dataset may be required for training to avoid performance degradation. In this work, we investigate a novel cross-domain strategy to reduce the need for manual data labeling by proposing an image-to-image translation model called live-cadaver GAN (LC-GAN) based on generative adversarial networks (GANs). More specifically, we consider a situation when a labeled cadaveric surgery dataset is available while the task is instrument segmentation on a live surgery dataset. We train LC-GAN to learn the mappings between the cadaveric and live datasets. To achieve instrument segmentation on live images, we can first translate the live images to fake-cadaveric images with LC-GAN, and then perform segmentation on the fake-cadaveric images with models trained on the real cadaveric dataset. With this cross-domain strategy, we fully leverage the labeled cadaveric dataset for segmentation on live images without the need to label the live dataset again. Two generators with different architectures are designed for LC-GAN to make use of the deep feature representation learned from the cadaveric image based instrument segmentation task. Moreover, we propose structural similarity loss and segmentation consistency loss to improve the semantic consistency during translation. The results demonstrate that LC-GAN achieves better image-to-image translation results, and leads to improved segmentation performance in the proposed cross-domain segmentation task.
Tasks Image-to-Image Translation
Published 2020-03-10
URL https://arxiv.org/abs/2003.04949v1
PDF https://arxiv.org/pdf/2003.04949v1.pdf
PWC https://paperswithcode.com/paper/lc-gan-image-to-image-translation-based-on

Simple and Accurate Uncertainty Quantification from Bias-Variance Decomposition

Title Simple and Accurate Uncertainty Quantification from Bias-Variance Decomposition
Authors Shi Hu, Nicola Pezzotti, Dimitrios Mavroeidis, Max Welling
Abstract Accurate uncertainty quantification is crucial for many applications where decisions are in play. Examples include medical diagnosis and self-driving vehicles. We propose a new method that is based directly on the bias-variance decomposition, where the parameter uncertainty is given by the variance of an ensemble divided by the number of members in the ensemble, and the aleatoric uncertainty plus the squared bias is estimated by training a separate model that is regressed directly on the errors of the predictor. We demonstrate that this simple sequential procedure provides much more accurate uncertainty estimates than the current state-of-the-art on two MRI reconstruction tasks.
Tasks Medical Diagnosis
Published 2020-02-13
URL https://arxiv.org/abs/2002.05582v1
PDF https://arxiv.org/pdf/2002.05582v1.pdf
PWC https://paperswithcode.com/paper/simple-and-accurate-uncertainty

Potential Neutralizing Antibodies Discovered for Novel Corona Virus Using Machine Learning

Title Potential Neutralizing Antibodies Discovered for Novel Corona Virus Using Machine Learning
Authors Rishikesh Magar, Prakarsh Yadav, Amir Barati Farimani
Abstract The fast and untraceable virus mutations take lives of thousands of people before the immune system can produce the inhibitory antibody. Recent outbreak of novel coronavirus infected and killed thousands of people in the world. Rapid methods in finding peptides or antibody sequences that can inhibit the viral epitopes of COVID-19 will save the life of thousands. In this paper, we devised a machine learning (ML) model to predict the possible inhibitory synthetic antibodies for Corona virus. We collected 1933 virus-antibody sequences and their clinical patient neutralization response and trained an ML model to predict the antibody response. Using graph featurization with variety of ML methods, we screened thousands of hypothetical antibody sequences and found 8 stable antibodies that potentially inhibit COVID-19. We combined bioinformatics, structural biology, and Molecular Dynamics (MD) simulations to verify the stability of the candidate antibodies that can inhibit the Corona virus.
Published 2020-03-18
URL https://arxiv.org/abs/2003.08447v1
PDF https://arxiv.org/pdf/2003.08447v1.pdf
PWC https://paperswithcode.com/paper/potential-neutralizing-antibodies-discovered

Adaptive Batching for Gaussian Process Surrogates with Application in Noisy Level Set Estimation

Title Adaptive Batching for Gaussian Process Surrogates with Application in Noisy Level Set Estimation
Authors Xiong Lyu, Mike Ludkovski
Abstract We develop adaptive replicated designs for Gaussian process metamodels of stochastic experiments. Adaptive batching is a natural extension of sequential design heuristics with the benefit of replication growing as response features are learned, inputs concentrate, and the metamodeling overhead rises. Motivated by the problem of learning the level set of the mean simulator response we develop four novel schemes: Multi-Level Batching (MLB), Ratchet Batching (RB), Adaptive Batched Stepwise Uncertainty Reduction (ABSUR), Adaptive Design with Stepwise Allocation (ADSA) and Deterministic Design with Stepwise Allocation (DDSA). Our algorithms simultaneously (MLB, RB and ABSUR) or sequentially (ADSA and DDSA) determine the sequential design inputs and the respective number of replicates. Illustrations using synthetic examples and an application in quantitative finance (Bermudan option pricing via Regression Monte Carlo) show that adaptive batching brings significant computational speed-ups with minimal loss of modeling fidelity.
Published 2020-03-19
URL https://arxiv.org/abs/2003.08579v1
PDF https://arxiv.org/pdf/2003.08579v1.pdf
PWC https://paperswithcode.com/paper/adaptive-batching-for-gaussian-process

Optimised Convolutional Neural Networks for Heart Rate Estimation and Human Activity Recognition in Wrist Worn Sensing Applications

Title Optimised Convolutional Neural Networks for Heart Rate Estimation and Human Activity Recognition in Wrist Worn Sensing Applications
Authors Eoin Brophy, Willie Muehlhausen, Alan F. Smeaton, Tomas E. Ward
Abstract Wrist-worn smart devices are providing increased insights into human health, behaviour and performance through sophisticated analytics. However, battery life, device cost and sensor performance in the face of movement-related artefact present challenges which must be further addressed to see effective applications and wider adoption through commoditisation of the technology. We address these challenges by demonstrating, through using a simple optical measurement, photoplethysmography (PPG) used conventionally for heart rate detection in wrist-worn sensors, that we can provide improved heart rate and human activity recognition (HAR) simultaneously at low sample rates, without an inertial measurement unit. This simplifies hardware design and reduces costs and power budgets. We apply two deep learning pipelines, one for human activity recognition and one for heart rate estimation. HAR is achieved through the application of a visual classification approach, capable of robust performance at low sample rates. Here, transfer learning is leveraged to retrain a convolutional neural network (CNN) to distinguish characteristics of the PPG during different human activities. For heart rate estimation we use a CNN adopted for regression which maps noisy optical signals to heart rate estimates. In both cases, comparisons are made with leading conventional approaches. Our results demonstrate a low sampling frequency can achieve good performance without significant degradation of accuracy. 5 Hz and 10 Hz were shown to have 80.2% and 83.0% classification accuracy for HAR respectively. These same sampling frequencies also yielded a robust heart rate estimation which was comparative with that achieved at the more energy-intensive rate of 256 Hz.
Tasks Activity Recognition, Heart rate estimation, Human Activity Recognition, Photoplethysmography (PPG), Transfer Learning
Published 2020-03-30
URL https://arxiv.org/abs/2004.00505v1
PDF https://arxiv.org/pdf/2004.00505v1.pdf
PWC https://paperswithcode.com/paper/optimised-convolutional-neural-networks-for

Take the Scenic Route: Improving Generalization in Vision-and-Language Navigation

Title Take the Scenic Route: Improving Generalization in Vision-and-Language Navigation
Authors Felix Yu, Zhiwei Deng, Karthik Narasimhan, Olga Russakovsky
Abstract In the Vision-and-Language Navigation (VLN) task, an agent with egocentric vision navigates to a destination given natural language instructions. The act of manually annotating these instructions is timely and expensive, such that many existing approaches automatically generate additional samples to improve agent performance. However, these approaches still have difficulty generalizing their performance to new environments. In this work, we investigate the popular Room-to-Room (R2R) VLN benchmark and discover that what is important is not only the amount of data you synthesize, but also how you do it. We find that shortest path sampling, which is used by both the R2R benchmark and existing augmentation methods, encode biases in the action space of the agent which we dub as action priors. We then show that these action priors offer one explanation toward the poor generalization of existing works. To mitigate such priors, we propose a path sampling method based on random walks to augment the data. By training with this augmentation strategy, our agent is able to generalize better to unknown environments compared to the baseline, significantly improving model performance in the process.
Published 2020-03-31
URL https://arxiv.org/abs/2003.14269v1
PDF https://arxiv.org/pdf/2003.14269v1.pdf
PWC https://paperswithcode.com/paper/take-the-scenic-route-improving

Dynamic transformation of prior knowledge into Bayesian models for data streams

Title Dynamic transformation of prior knowledge into Bayesian models for data streams
Authors Tran Xuan Bach, Nguyen Duc Anh, Ngo Van Linh, Khoat Than
Abstract We consider how to effectively use prior knowledge when learning a Bayesian model from streaming environments where the data come infinitely and sequentially. This problem is highly important in the era of data explosion and rich sources of precious external knowledge such as pre-trained models, ontologies, Wikipedia, etc. We show that some existing approaches can forget any knowledge very fast. We then propose a novel framework that enables to incorporate the prior knowledge of different forms into a base Bayesian model for data streams. Our framework subsumes some existing popular models for time-series/dynamic data. Extensive experiments show that our framework outperforms existing methods with a large margin. In particular, our framework can help Bayesian models generalize well on extremely short text while other methods overfit. The implementation of our framework is available at https://github.com/bachtranxuan/TPS.git.
Tasks Time Series
Published 2020-03-13
URL https://arxiv.org/abs/2003.06123v3
PDF https://arxiv.org/pdf/2003.06123v3.pdf
PWC https://paperswithcode.com/paper/dynamic-transformation-of-prior-knowledge
comments powered by Disqus