January 31, 2020

3214 words 16 mins read

Paper Group AWR 443

Galaxy Zoo: Probabilistic Morphology through Bayesian CNNs and Active Learning. Fence GAN: Towards Better Anomaly Detection. The Mathematics of Text Structure. Deep Active Learning for Axon-Myelin Segmentation on Histology Data. On the effect of age perception biases for real age regression. Efficient Saliency Maps for Explainable AI. Fooling Neura …

Galaxy Zoo: Probabilistic Morphology through Bayesian CNNs and Active Learning


Title	Galaxy Zoo: Probabilistic Morphology through Bayesian CNNs and Active Learning
Authors	Mike Walmsley, Lewis Smith, Chris Lintott, Yarin Gal, Steven Bamford, Hugh Dickinson, Lucy Fortson, Sandor Kruk, Karen Masters, Claudia Scarlata, Brooke Simmons, Rebecca Smethurst, Darryl Wright
Abstract	We use Bayesian convolutional neural networks and a novel generative model of Galaxy Zoo volunteer responses to infer posteriors for the visual morphology of galaxies. Bayesian CNN can learn from galaxy images with uncertain labels and then, for previously unlabelled galaxies, predict the probability of each possible label. Our posteriors are well-calibrated (e.g. for predicting bars, we achieve coverage errors of 11.8% within a vote fraction deviation of 0.2) and hence are reliable for practical use. Further, using our posteriors, we apply the active learning strategy BALD to request volunteer responses for the subset of galaxies which, if labelled, would be most informative for training our network. We show that training our Bayesian CNNs using active learning requires up to 35-60% fewer labelled galaxies, depending on the morphological feature being classified. By combining human and machine intelligence, Galaxy Zoo will be able to classify surveys of any conceivable scale on a timescale of weeks, providing massive and detailed morphology catalogues to support research into galaxy evolution.
Tasks	Active Learning
Published	2019-05-17
URL	https://arxiv.org/abs/1905.07424v2
PDF	https://arxiv.org/pdf/1905.07424v2.pdf
PWC	https://paperswithcode.com/paper/galaxy-zoo-probabilistic-morphology-through
Repo	https://github.com/mwalmsley/galaxy-zoo-bayesian-cnn
Framework	tf

Fence GAN: Towards Better Anomaly Detection


Title	Fence GAN: Towards Better Anomaly Detection
Authors	Cuong Phuc Ngo, Amadeus Aristo Winarto, Connie Kou Khor Li, Sojeong Park, Farhan Akram, Hwee Kuan Lee
Abstract	Anomaly detection is a classical problem where the aim is to detect anomalous data that do not belong to the normal data distribution. Current state-of-the-art methods for anomaly detection on complex high-dimensional data are based on the generative adversarial network (GAN). However, the traditional GAN loss is not directly aligned with the anomaly detection objective: it encourages the distribution of the generated samples to overlap with the real data and so the resulting discriminator has been found to be ineffective as an anomaly detector. In this paper, we propose simple modifications to the GAN loss such that the generated samples lie at the boundary of the real data distribution. With our modified GAN loss, our anomaly detection method, called Fence GAN (FGAN), directly uses the discriminator score as an anomaly threshold. Our experimental results using the MNIST, CIFAR10 and KDD99 datasets show that Fence GAN yields the best anomaly classification accuracy compared to state-of-the-art methods.
Tasks	Anomaly Detection
Published	2019-04-02
URL	http://arxiv.org/abs/1904.01209v1
PDF	http://arxiv.org/pdf/1904.01209v1.pdf
PWC	https://paperswithcode.com/paper/fence-gan-towards-better-anomaly-detection
Repo	https://github.com/phuccuongngo99/Fence_GAN
Framework	tf

The Mathematics of Text Structure


Title	The Mathematics of Text Structure
Authors	Bob Coecke
Abstract	In previous work we gave a mathematical foundation, referred to as DisCoCat, for how words interact in a sentence in order to produce the meaning of that sentence. To do so, we exploited the perfect structural match of grammar and categories of meaning spaces. Here, we give a mathematical foundation, referred to as DisCoCirc, for how sentences interact in texts in order to produce the meaning of that text. First we revisit DisCoCat. While in DisCoCat all meanings are fixed as states (i.e. have no input), in DisCoCirc word meanings correspond to a type, or system, and the states of this system can evolve. Sentences are gates within a circuit which update the variable meanings of those words. Like in DisCoCat, word meanings can live in a variety of spaces e.g. propositional, vectorial, or cognitive. The compositional structure are string diagrams representing information flows, and an entire text yields a single string diagram in which word meanings lift to the meaning of an entire text. While the developments in this paper are independent of a physical embodiment (cf. classical vs. quantum computing), both the compositional formalism and suggested meaning model are highly quantum-inspired, and implementation on a quantum computer would come with a range of benefits. We also praise Jim Lambek for his role in mathematical linguistics in general, and the development of the DisCo program more specifically.
Tasks
Published	2019-04-06
URL	https://arxiv.org/abs/1904.03478v2
PDF	https://arxiv.org/pdf/1904.03478v2.pdf
PWC	https://paperswithcode.com/paper/the-mathematics-of-text-structure
Repo	https://github.com/toumix/discopy
Framework	none

Deep Active Learning for Axon-Myelin Segmentation on Histology Data


Title	Deep Active Learning for Axon-Myelin Segmentation on Histology Data
Authors	Melanie Lubrano di Scandalea, Christian S. Perone, Mathieu Boudreau, Julien Cohen-Adad
Abstract	Semantic segmentation is a crucial task in biomedical image processing, which recent breakthroughs in deep learning have allowed to improve. However, deep learning methods in general are not yet widely used in practice since they require large amount of data for training complex models. This is particularly challenging for biomedical images, because data and ground truths are a scarce resource. Annotation efforts for biomedical images come with a real cost, since experts have to manually label images at pixel-level on samples usually containing many instances of the target anatomy (e.g. in histology samples: neurons, astrocytes, mitochondria, etc.). In this paper we provide a framework for Deep Active Learning applied to a real-world scenario. Our framework relies on the U-Net architecture and overall uncertainty measure to suggest which sample to annotate. It takes advantage of the uncertainty measure obtained by taking Monte Carlo samples while using Dropout regularization scheme. Experiments were done on spinal cord and brain microscopic histology samples to perform a myelin segmentation task. Two realistic small datasets of 14 and 24 images were used, from different acquisition settings (Serial Block-Face Electron Microscopy and Transmitting Electron Microscopy) and showed that our method reached a maximum Dice value after adding 3 uncertainty-selected samples to the initial training set, versus 15 randomly-selected samples, thereby significantly reducing the annotation effort. We focused on a plausible scenario and showed evidence that this straightforward implementation achieves a high segmentation performance with very few labelled samples. We believe our framework may benefit any biomedical researcher willing to obtain fast and accurate image segmentation on their own dataset. The code is freely available at https://github.com/neuropoly/deep-active-learning.
Tasks	Active Learning, Semantic Segmentation
Published	2019-07-11
URL	https://arxiv.org/abs/1907.05143v1
PDF	https://arxiv.org/pdf/1907.05143v1.pdf
PWC	https://paperswithcode.com/paper/deep-active-learning-for-axon-myelin
Repo	https://github.com/neuropoly/axondeepseg
Framework	tf

On the effect of age perception biases for real age regression


Title	On the effect of age perception biases for real age regression
Authors	Julio C. S. Jacques Junior, Cagri Ozcinar, Marina Marjanovic, Xavier Baró, Gholamreza Anbarjafari, Sergio Escalera
Abstract	Automatic age estimation from facial images represents an important task in computer vision. This paper analyses the effect of gender, age, ethnic, makeup and expression attributes of faces as sources of bias to improve deep apparent age prediction. Following recent works where it is shown that apparent age labels benefit real age estimation, rather than direct real to real age regression, our main contribution is the integration, in an end-to-end architecture, of face attributes for apparent age prediction with an additional loss for real age regression. Experimental results on the APPA-REAL dataset indicate the proposed network successfully take advantage of the adopted attributes to improve both apparent and real age estimation. Our model outperformed a state-of-the-art architecture proposed to separately address apparent and real age regression. Finally, we present preliminary results and discussion of a proof of concept application using the proposed model to regress the apparent age of an individual based on the gender of an external observer.
Tasks	Age Estimation
Published	2019-02-20
URL	http://arxiv.org/abs/1902.07653v1
PDF	http://arxiv.org/pdf/1902.07653v1.pdf
PWC	https://paperswithcode.com/paper/on-the-effect-of-age-perception-biases-for
Repo	https://github.com/juliojj/app-real-age
Framework	tf

Efficient Saliency Maps for Explainable AI


Title	Efficient Saliency Maps for Explainable AI
Authors	T. Nathan Mundhenk, Barry Y. Chen, Gerald Friedland
Abstract	We describe an explainable AI saliency map method for use with deep convolutional neural networks (CNN) that is much more efficient than popular fine-resolution gradient methods. It is also quantitatively similar or better in accuracy. Our technique works by measuring information at the end of each network scale which is then combined into a single saliency map. We describe how saliency measures can be made more efficient by exploiting Saliency Map Order Equivalence. We visualize individual scale/layer contributions by using a Layer Ordered Visualization of Information. This provides an interesting comparison of scale information contributions within the network not provided by other saliency map methods. Using our method instead of Guided Backprop, coarse-resolution class activation methods such as Grad-CAM and Grad-CAM++ seem to yield demonstrably superior results without sacrificing speed. This will make fine-resolution saliency methods feasible on resource limited platforms such as robots, cell phones, low-cost industrial devices, astronomy and satellite imagery.
Tasks
Published	2019-11-26
URL	https://arxiv.org/abs/1911.11293v2
PDF	https://arxiv.org/pdf/1911.11293v2.pdf
PWC	https://paperswithcode.com/paper/efficient-saliency-maps-for-explainable-ai-1
Repo	https://github.com/LLNL/fastcam
Framework	pytorch

Fooling Neural Network Interpretations via Adversarial Model Manipulation


Title	Fooling Neural Network Interpretations via Adversarial Model Manipulation
Authors	Juyeon Heo, Sunghwan Joo, Taesup Moon
Abstract	We ask whether the neural network interpretation methods can be fooled via adversarial model manipulation, which is defined as a model fine-tuning step that aims to radically alter the explanations without hurting the accuracy of the original models, e.g., VGG19, ResNet50, and DenseNet121. By incorporating the interpretation results directly in the penalty term of the objective function for fine-tuning, we show that the state-of-the-art saliency map based interpreters, e.g., LRP, Grad-CAM, and SimpleGrad, can be easily fooled with our model manipulation. We propose two types of fooling, Passive and Active, and demonstrate such foolings generalize well to the entire validation set as well as transfer to other interpretation methods. Our results are validated by both visually showing the fooled explanations and reporting quantitative metrics that measure the deviations from the original explanations. We claim that the stability of neural network interpretation method with respect to our adversarial model manipulation is an important criterion to check for developing robust and reliable neural network interpretation method.
Tasks
Published	2019-02-06
URL	https://arxiv.org/abs/1902.02041v3
PDF	https://arxiv.org/pdf/1902.02041v3.pdf
PWC	https://paperswithcode.com/paper/fooling-neural-network-interpretations-via
Repo	https://github.com/rmrisforbidden/Fooling_Neural_Network-Interpretations
Framework	pytorch

Gaussian YOLOv3: An Accurate and Fast Object Detector Using Localization Uncertainty for Autonomous Driving


Title	Gaussian YOLOv3: An Accurate and Fast Object Detector Using Localization Uncertainty for Autonomous Driving
Authors	Jiwoong Choi, Dayoung Chun, Hyun Kim, Hyuk-Jae Lee
Abstract	The use of object detection algorithms is becoming increasingly important in autonomous vehicles, and object detection at high accuracy and a fast inference speed is essential for safe autonomous driving. A false positive (FP) from a false localization during autonomous driving can lead to fatal accidents and hinder safe and efficient driving. Therefore, a detection algorithm that can cope with mislocalizations is required in autonomous driving applications. This paper proposes a method for improving the detection accuracy while supporting a real-time operation by modeling the bounding box (bbox) of YOLOv3, which is the most representative of one-stage detectors, with a Gaussian parameter and redesigning the loss function. In addition, this paper proposes a method for predicting the localization uncertainty that indicates the reliability of bbox. By using the predicted localization uncertainty during the detection process, the proposed schemes can significantly reduce the FP and increase the true positive (TP), thereby improving the accuracy. Compared to a conventional YOLOv3, the proposed algorithm, Gaussian YOLOv3, improves the mean average precision (mAP) by 3.09 and 3.5 on the KITTI and Berkeley deep drive (BDD) datasets, respectively. Nevertheless, the proposed algorithm is capable of real-time detection at faster than 42 frames per second (fps) and shows a higher accuracy than previous approaches with a similar fps. Therefore, the proposed algorithm is the most suitable for autonomous driving applications.
Tasks	Autonomous Driving, Autonomous Vehicles, Object Detection
Published	2019-04-09
URL	https://arxiv.org/abs/1904.04620v2
PDF	https://arxiv.org/pdf/1904.04620v2.pdf
PWC	https://paperswithcode.com/paper/gaussian-yolov3-an-accurate-and-fast-object
Repo	https://github.com/motokimura/PyTorch_Gaussian_YOLOv3
Framework	pytorch

PAC-Bayesian Contrastive Unsupervised Representation Learning


Title	PAC-Bayesian Contrastive Unsupervised Representation Learning
Authors	Kento Nozawa, Pascal Germain, Benjamin Guedj
Abstract	Contrastive unsupervised representation learning (CURL) is the state-of-the-art technique to learn representations (as a set of features) from unlabelled data. While CURL has collected several empirical successes recently, theoretical understanding of its performance was still missing. In a recent work, Arora et al. (2019) provide the first generalisation bounds for CURL, relying on a Rademacher complexity. We extend their framework to the flexible PAC-Bayes setting, allowing us to deal with the non-iid setting. We present PAC-Bayesian generalisation bounds for CURL, which are then used to derive a new representation learning algorithm. Numerical experiments on real-life datasets illustrate that our algorithm achieves competitive accuracy, and yields non-vacuous generalisation bounds.
Tasks	Representation Learning, Unsupervised Representation Learning
Published	2019-10-10
URL	https://arxiv.org/abs/1910.04464v2
PDF	https://arxiv.org/pdf/1910.04464v2.pdf
PWC	https://paperswithcode.com/paper/pac-bayesian-contrastive-unsupervised
Repo	https://github.com/nzw0301/pb-contrastive
Framework	pytorch

Gaussian Differential Privacy


Title	Gaussian Differential Privacy
Authors	Jinshuo Dong, Aaron Roth, Weijie J. Su
Abstract	Differential privacy has seen remarkable success as a rigorous and practical formalization of data privacy in the past decade. This privacy definition and its divergence based relaxations, however, have several acknowledged weaknesses, either in handling composition of private algorithms or in analyzing important primitives like privacy amplification by subsampling. Inspired by the hypothesis testing formulation of privacy, this paper proposes a new relaxation, which we term $f$-differential privacy' ($f$-DP). This notion of privacy has a number of appealing properties and, in particular, avoids difficulties associated with divergence based relaxations. First, $f$-DP preserves the hypothesis testing interpretation. In addition, $f$-DP allows for lossless reasoning about composition in an algebraic fashion. Moreover, we provide a powerful technique to import existing results proven for original DP to $f$-DP and, as an application, obtain a simple subsampling theorem for $f$-DP. In addition to the above findings, we introduce a canonical single-parameter family of privacy notions within the $f$-DP class that is referred to as Gaussian differential privacy’ (GDP), defined based on testing two shifted Gaussians. GDP is focal among the $f$-DP class because of a central limit theorem we prove. More precisely, the privacy guarantees of \emph{any} hypothesis testing based definition of privacy (including original DP) converges to GDP in the limit under composition. The CLT also yields a computationally inexpensive tool for analyzing the exact composition of private algorithms. Taken together, this collection of attractive properties render $f$-DP a mathematically coherent, analytically tractable, and versatile framework for private data analysis. Finally, we demonstrate the use of the tools we develop by giving an improved privacy analysis of noisy stochastic gradient descent.
Tasks
Published	2019-05-07
URL	https://arxiv.org/abs/1905.02383v3
PDF	https://arxiv.org/pdf/1905.02383v3.pdf
PWC	https://paperswithcode.com/paper/gaussian-differential-privacy
Repo	https://github.com/woodyx218/Deep-Learning-with-GDP
Framework	tf

BUT-FIT at SemEval-2019 Task 7: Determining the Rumour Stance with Pre-Trained Deep Bidirectional Transformers


Title	BUT-FIT at SemEval-2019 Task 7: Determining the Rumour Stance with Pre-Trained Deep Bidirectional Transformers
Authors	Martin Fajcik, Lukáš Burget, Pavel Smrz
Abstract	This paper describes our system submitted to SemEval 2019 Task 7: RumourEval 2019: Determining Rumour Veracity and Support for Rumours, Subtask A (Gorrell et al., 2019). The challenge focused on classifying whether posts from Twitter and Reddit support, deny, query, or comment a hidden rumour, truthfulness of which is the topic of an underlying discussion thread. We formulate the problem as a stance classification, determining the rumour stance of a post with respect to the previous thread post and the source thread post. The recent BERT architecture was employed to build an end-to-end system which has reached the F1 score of 61.67% on the provided test data. It finished at the 2nd place in the competition, without any hand-crafted features, only 0.2% behind the winner.
Tasks	Rumour Detection
Published	2019-02-25
URL	http://arxiv.org/abs/1902.10126v2
PDF	http://arxiv.org/pdf/1902.10126v2.pdf
PWC	https://paperswithcode.com/paper/but-fit-at-semeval-2019-task-7-determining
Repo	https://github.com/MFajcik/RumourEval2019
Framework	pytorch

Back to the Future – Sequential Alignment of Text Representations


Title	Back to the Future – Sequential Alignment of Text Representations
Authors	Johannes Bjerva, Wouter Kouw, Isabelle Augenstein
Abstract	Language evolves over time in many ways relevant to natural language processing tasks. For example, recent occurrences of tokens ‘BERT’ and ‘ELMO’ in publications refer to neural network architectures rather than persons. This type of temporal signal is typically overlooked, but is important if one aims to deploy a machine learning model over an extended period of time. In particular, language evolution causes data drift between time-steps in sequential decision-making tasks. Examples of such tasks include prediction of paper acceptance for yearly conferences (regular intervals) or author stance prediction for rumours on Twitter (irregular intervals). Inspired by successes in computer vision, we tackle data drift by sequentially aligning learned representations. We evaluate on three challenging tasks varying in terms of time-scales, linguistic units, and domains. These tasks show our method outperforming several strong baselines, including using all available data. We argue that, due to its low computational expense, sequential alignment is a practical solution to dealing with language evolution.
Tasks	Decision Making, Rumour Detection
Published	2019-09-08
URL	https://arxiv.org/abs/1909.03464v3
PDF	https://arxiv.org/pdf/1909.03464v3.pdf
PWC	https://paperswithcode.com/paper/back-to-the-future-sequential-alignment-of
Repo	https://github.com/wmkouw/ssa-nlp
Framework	none

Reinforcement Learning based Interconnection Routing for Adaptive Traffic Optimization


Title	Reinforcement Learning based Interconnection Routing for Adaptive Traffic Optimization
Authors	Sheng-Chun Kao, Chao-Han Huck Yang, Pin-Yu Chen, Xiaoli Ma, Tushar Krishna
Abstract	Applying Machine Learning (ML) techniques to design and optimize computer architectures is a promising research direction. Optimizing the runtime performance of a Network-on-Chip (NoC) necessitates a continuous learning framework. In this work, we demonstrate the promise of applying reinforcement learning (RL) to optimize NoC runtime performance. We present three RL-based methods for learning optimal routing algorithms. The experimental results show the algorithms can successfully learn a near-optimal solution across different environment states. Reproducible Code: github.com/huckiyang/interconnect-routing-gym
Tasks
Published	2019-08-13
URL	https://arxiv.org/abs/1908.04484v1
PDF	https://arxiv.org/pdf/1908.04484v1.pdf
PWC	https://paperswithcode.com/paper/reinforcement-learning-based-interconnection
Repo	https://github.com/felix0901/interconnect-routing-gym
Framework	none

Minimum Delay Object Detection From Video


Title	Minimum Delay Object Detection From Video
Authors	Dong Lao, Ganesh Sundaramoorthi
Abstract	We consider the problem of detecting objects, as they come into view, from videos in an online fashion. We provide the first real-time solution that is guaranteed to minimize the delay, i.e., the time between when the object comes in view and the declared detection time, subject to acceptable levels of detection accuracy. The method leverages modern CNN-based object detectors that operate on a single frame, to aggregate detection results over frames to provide reliable detection at a rate, specified by the user, in guaranteed minimal delay. To do this, we formulate the problem as a Quickest Detection problem, which provides the aforementioned guarantees. We derive our algorithms from this theory. We show in experiments, that with an overhead of just 50 fps, we can increase the number of correct detections and decrease the overall computational cost compared to running a modern single-frame detector.
Tasks	Object Detection
Published	2019-08-29
URL	https://arxiv.org/abs/1908.11092v1
PDF	https://arxiv.org/pdf/1908.11092v1.pdf
PWC	https://paperswithcode.com/paper/minimum-delay-object-detection-from-video
Repo	https://github.com/donglao/mindelay
Framework	none

Multiplayer AlphaZero


Title	Multiplayer AlphaZero
Authors	Nick Petosa, Tucker Balch
Abstract	The AlphaZero algorithm has achieved superhuman performance in two-player, deterministic, zero-sum games where perfect information of the game state is available. This success has been demonstrated in Chess, Shogi, and Go where learning occurs solely through self-play. Many real-world applications (e.g., equity trading) require the consideration of a multiplayer environment. In this work, we suggest novel modifications of the AlphaZero algorithm to support multiplayer environments, and evaluate the approach in two simple 3-player games. Our experiments show that multiplayer AlphaZero learns successfully and consistently outperforms a competing approach: Monte Carlo tree search. These results suggest that our modified AlphaZero can learn effective strategies in multiplayer game scenarios. Our work supports the use of AlphaZero in multiplayer games and suggests future research for more complex environments.
Tasks
Published	2019-10-29
URL	https://arxiv.org/abs/1910.13012v3
PDF	https://arxiv.org/pdf/1910.13012v3.pdf
PWC	https://paperswithcode.com/paper/multiplayer-alphazero
Repo	https://github.com/petosa/multiplayer-alphazero
Framework	pytorch